Dependencies
-
A Dependency Bump Changed the Shape of My Vectors
The commit is titled
feat(deps): consolidate dependency updates. Sounds like a Tuesday. Bump some carets, watch CI go green, move on.What it actually did was cut every embedding in the database in half and rename the table they live in.
The one line that did it
- "@logan/libsql-search": "jsr:^0.1.3", + "@logan/libsql-search": "jsr:^0.7.1",That’s a jump across 14 published versions. I checked the JSR registry rather than trusting my memory of it: between 0.1.3 and 0.7.1 there are 0.1.4, 0.1.5, 0.1.6, 0.2.0 through 0.2.4, 0.3.0, 0.4.0, 0.5.0, 0.6.0, and 0.7.0. The package is at 0.11.1 now, so I was already well behind when I started.
Everything downstream of that line is the interesting part, and none of it appears in the commit message.
The embedding runtime got swapped underneath me
The library generates embeddings locally, no API key required. In 0.1.x it did that with
@xenova/transformers. In 0.7.x it doesn’t.I didn’t take the changelog’s word for this. I checked what’s actually installed:
$ grep -c "xenova" pnpm-lock.yaml 0 $ grep -n "onnxruntime" pnpm-lock.yaml | head -3 2455: [email protected]: 2461: [email protected]:Zero references to the old runtime.
[email protected]in its place, and nothing undernode_modules/@xenovaon disk. The inference engine changed.There’s a second place that tell shows up, and it’s one I’d never thought to read as a signal before. pnpm keeps an allowlist of packages permitted to run install scripts, because native modules need to compile:
-onlyBuiltDependencies: - - sharp - - "@xenova/transformers" +allowBuilds: + esbuild: true + onnxruntime-node: true + protobufjs: true + sharp: trueTwo things happened in that hunk. pnpm 10 to 11 renamed the field, which is why it’s a rewrite rather than an edit. And the membership changed:
@xenova/transformersout,onnxruntime-nodeandprotobufjsin. Your native build allowlist is a map of every package that compiles C++ on your machine. When entries appear and disappear there, something real moved.768 became 384
This is the part with actual consequences.
- embedding F32_BLOB(768),The new local model emits 384-dimension vectors instead of 768. Those are not interchangeable. A 384-float vector cannot be compared against a 768-float vector, so every embedding computed before this commit became unusable the moment it landed.
Which is why the table name is now pinned to the dimension:
export const SEARCH_TABLE_NAME = 'articles_local_384'; export const LOCAL_EMBEDDING_DIMENSIONS = 384;That two-line file is the best decision in the whole commit. The old code had
768written intoscripts/init-db.tsas raw DDL, and separately into the indexer, and separately into the search query. Three places, no shared constant, and nothing that would fail loudly if one drifted. Now the dimension lives in one module and the table it belongs to carries the number in its name, so a mismatched index can’t quietly overwrite a good one. It creates a new table instead.The hand-written schema went away with it.
scripts/init-db.tslost 37 lines and gained 13: aCREATE TABLE, alibsql_vector_idxindex, two more indexes, and their log lines all collapsed into one call.await createTable(client, SEARCH_TABLE_NAME, LOCAL_EMBEDDING_DIMENSIONS);I’m of two minds about that. Less schema I maintain is good. But the vector index definition is now something I have to go read the library source to see, and if it changes in 0.12 I won’t find out from my own repo.
Also smuggled in
Since I was already reading the diff instead of the title:
- TypeScript 6 to 7. A major version, in a commit that says “dependency updates.”
tsc --versionconfirms 7.0.2 is what’s installed. baseUrldeleted from tsconfig.json. One line, no comment. Thepathsmapping for@/*still works because modern resolution doesn’t need it, butignoreDeprecations: "6.0"is still sitting in there pointing at the previous major.- pnpm 10.20.0 to 11.23.0, and Node’s floor moved from 22.12.0 to 22.13.0.
The one that actually left
I removed
winstonin the same window, and the skeptical move is to assume it’s still there. Optional and transitive dependencies survive manifest deletions constantly, so “removed from package.json” and “gone” are different claims.This time they matched. Zero lockfile references, nothing on disk, no imports anywhere in source. The reason is that
logan-loggerwent 1.1.16 to 2.5.1, and version 2.5.1’s ownpackage.jsondeclares no dependencies at all. Winston was its transport layer, and dropping it took the whole subtree out.So the removal was clean. I’m mentioning it because I expected it not to be, and reporting only the findings that confirm your suspicions is how you end up with a blog full of nonsense.
The caret did its job
Worth being fair about what went wrong here, because it isn’t semver.
^0.1.3does not resolve to 0.7.1. For 0.x releases the caret pins to the minor, so^0.1.3stays inside0.1.x. Getting to 0.7.1 required editing that string by hand. No install silently pulled a breaking change in, and no lockfile refresh could have.The upgrade was deliberate. The migration note even made it into the repo’s
CLAUDE.md, saying to rundb:initbeforeindexwhen coming from the legacy 768-dimension table.What failed is the commit message.
feat(deps): consolidate dependency updatesis a true statement that hides a data migration, a runtime swap, and a compiler major. Six months from now, bisecting a search quality regression, that title is going to send me straight past the commit that caused it.Things I’m doing differently:
git diff <base>..HEAD -- pnpm-lock.yamlbefore believing any manifest. Additions, removals, and version changes are three separate stories.- Read the native build allowlist as a diff.
allowBuildsin pnpm 11,onlyBuiltDependenciesin 10. Changes there mean a compiled dependency moved. - Put the dimension in the table name.
articles_local_384cannot be corrupted by a 768-dimension writer.articlescan. - Don’t bundle a compiler major with anything. TypeScript 6 to 7 deserved its own commit and its own revert point.
- If a dep bump invalidates stored data, the commit message says so.
feat(deps)!:with aBREAKING CHANGEfooter costs nothing and is the only thing future-me will actually see.
One thing I have not done and won’t pretend otherwise: I haven’t measured whether 384-dimension search results are worse than the 768-dimension ones were. Half the floats for a corpus this small is likely a fine trade, and it’s faster, but “likely fine” is a guess and I should go get a number.
Sources
- @logan/libsql-search on JSR — version history, if you want to count the gap yourself
- ONNX Runtime for Node.js — the inference engine that replaced the old transformers package
- pnpm settings reference —
allowBuilds, and what it replaced in pnpm 10 - Turso: AI and embeddings —
F32_BLOBandlibsql_vector_idx, the storage side of all this - semantic-docs — the repo, and PR #88 is the commit in question
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
- TypeScript 6 to 7. A major version, in a commit that says “dependency updates.”
-
I Gave Install-Script Permission to a Package I Don't Have
There’s exactly one file in one of my wiki projects that’s a security control rather than a config file, and I hadn’t read it in months.
pnpm 11 removed
onlyBuiltDependenciesalong with four related settings and replaced all of them with a singleallowBuildsmap, described in the docs as “a map of package matchers to explicitly allow (true) or disallow (false) script execution.” A postinstall script is arbitrary code running on your machine at install time with your permissions, so the list of packages allowed to have one is the shortest and highest-consequence list in the repo.Mine has six entries. Two
true, fourfalse. Here’s thetruehalf:allowBuilds: sharp: true # native libvips bindings (image processing) "@xenova/transformers": true # native ML runtimeAnd here’s the count:
$ grep -c 'transformers' pnpm-lock.yaml 0 $ grep -rn 'xenova' --include='*.json' --include='*.yaml' --include='*.ts' . | grep -v node_modules pnpm-workspace.yaml:28: "@xenova/transformers": true # native ML runtimeZero references in the lockfile. One reference in the entire repository, and it’s the line granting the permission.
This project used to run embeddings locally. That approach is gone, search goes through a hosted retrieval service now, and the project’s own direction notes say not to reintroduce local embeddings. The dependency left. The standing permission to execute native build scripts on my machine stayed behind, waiting for a package that’s never going to be installed.
A dead grant isn’t a vulnerability, it’s a broken instrument
Nobody’s exploiting this. The package isn’t in the tree, so nothing runs. I want to be clear about that before anyone gets excited.
What it costs me is the ability to trust the file. An allowlist works as a control only if every entry means I read this package’s install script and I accept it. The moment one entry instead means this was true in a previous version of the project, the list becomes a record of the past, and the next person to open it has no way to tell the two kinds of entry apart without re-deriving all six from the lockfile. The next person is me, in four months, at which point I will absolutely assume the file is current.
This is the failure mode of every allowlist I’ve kept. They only grow. Removing an entry requires noticing that something left, and nothing tells you when a dependency stops existing.
The other
trueentry has the opposite problemsharpis granted a native build.sharpis also not in mypackage.json. It shows up anyway:$ pnpm why sharp [email protected] └─┬ [email protected] ├─┬ @astrojs/[email protected] │ └── <package> (dependencies) └── <package> (dependencies)Astro declares it as an optional dependency. I wrote about that mechanic in a different repo a few days ago, so I’ll skip the re-explanation. What’s new to me here is that the same workspace file also pins it:
overrides: vite: ^8.2.2 sharp: ^0.35.3The comment sitting above that block warns that several of these pins are CVE remediations for transitive dependencies, and not to relax a bound without checking the advisory it was added for. That’s a good note. It’s also attached to a package I never asked for, whose version I control only because I reached into the resolver and overrode somebody else’s optional dependency.
The
falseentries turned out to be the good news"@prisma/engines": false prisma: false better-sqlite3: false esbuild: falseThese four aren’t a new restriction, and it took me a minute to work out why they’re written down at all.
Under pnpm 10,
onlyBuiltDependencieswas an exclusive allowlist. Anything not on it got skipped. All four of these were already being skipped, silently, for as long as this project has existed. pnpm 11’sstrictDepBuildsrefuses to skip quietly, and exits non-zero when a dependency has an unreviewed build script.So the upgrade changed no behavior at all. It changed whether the behavior was visible, and converted four invisible skips into four decisions I had to write down and sign. That’s the good version of a breaking change, and it’s the reason the dead
@xenova/transformersentry was sitting right there for me to trip over.One more thing, since I had the shell open
$ grep -c '0.34.5' pnpm-lock.yaml 0 $ du -sh node_modules/.pnpm/@[email protected] 15M[email protected]has zero references in the current lockfile and is still on disk, next to 15M of libvips binaries compiled for it, which in turn sit next to the 17M of libvips 1.3.2 that the current resolution actually uses. The override moved from^0.34.4to^0.35.3in a dependency sweep last week, and the old native payload never left. That’spnpm store pruneterritory rather than a correctness bug, but it’s 15M of compiled image-processing code on a machine whose project doesn’t declare an image-processing dependency.What I’m changing
- Reconcile the allowlist against the lockfile in CI. Every key in
allowBuildsshould resolve to something inpnpm-lock.yaml, or the build should complain. It’s maybe ten lines of test and I don’t have it. - Say why, next to every entry. Two of my six have a reason comment. The reason is worth more than the package name, because the reason is the thing that expires.
- Re-read the file whenever an override moves. The pins and the build grants describe the same dependency graph, and they drifted apart without a word.
The security value of an allowlist lives in the reviewing, not in the file. I’ve been maintaining the file.
Sources
- pnpm build settings —
allowBuildsandstrictDepBuilds, plus the migration table from the five settings pnpm 11 removed - pnpm 11.0 release notes — the removal of
onlyBuiltDependenciesand friends - pnpm settings index — the full
pnpm-workspace.yamlsurface
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
- Reconcile the allowlist against the lockfile in CI. Every key in
-
I Removed Three Dependencies. One of Them Actually Left.
In forty-six minutes one morning I opened three pull requests, and every one of them deleted a dependency.
The commits are timestamped, so I can be precise about it:
- 07:33 replace Sharp transforms with
Bun.Image - 07:55 enforce signed upload content type (the AWS SDK removal that didn’t happen)
- 08:19 adopt
Bun.YAMLfor sync
Two merged with the dependency gone from
package.json. One I abandoned on purpose. That’s a decent morning, and it’s how I described it to myself at the time.Then I went back and looked at the lockfile.
Only one of those packages actually left.
Sharp Is Still Sitting in node_modules Right Now
The image one looked like the clean win. My media pipeline generates thumbnail, medium, and large variants, plus WebP versions, and it used Sharp to do it. Bun 1.4 ships a native
Bun.Image, so I rewrotesrc/lib/media.tsonto it and pulled"sharp": "^0.35.3"out of my dependencies.The benchmark was encouraging. Same workload, 2400×1600 inputs, separate processes,
/usr/bin/time -lon macOS:Input Sharp peak RSS Bun.Image peak RSS Elapsed JPEG 218 MB 75 MB 224ms → 207ms PNG 185 MB 80 MB 389ms → 549ms WebP 177 MB 77 MB 238ms → 180ms Memory is not close. Roughly a third of the footprint across all three formats. Speed is a wash, and PNG got noticeably worse, about 40% slower. The encoded output isn’t byte-identical either: JPEG came out 9% larger, PNG 2% smaller, WebP 3% larger. Fine for my purposes, but it’s a re-encode, not a port.
One caveat I want to be honest about, because it undercuts the comparison: Sharp wouldn’t load under Bun 1.4.0 on this machine at all. It failed on a missing
libvips-cpp.8.18.3.dylib. Undernode -e 'import("sharp")'it imported fine. So that table is Sharp-under-Node against Bun.Image-under-Bun, which means it’s measuring two runtimes, not two image libraries.Anyway. I removed Sharp, the tests passed, I merged it.
Sharp is still installed. Here it is:
$ grep -c '"@img/sharp-\|"sharp"' bun.lock 27 $ ls -d node_modules/sharp node_modules/sharpAstro 7 declares Sharp as an optional dependency for its own image integration. My lockfile says so in as many words:
"[email protected]" ... "optionalDependencies": { "sharp": "0.35.2" }So when I deleted my direct dependency, the resolver didn’t drop Sharp. It just stopped hearing my opinion about which version to use, and fell back to Astro’s. Sharp went from 0.35.3 to 0.35.2. My dependency removal was, in lockfile terms, a downgrade.
The One That Actually Left
The YAML change is the boring one and it’s the only real removal.
My blog sync script parsed frontmatter with the
yamlpackage. Bun 1.4 hasBun.YAMLwithparseandstringify, which is the entire surface I was using:const data = Bun.YAML.parse(frontmatter);That’s it.
"yaml": "^2.9.0"came out ofpackage.json, one entry came out ofbun.lock, and nothing else in the tree wanted it. Gone.No benchmark for this one. I didn’t run one, because a dependency that does one thing I can do with a builtin doesn’t need a performance argument to justify deleting it.
The Third One I Talked Myself Out Of
The AWS SDK is the one I wrote about separately. Short version: Bun’s native
S3Clientpresigns URLs about 32x faster and would take 25 packages out of the lockfile, and I kept the AWS SDK anyway, because Bun 1.4 can’t sendCache-Controlon writes and can’t forceContent-Typeinto a presigned URL’s signed headers. Faster, smaller, and not behaviorally equivalent.That one at least failed honestly. I evaluated it, wrote down why not, and the packages are still there because I decided they should be.
Three Words That Mean Three Different Things
Here’s what I actually learned, and it’s a vocabulary problem more than a technical one. “I removed a dependency” turns out to describe at least three unrelated states:
- It’s out of my
package.json. Sharp, yaml. This is the one people mean when they say it, and it’s the weakest of the three. - It’s out of the lockfile and off the disk. Only yaml. Sharp is still there, 27 entries, physically present in
node_modules. - It’s out of
trustedDependencies. This is the one I undersold at the time.esbuildis now the only package in my project allowed to run install lifecycle scripts. Sharp used to be in that list and isn’t anymore.
Number three is the one that actually changed my exposure. Sharp is still on disk, but nothing in my code calls it, and it no longer gets to execute arbitrary code at install time. Number two never happened. Number one is mostly bookkeeping.
I would have told you I removed two dependencies that morning. The lockfile says I removed one, downgraded another, and revoked install-script trust from a package that’s still sitting right there. All three of those are fine outcomes. They’re just not the same outcome, and I had been filing them under one word.
Check your lockfile after you delete something. Transitive optional dependencies do not care what your
package.jsonsays.Sources
- Bun.Image runtime documentation — the image API that replaced Sharp
- Bun.YAML runtime documentation —
parseandstringify, nothing else - Bun 1.4 release announcement — released 2026-08-20
- Astro images guide — Astro’s built-in image service, which is why Sharp stays
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
- 07:33 replace Sharp transforms with
-
Defending Your Node Modules: Security Tools and When to Rewrite Dependencies
This week I’ve been on a bit of a JavaScript kick; writing about why Vitest beats Jest, comparing package managers, diving into Svelte 5. But there’s one topic that we shouldn’t forget: security.
node_modulesis the black hole directory that we all joke about and pretend is fine.Let’s talk about how to actually defend against problems lurking in those deep depths when a rewrite might make sense.
The Security Toolkit You Actually Need
You’re going to need a mix of tools to both detect bad code and prevent it from running. No single tool covers everything (or should), so here are some options to consider:
Socket does behavioral analysis on packages. It looks at what the code is actually doing. Is it accessing the network? Reading environment variables? Running install scripts? These are the sketchy behaviors that signal a compromised or malicious package. Socket is great at catching supply chain attacks that traditional vulnerability scanners miss entirely.
Snyk handles vulnerability scanning. It checks your entire dependency tree against a massive database of known vulnerabilities and is really good at finding transitive problems, those vulnerabilities buried three or four levels deep in your dependency chain that you’d never find manually.
LavaMoat takes a different approach. It creates a runtime policy that prevents libraries from doing things they shouldn’t be doing, like making network requests when they’re supposed to be a string formatting utility. Think of it as a permissions system for your dependencies.
And then there’s Dependabot from GitHub, which automatically opens pull requests to update vulnerable dependencies. This is honestly the minimum of what you should be doing. If you’re not running Dependabot like service, start now.
Each of these tools catches different things. Socket finds malicious behavior, Snyk finds known vulnerabilities, LavaMoat enforces runtime boundaries, and Dependabot keeps things updated. Together, they give you solid coverage.
When to Vendor or Rewrite a Dependency
Now let’s talk about something I think more developers should be doing: auditing your dependencies and asking when a rewrite makes sense.
With AI tools available now, this has become incredibly practical. Here’s when I think you should seriously consider replacing a dependency with your own code:
-
You’re using 1% of the library. If you imported a massive package just to use one function, you don’t need the whole thing. Have your AI tool write a custom function that does exactly what you need. You shouldn’t be importing a huge library for a single utility. It’s … ahhh, well, stupide.
-
It’s a simple helper. Things like
isEven,leftPad, or a basic string formatter. AI can write these in seconds, and you eliminate an entire dependency from your tree. Fewer dependencies means a smaller attack surface. -
The package is abandoned. The last update was years ago, there’s a pile of open issues, and nobody’s home. You’re better off asking your LLM to rewrite the functionality for your specific project. Own the code yourself instead of depending on something that’s collecting dust.
When You Should Absolutely NOT Rewrite
This is just as important. Some things should stay as battle-tested community libraries, no matter how good your AI tools are:
-
Cryptography, authentication, and authorization. It would be incredibly foolish to try to rewrite bcrypt or roll your own JWT validation. These libraries have been audited, attacked, and hardened over years. Use them.
-
Complex parsers with extensive rule sets. A markdown parser, for example, has a ton of edge cases and rules that need to be exactly right. You don’t want to accidentally ship your own flavor of markdown. Same goes for HTML sanitizers, getting sanitization wrong means introducing XSS vulnerabilities. Trust the community libraries here.
-
Date and time math. Time zones are a deceptively hard problem in programming. Don’t rewrite
date-fnsordayjs. Just don’t. -
Libraries that wrap external APIs. If something integrates with Stripe, AWS, or any API that changes frequently, you do not want to maintain that yourself. The official SDK maintainers track API changes so you don’t have to. Just, no and thank you.
The pattern is pretty clear: if getting it wrong has security implications or if the domain is genuinely complex with lots of edge cases, use the established library. If it’s simple utility code or you’re barely using the package, consider a rewrite.
A Fun Side Project Idea
If you’re looking for yet another side project (YASP), that is one that would be a super useful CLI tool. I’d probably reach for Go and build a TUI tool that scans your
node_modulesand generates a list of rewrite recommendations.I think that’d be a really fun build, and honestly something the JavaScript ecosystem could use.
-