Databases

• pgvector vs Pinecone: You Probably Don't Need a Separate Vector Database

  Every time someone starts building a RAG pipeline, the same question will come up: do I need a “real” vector database like Pinecone, or can I just use pgvector with the Postgres I already have?

  I can imagine teams agonizing over this decision for weeks. So maybe this will save you some time?

  The Case for Staying Put

  If you already have a PostgreSQL instance in your stack, adding pgvector is almost always the right first move.

  You manage one stateful service instead of two. Your existing backup strategy, monitoring, and security all stay the same. Your vector embeddings live next to your metadata, so you get ACID compliance and standard SQL joins. No syncing between two data stores. No eventual consistency headaches.

  Performance? From what I found, for datasets under a few million vectors, pgvector with HNSW indexes is fast. Really fast. It satisfies the latency requirements of most applications without breaking a sweat.

  And you’re not paying for another SaaS subscription…

  When Pinecone Actually Makes Sense

  Pinecone is a purpose-built vector database designed for high-dimensional data at massive scale. It’s serverless and fully managed.

  If you’re dealing with hundreds of millions or billions of vectors, a specialized engine handles memory and disk I/O for similarity searches more efficiently than Postgres can. Pinecone also gives you native namespace support, metadata filtering optimized for vector search, and live index updates that are faster than re-indexing a large Postgres table.

  Those are real advantages. At a certain scale.

  The Decision Is Simpler Than You Think

  Stay with Postgres + pgvector if:

  • You want to minimize infra sprawl and moving parts
  • Your vector dataset is under 5 to 10 million records
  • You rely on relational joins between vectors and other business data
  • You have existing observability and DBA expertise for Postgres

  Consider Pinecone if:

  • Your Postgres instance needs massive, expensive vertical scaling just to keep the vector index in memory
  • You don’t want to tune HNSW parameters, mmap settings, or vacuuming schedules for large vector tables
  • You need sub-millisecond similarity search at a scale where Postgres starts to struggle

  That is what I would use to make that decision.

  Most teams are probably nowhere near the scale where Pinecone becomes necessary. They have a few hundred thousand vectors, maybe a million or two. Postgres handles that without flinching. Adding a separate managed vector database at that point is just adding operational complexity for no measurable benefit.

  The trap is thinking you need to “plan ahead” for scale you don’t have yet. You can always migrate later if you actually hit the ceiling. Moving from pgvector to Pinecone is a well-documented path. But moving from two services back to one because you overengineered your stack? That’s a conversation nobody wants to have.

  Start with what you have. Add complexity when the numbers force you to, not when a vendor’s marketing page makes you nervous.

  I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].

  Apr 25, 2026 / DevOps / AI / Programming / Databases

  Enjoyed this?

• Everything Is Eventually a Database Problem

  I think there’s a saying that goes something like “code is ephemeral, but data is forever.” That’s never been more true than right now. Code is easier than ever to create, anyone can spin up a working app with an AI agent and minimal experience. But your data structure? That’s the thing that sticks around and haunts you.

  Data modeling is one of those topics that doesn’t get enough attention, especially given how critical it is. You need to understand how your data is stored, how to structure it, and what tradeoffs you’re making in how you access it. Get it right early, and your code stays elegant and straightforward. Get it wrong, and your codebase becomes a forever series of workarounds…

  Microservices Won’t Save You

  For teams moving from a monolith to microservices, if the data stays tightly coupled, you don’t really have microservices; you have a distributed monolith with extra network hops.

  Yes, data can be coupled just like code can be coupled. If all your different services are still hitting the same database with the same schema, you have a problem. You need separate data structures for your services, not a monolithic architecture hiding behind a microservices facade.

  The Caching Trap

  So what happens when you have a lot of data and your queries get slow? You’ve done all the easy stuff; optimized queries, added indexes, followed best practices. But things are still slow.

  Every senior engineer’s first instinct is the same: “Let’s add Redis in front of it.” Or “more read replicas.” And sure, that works, but you have just added complexity and now you have to deal with cache invalidation.

  What happens when you have stale data? How do you recache current data, and when does that happen?

  Are you caching on the browser side too? Understanding where data can be cached and how to invalidate it is another genuinely difficult problem to solve. You’re just trading one set of problems for a different set of problems.

  You Can’t Predict Every Future Question

  If you’re selling things on the internet, chances are, you will care about event sourcing at some point. A lot of interesting business problems don’t care about the current state of a user, they care about the intent and history. So how you store intent and history is probably different from your ACID-compliant Postgres table that you’ve worked hard to normalize.

  You can get your data structure perfect for displaying products and processing sales, then run into a completely new set of requirements that changes everything about how your data needs to be structured.

  It’s genuinely hard to foresee all the potential questions you’ll need to answer in the future.

  Why This Matters Now

  Everything you do on a computer stores data somewhere, it’s just a matter of persistence.

  Which is why; everything software-related is eventually a database problem.

  Data modeling isn’t glamorous, but getting it right is the difference between a system that scales gracefully and one that fights you every step of the way.

  Mar 16, 2026 / Programming / Databases / Architecture

  Enjoyed this?

• Just discovered DB Pro, a new desktop app for SQLite and LibSQL databases. Looks pretty promising. Meanwhile, I’m still waiting for DataGrip to get proper LibSQL support. Come on JetBrains, just give me Turso already!

  Jan 6, 2026 / Sqlite / Databases / Tools