Run Your Whole Agent Stack on a $5 Box

Sat, 01 Aug 2026 10:00:00 -0500

I SSH’d into my home server this afternoon and ran docker stats on the memory layer that every one of my coding agent sessions talks to. Here’s what came back:

mem0-qdrant    28.09MiB / 60.75GiB    2.13%
mem0-neo4j     612.7MiB / 60.75GiB    0.77%

640 megabytes. Vector store and graph store, both up for three weeks straight, serving every remember and recall call my agents make. The entire persistent memory for my AI tooling uses less RAM than one Chrome tab with Figma open.

So let’s talk about why you’re paying a monthly subscription for this.

Local-First, Not Local-Only

I want to be precise here, because “self-hosted AI” has become a phrase people use to mean nine different things.

My setup is local-first, not local-only. The state lives on my hardware. The memories, the embeddings, the graph relationships, everything my agents have learned about my projects, all of it sits on a box I own, in a Docker volume I can tar and carry away. Nobody can deprecate it, price-hike it, or sunset it.

The inference does not. My embedder points at Mistral’s managed API. I’ll get to why, and how to swap it, but I’m not going to pretend otherwise in a post about self-hosting.

That embedder is the only thing that leaves my network, and only when something actually gets embedded, so when writing a memory and searching for one. Listing, deleting, and every graph operation are local with zero API calls.

State is what you can’t get back. Compute is a commodity you rent by the token. Losing access to an API means switching providers. Losing two years of accumulated project context means starting over.

The Four Pieces

Qdrant is semantic search. When I ask what it remembers about my package manager preferences, Qdrant turns that into a similarity query and hands back the relevant memories. It’s Rust, it’s fast, and at 28MB resident it’s essentially free to run. One gotcha: vector dimensions are fixed when the collection is created. Swap embedding models and you need a new collection, not a migration. I learned that the annoying way.

Neo4j is the graph store. Vectors are great at “find me things that sound like this” and bad at “what depends on what.” The graph holds explicit subject-predicate-object facts, so project X built_with Python 3.13 is a traversable edge instead of a fuzzy match. It’s the heavy one at 613MB, but it’s a JVM, so that’s mostly heap floor rather than working set. If you’re squeezing onto the smallest possible VPS, interrogate this one first.

mem0 is the orchestration on top: what gets extracted from a conversation, what gets deduped against existing memories, what gets written where. That’s the difference between a database and a memory system.

The MCP server is what makes any of it useful. A small Go binary that speaks Model Context Protocol over stdio to Claude Code, exposing eight tools: remember, recall, list_memories, forget, memory_stats, add_relation, recall_related, forget_relation.

The topology is deliberately boring:

Mac                                    Home server
┌──────────────┐   ┌──────────┐        ┌─────────────────┐
│ Claude Code  │◄─►│ mem0-mcp │  LAN   │ Qdrant + Neo4j  │
│              │   │ (Go)     │───────►│                 │
└──────────────┘   └──────────┘        └─────────────────┘
       stdio                    HTTP + bolt

Client binary on my laptop, containers on a box. No cloud in the middle, no account, no dashboard, no seat license.

The Ansible Role Is the Whole Argument

Anyone can docker compose up a stack once. That’s a weekend, not infrastructure. What makes this real is that it’s a role in a repo, and rebuilding it on a fresh box is one command:

ansible-playbook -i common_hosts home.yml --tags mem0

That role does the unglamorous work:

Installs a read-only deploy key scoped to exactly one repo, with an SSH Host alias so it can’t collide with my personal GitHub key
Clones and updates the source at a pinned branch
Templates a .env with secrets pulled from Ansible Vault, no_log: true so nothing leaks into terminal output on a --diff run
Brings up the compose stack with remove_orphans: true, so when I dropped a service upstream, the stale container went with it instead of lingering forever

Be careful with your secrets and how you are creating your .env files!

The Honest Part About the API Key

I self-host the state and rent the inference. Two reasons.

The first reason is speed. I ran embeddings locally before this, on CPU, and it was miserable: roughly 87 seconds to embed 32 memories, against about 2 seconds through a hosted API. That is a 45x difference on an operation sitting directly in the path of every remember and recall. A good model on a CPU is still a slow model, and this was never a quality problem.

The second is that embeddings have gotten cheap enough that not worth the time to setup your own embedding service. Mistral charges $0.10 per million tokens for mistral-embed. Google’s gemini-embedding-001 is $0.15 per million, halved on their batch API. Both are good models. Both bill you.

Cloudflare is the worth knowing about if you’d rather not pay at all. Workers AI includes 10,000 neurons per day free, on the free plan as well as the paid one. Neurons are their normalized compute unit, and bge-m3 costs 1,075 of them per million input tokens — so that daily allowance is roughly nine million tokens a day, at no cost. Past it you’re at $0.012 per million, which is an order of magnitude under the paid competition. For a personal memory layer, nine million tokens a day is not a trial. It’s just free.

One detail if you’re swapping: bge-m3 emits 1024-dimension vectors, the same as mistral-embed. Go back to that Qdrant gotcha — matching dimensions means your existing collection still works. Mismatched ones mean starting over.

And the escape hatch is already built. The env vars in my role are TEI_BASE_URL, TEI_MODEL, TEI_DIMENSIONS. Generic OpenAI-compatible embedder knobs, named after Text Embeddings Inference for historical reasons and pointed at Mistral today. Aim them at a self-hosted TEI container, at Ollama, at anything speaking that shape, and the rest of the stack doesn’t notice.

That’s what local-first buys you. Not purity. Optionality.

So, the $5 Box

My server has 60GB of RAM, which is absurd overkill and exists because it does a dozen other things. The stack itself measured 640MB with three weeks of uptime, essentially zero CPU at idle.

That fits comfortably on a small cloud VPS in the few-dollars-a-month range. Check current pricing yourself rather than trusting a number in a blog post, but the shape is: a 2 vCPU / 4GB instance from Hetzner or similar costs less per month than one seat of most AI memory SaaS products, and you get to run everything else on it too.

Your real constraint is RAM, specifically Neo4j’s JVM floor. On a 1GB instance you’d be fighting it. At 2GB you’re fine. At 4GB you’ll forget it’s running.

Why I Care

The indie web ethos is about noticing that renting your identity from a platform means the platform decides what happens to it.

We’re about to make the same mistake with agent memory, except worse, because the thing being accumulated this time is a working model of how you think and what you’re building. Every “our AI remembers you across sessions” product is a proposal that you deposit that into someone else’s database and hope the pricing page stays reasonable.

Qdrant is Apache 2.0. Neo4j Community is GPL. Docker Compose is a YAML file. Ansible is idempotent YAML. Nothing in this stack is exotic. The barrier to owning your agent memory is an afternoon and 640 megabytes.

Not everyone needs this, and I’m not going to pretend a solo dev with three side projects is being exploited by a $20 subscription. But if you’re accumulating context you’d be genuinely sad to lose, the math changes. Own the state, rent the compute, and keep the role in version control so the whole thing is reproducible on a box you haven’t bought yet.

Moving the embedder onto Cloudflare’s free tier is next on my list, what’s on yours?

I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @logan@llbbl.blog.