Agentic
-
Five Places RAG Shows Up in Agentic Systems
Ask most people what RAG is and they’ll tell you it’s semantic document search. You chunk up a pile of text, embed it, stuff it in a vector database, and pull the relevant bits back at query time. That’s the textbook example, and it’s a good one. But retrieval augmented generation does a lot more work inside agentic systems than “search the docs,” and I think it’s worth walking through where it actually shows up. So let’s talk about it.
1. High-Precision Semantic Search
This is the one everybody knows, so let’s get it out of the way first. You take raw text and convert it into high-dimensional vectors, where distance corresponds to conceptual similarity. Store those vectors, index them, and you can look things up by meaning instead of exact keywords.
The interesting part is how you index them, because the algorithm you pick is a real tradeoff.
HNSW (Hierarchical Navigable Small World) builds a multi-layer graph. The upper layers have long-distance links for fast routing across the space, and the lower layers have short-distance links for local search. You get low query latency and near-perfect recall. The catch is memory. It wants to keep the raw vectors around, so the footprint gets big.
IVF-PQ (Inverted File with Product Quantization) goes the other direction. It partitions the vector space into cells using k-means clustering, then compresses the high-dimensional vectors into compact quantized codes. Partition, then squish. That cuts memory consumption dramatically, which makes it a great fit for massive datasets with millions of vectors. The price you pay is recall accuracy, since all that compression throws away detail, and rebuilds get slower when you add new data.
Neither one is “correct.” You pick based on whether you’re optimizing for recall or for fitting the index in memory.
2. Tool Selection (RATS)
Here’s where it gets less obvious. Picture a CLI harness. As your developer toolkit grows, your agent slowly gets “equipped” with dozens or hundreds of possible actions. APIs, database calls, helpers, command executors. At some point you’ve just overloaded the thing with too much stuff.
Three bad things happen when you do that:
- Tool space interference (TSI). Overlapping tool descriptions confuse the agent, and it calls the wrong one.
- Context window saturation. Every tool schema, whether it’s JSON or Markdown, eats thousands of tokens. Pile up enough MCP servers and custom skills and you’re soaking the context window, which drives up cost and latency.
- The lost-middle problem. Models tend to ignore tools and instructions buried in the middle of a very long prompt.
Retrieval augmented tool selection fixes this by treating your tools like a corpus. Instead of dumping every schema into the prompt, you retrieve only the handful of tools relevant to the current task. The agent sees a short, sharp menu instead of the entire pantry.
3. Dynamic Few-Shot Prompting
Few-shot prompting is a reliable way to enforce formatting constraints like a strict JSON schema, teach reasoning paradigms like chain of thought, or train an agent on error recovery. The problem is that static examples baked into a prompt are a guess. They might not match the task in front of you.
RAG lets you select the examples at runtime. You curate a database of gold-standard trajectories, each one pairing a specific query or error case with the correct step-by-step reasoning, tool calls, and final output that solved it. When a new task comes in, you search that database using the user’s intent, grab the top few most similar past trajectories, and prepend them to the system instructions.
So the agent always gets examples that actually resemble what it’s being asked to do, instead of whatever examples you happened to hardcode three weeks ago.
4. Long-Term Agent Memory
Work directly with a model and it forgets everything the moment the session closes. Your preferences, your corrections, the choices you already made. Gone. For an agent to be useful, it needs persistent memory across sessions. I’ve written about this before.
One system here is mem0, which uses a hybrid RAG architecture to persist state. It does asynchronous fact extraction, conflict resolution when new information contradicts old, and grounds the retrieved memories back into the prompt. The retrieval layer is what lets the agent surface the right past fact at the right time instead of replaying the entire history.
5. Evaluation and Test Harnesses
Testing AI in production is hard because the outputs aren’t deterministic. So you build evaluation harnesses that run your agent across hundreds of test cases, and RAG turns out to be a quiet workhorse in that loop.
Two ways it helps:
- Diffing the test suite. Running every eval on every pull request is slow and expensive. Instead, query a vector index of your test suite using the git diff as the query, and run only the cases relevant to the code you actually touched.
- Semantic assertions. Exact string matching is useless when you’re verifying something like an agent’s summary. Instead, the harness retrieves historic successful runs and uses vector similarity to ask whether the new output matches the intent and tone of the target, rather than matching it character for character.
None of this replaces the document-search version of RAG. It’s the same core trick, embed things, retrieve by similarity, ground the result, pointed at different problems: which tool, which example, which memory, which test. Once you start seeing retrieval as a general-purpose way to feed an agent the relevant slice of a much bigger pile, it shows up everywhere. I’ll probably keep finding more.
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
Sources
- Yury A. Malkov and Dmitry A. Yashunin, “Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs” (arXiv:1603.09320, 2016) — details on the multi-layer graph architecture and logarithmic complexity of the HNSW index.
- Hervé Jégou, Matthijs Douze, and Cordelia Schmid, “Product Quantization for Nearest Neighbor Search” (IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011) — explains compressing high-dimensional vectors and combining product quantization with inverted file indexing (IVF-PQ).
- mem0ai, “mem0: The Memory Layer for Personalized AI” (GitHub) — documentation and codebase for the persistent, self-improving memory layer for AI agents.
- Mostafa Ibrahim, “Agentic RAG vs Classic RAG: From a Pipeline to a Control Loop” (Towards Data Science, March 2026) — commentary on the shift from static document retrieval to agentic control loops and its associated system failure modes.
- Microsoft Research, “Tool-space interference in the MCP era: Designing for agent compatibility at scale” — the tool-space interference (TSI) problem from section 2, where overlapping tool descriptions degrade agent tool selection.
- rewire.it, “Dynamic Tool Allocation for AI Agents (The RATS Pattern)” — the retrieval-augmented tool selection (RATS) pattern from section 2: a router retrieves a relevant subset of tools from a larger catalog.
-
A Field Glossary for Agentic Knowledge Work
Everywhere you look right now, somebody is saying agentic this and agent that. Harness, scaffold, skill, subagent, agentic OS. The vocabulary is piling up faster than anyone can keep track of, and a lot of it gets used loosely, sometimes by people who don’t actually know what the words mean.
So I figured I’d write down my own working glossary. This isn’t a textbook, and I’m not pretending these are official definitions. It’s how I think about the terms when I’m doing the work. If you’ve been nodding along in conversations without being totally sure what a harness is, this one’s for you.
Agent
There are a lot of flavors of agent, and I’m not going to catalog all of them. Generally speaking, an agent is some code wrapped around an LLM that runs in a loop, has access to tools, and can act using those tools.
That’s the key difference from a plain prompt-and-response call. A direct call to the model gives you one answer and stops. An agent has some degree of autonomy. It can decide to use a tool, look at the result, and keep going. The loop and the tools are what make it an agent instead of a chatbot.
Harness
A harness is the program that sits on top of the model. It manages conversation state, runs the reasoning loop, gives the model access to tools, and enforces the guardrails, things like permissions, controls, and budget.
Here’s an easy way to understand it. The model is the intelligence. The harness is the control on the intelligence. The harness sits between you and the LLM.
A harness can show up in a lot of places. It might be a CLI. It might be a GUI or an app on your phone. It might be a chat thread. You could wire up something like OpenClaw to talk to you in WhatsApp or Telegram, and that chat becomes your harness, while OpenClaw is also a harness underneath. So yes, harnesses can call other harnesses. It’s turtles a little way down.
Scaffold
You’ll hear people say scaffold or scaffolding. This is usually just another word for harness. The prompt, the tools, and the control structure wrapped around the model. Same idea, different label.
Framework or SDK
These are the libraries you build harnesses with. LangChain, the various agent SDKs, or a ready-to-run harness like Claude Code or Hermes.
Worth flagging that framework and SDK mean something specific in regular programming. In the agentic context they’re a little looser. They’re what you build agents and harnesses out of. And it doesn’t have to be off-the-shelf. You can absolutely build your own framework for building your own harnesses if that’s where your head is at.
Context Engineering
This is the big one. The term comes from Karpathy, and while it mattered even more a year ago than it does today, it still applies.
Context engineering is deliberately managing what’s in the context window of the current session. It’s the work of deciding what gets loaded into context and, just as importantly, what gets left out. It’s the successor to what we used to call prompt engineering. The framing shifted because the prompt is only one piece of what the model sees, and the rest of it matters a lot.
MCP
Model Context Protocol. I won’t go deep here, it deserves its own post. MCP is an open protocol for exposing tools and data to an LLM harness. It’s the standard way for your harness to reach out and use third-party software or pull in outside data.
Skills
Skills are a major, important thing, and I’m not going to do them justice in a glossary entry. But here’s the definition.
A skill is a reusable, often self-created capability that bundles up the instructions an agent needs to accomplish a specific task. You can find skills all over the internet now. Everybody’s got their own. You can generate your own pretty easily with the CLI, and harnesses like Claude Code or Hermes can even author their own skills. The word does the work here. Skills are capabilities. It’s how you extend what your agents and harnesses can actually do.
Subagent
A subagent is an isolated child agent spawned from inside a working session.
Say you’re in your harness of choice with a main session running. That orchestration session can fork off a new agent with its own context window, hand it a specific task, and say go do this. The subagent runs on its own, often in parallel with others, working in the background. The main session knows when it finishes and can check the work.
A lot of the time you’ll have a second subagent review the first one’s output. That review loop is the whole idea behind the agentic maturity model, which is a way of thinking about how to structure this kind of work. It’s on GitHub if you want to dig in.
Agentic OS
This is an orchestration layer that combines agents, memory, and tools. It’s not really an operating system, but the name has stuck. You take all these concepts, the skills, the agents, the memory, the tools, and combine them into one organized whole. People are calling that amalgamation an agentic OS.
Second Brain / PKM
A personal knowledge vault. I posted about this just this week. It’s a personal knowledge base that the model can read, search, and extend. Your notes, your references, your accumulated thinking, made available to the agent.
Vibe Coding vs. Agentic Engineering
This is the distinction I care about a lot.
Vibe coding is not really knowing what you’re doing or how it’s being done. Anyone can vibe code. You describe what you want and you accept what comes back.
Agentic engineering is knowing what you’re doing and caring about how it gets done. Not everyone can do that part.
The way people put it is that vibe coding raises the floor and agentic engineering raises the ceiling. Vibe coding lets anyone build something. Agentic engineering lets a professional move a lot faster than they used to. Both are real. They’re not the same thing.
That’s the glossary, at least the version that lives in my head. None of these terms are settled, and half of them will probably mean something slightly different in six months. But hopefully this has helped!
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
Sources
- Vibe coding — Wikipedia — the term coined by Andrej Karpathy (Feb 2025); the “vibe coding vs. agentic engineering” distinction the post draws.
- Andrej Karpathy, "+1 for ‘context engineering’ over ‘prompt engineering’" (X, June 25, 2025) — the origin of the term “context engineering” as the successor to prompt engineering.
- Model Context Protocol — Wikipedia — MCP as the open standard introduced by Anthropic (Nov 2024) for exposing tools and data to LLM harnesses.
- Emma Roth, “Anthropic launches tool to connect AI systems directly to datasets” (The Verge, Nov 25, 2024) — news coverage of the MCP launch.
- Agentic Maturity Model — GitHub — the AMM referenced in the Subagent entry; Level 3 describes the worker-plus-reviewer subagent pattern.
-
The Agentic Dev Space Is Moving Fast, and I'm Having a Blast
Step back and look at the last six months of software development. It’s almost hard to believe how much has shifted. Back in January we were doing “vibe coding” with basic autocomplete. By June we’re handing fully autonomous agents a feature, walking off to grab a coffee, and coming back to find it architected, written, and tested.
It’s easy to look at the volume of new tools dropping every single day and feel a little fatigued. But honestly? I’m just excited. This is one of the most fun eras of programming I’ve worked through in decades.
Here’s a quick look at the tools that have already changed my workflow, and the ones I want to dig into next.
The Tools I’ve Adopted
If you read my post earlier this week, you know Supacode has become my daily driver. It bridged the gap between raw terminal access and a UI that understands how context-heavy agentic work is. That’s a harder problem than it sounds, and it’s really feeling good so far.
I also have to shout out the tools that paved the way earlier this year. Claude Code and Antigravity proved out the model of a CLI-native agent that could navigate your file system and do the work. Running Claude Code daily—and turning to Antigravity occasionally when I have access—has completely retrained my habits. I stopped typing every line of code and started acting more like a technical lead reviewing pull requests from a tireless junior developer. That shift in posture is the real unlock, and these were the tools that taught me it.
The Tools on My Radar
Because the space moves so fast, my “stuff I want to learn” backlog keeps growing faster than I can clear it. Here are the three I’m most eager to explore when I carve out the time:
- Hermes. I keep hearing great things about how it handles multi-step reasoning and huge context windows. I haven’t had the right project to throw at it yet, but it’s at the top of the list.
- Pi Agents. The concept here is highly specialized, networked agents working in tandem. Instead of one monolithic agent doing everything, you’d have a “frontend agent” talking to a “database agent.” That feels like a different way to structure the work, and I want to see if it holds up in practice.
- Evaluating Memory Systems. I’ve been diving into how agents remember things over time. I wrote a Mem0 MCP server that runs locally, and I’ve added hooks for Mem0 right into my CLI agents. It’s been a great way to slowly improve my own local agentic memory system. I’m really eager to see how other memory systems (like LangChain’s implementations) work under the hood, and I want to spend more time comparing and contrasting them.
You Don’t Have to Learn It All Today
The best part about this explosion of tooling is that you don’t need to know all of it. You don’t.
Find a tool that solves an immediate problem in your workflow, master that one, and let the rest sit on your radar until you need them. The ecosystem will keep evolving whether or not you’re watching every release. The tools will keep getting better. Chasing every launch is a great way to learn nothing well.
So pick one. Get good at it. Let the backlog wait.
What’s the one agentic tool sitting on your radar that you just haven’t had time to dig into yet?
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
I published an Agentic Maturity Model on GitHub, a mental framework for thinking about and categorizing AI tools. It’s open to contributions and I’m looking for coauthors.