AI
-
How to Pick an Embedding Model (Without Overthinking It)
It’s easy to get deep into vector database comparisons, HNSW vs. IVF, pgvector vs. Pinecone, Qdrant vs. Chroma, and completely skip over the thing that actually matters most: the embedding model.
The way I think about it, the embedding model is the brain of your retrieval system. The vector database is just its filing cabinet. If the model creates poor mathematical representations of your data, no amount of indexing strategy or database performance is going to save you. You’ll get fast, confident, wrong results.
So let’s talk about how to pick a model.
Dimensionality: More Isn’t Always Better
Embeddings are high-dimensional vectors. Common sizes are 384, 768, 1536, or 3072 dimensions. Higher dimensions capture more nuance, but they also mean more storage, more memory, and slower search.
For a lean, local-first setup, something like
all-MiniLM-L6-v2at 384 dimensions gives you a surprisingly good balance of speed and accuracy. You don’t need 3072 dimensions to search your notes. Save the big vectors for when you actually have a reason.Sequence Length: The Silent Data Killer
Sequence length determines how much text the model can look at to create a single vector. If you’re embedding long technical docs or sprawling Markdown files and your model caps out at 512 tokens, it’s just truncating everything past that point. Your carefully written documentation gets chopped, and the embedding only represents the first few paragraphs.
Modern long-context embedding models handle 8k to 32k tokens, which lets you embed entire chapters or large code blocks as single semantic units. If your content is longer than a few paragraphs, check this number before anything else.
Domain Matters More Than You Think
General-purpose models like OpenAI’s
text-embedding-3-smallwork well across most tasks. They’ve been trained on massive, diverse datasets and they’re solid defaults.If you’re searching a codebase or technical documentation, models fine-tuned on programming languages (like
voyage-code-2) will outperform the general ones. The same applies to medical or legal text, where domain-specific jargon means the difference between a relevant result and a completely wrong one.Check MTEB Before You Commit
The Massive Text Embedding Benchmark (MTEB) is the industry standard for comparing models. It breaks performance into sub-categories like Retrieval, Summarization, and Clustering. If you’re building RAG, look at the Retrieval scores specifically. A model that ranks well for clustering might be mediocre at retrieval, and vice versa.
Local vs. API: Pick Your Tradeoff
This decision is as important as the model itself.
- Local models (via HuggingFace or Ollama) keep everything offline. Zero per-request costs, full privacy. Something like
bge-small-en-v1.5running locally is perfect for personal knowledge management or anything where your data shouldn’t leave your machine. - Hosted APIs (OpenAI, Voyage, Cohere) give you the highest performance and longest context windows without managing GPU infrastructure. Better for enterprise scale where you’re willing to trade privacy and recurring costs for accuracy.
Local models make sense for personal projects and hosted APIs make sense when the scale demands it. There’s no universal right answer, but there is a wrong one: picking a deployment model without thinking about where your data lives.
The vector database conversation is important, but it’s second in line to getting the embedding model right first. Everything downstream depends on it.
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
- Local models (via HuggingFace or Ollama) keep everything offline. Zero per-request costs, full privacy. Something like
-
Pandoc vs MarkItDown: Two Tools, Two Eras
Pandoc has been the gold standard for document conversion for nearly two decades. But there’s a newer tool from Microsoft called MarkItDown, and while the names sound like they do similar things, they were built for completely different reasons.
Pandoc is a universal document converter designed for human publishing. It converts almost any format into almost any other format while preserving complex typography, citations, and formatting. MarkItDown is a specialized extraction tool designed for AI. It converts various files strictly into Markdown so that LLMs and RAG pipelines can read and process the text.
Same input files, very different goals.
Pandoc: The Universal Translator
Pandoc has been around since 2006, written in Haskell, and it operates on an Abstract Syntax Tree. It reads a document, builds a complex internal model of its structure, and then translates that structure into your desired output. We’re talking 40+ output formats here. PDF, Word, HTML, LaTeX, EPUB, you name it.
Where it really shines is academic and technical writing. It natively understands LaTeX math, footnotes, bibliographies, and cross-referencing. You can turn a Word doc into Markdown, edit it, and use Pandoc to turn it back into a perfectly formatted PDF. Two-way conversion that actually works.
You can also write custom filters in Lua or Python to programmatically alter documents during conversion. Want to automatically downgrade all your H2s to H3s? Pandoc has you covered.
MarkItDown: The LLM Feeder
MarkItDown was released by Microsoft in late 2024 to solve a very modern problem. LLMs need clean, structured text to “read” documents, but corporate data is locked inside messy formats like multi-tab Excel spreadsheets, image-heavy PowerPoints, and ZIP archives.
It’s a Python library first, CLI second. It drops into your scripts in a few lines of code, which makes it easy to wire up with LangChain, LlamaIndex, or raw API calls. The output is always Markdown. That’s it. No PDF generation, no Word docs, no EPUB. Just clean text that an AI can process.
The interesting trick is what it does with images and audio. Feed it a PDF with diagrams and MarkItDown can connect to an LLM like GPT-4o to look at the image and write a Markdown description of what it sees. It can also transcribe audio files. That’s a fundamentally different approach from Pandoc, which preserves images as files rather than describing them.
Quick Comparison
Feature Pandoc MarkItDown Primary Goal Universal document conversion Document ingestion for AI Output Formats 40+ (PDF, Word, HTML, LaTeX, etc.) Only Markdown Language Haskell (standalone CLI) Python (library-first) Image Handling Preserves and extracts image files Uses OCR/LLM Vision to describe images as text Complex Formatting Citations, bibliographies, LaTeX math, custom filters Basic structural support (headings, tables, slides) So Which One Do You Want?
Pandoc if you’re writing a book, research paper, or blog and need polished output in multiple formats. If you need to maintain citations, complex formatting, or convert files out of Markdown into something else, Pandoc is your tool.
MarkItDown if you’re building an AI agent, chatbot, or search tool and need to extract text from a pile of PDFs, Excel files, and PowerPoints. If you only care about getting raw structured text and don’t care about the visual layout of the original document, MarkItDown is purpose-built for that.
They’re not competitors. Pandoc is for publishing. MarkItDown is for feeding AI. Pick the one that matches what you’re actually trying to do.
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
pgvector vs Pinecone: You Probably Don't Need a Separate Vector Database
Every time someone starts building a RAG pipeline, the same question will come up: do I need a “real” vector database like Pinecone, or can I just use pgvector with the Postgres I already have?
I can imagine teams agonizing over this decision for weeks. So maybe this will save you some time?
The Case for Staying Put
If you already have a PostgreSQL instance in your stack, adding
pgvectoris almost always the right first move.You manage one stateful service instead of two. Your existing backup strategy, monitoring, and security all stay the same. Your vector embeddings live next to your metadata, so you get ACID compliance and standard SQL joins. No syncing between two data stores. No eventual consistency headaches.
Performance? From what I found, for datasets under a few million vectors,
pgvectorwith HNSW indexes is fast. Really fast. It satisfies the latency requirements of most applications without breaking a sweat.And you’re not paying for another SaaS subscription…
When Pinecone Actually Makes Sense
Pinecone is a purpose-built vector database designed for high-dimensional data at massive scale. It’s serverless and fully managed.
If you’re dealing with hundreds of millions or billions of vectors, a specialized engine handles memory and disk I/O for similarity searches more efficiently than Postgres can. Pinecone also gives you native namespace support, metadata filtering optimized for vector search, and live index updates that are faster than re-indexing a large Postgres table.
Those are real advantages. At a certain scale.
The Decision Is Simpler Than You Think
Stay with Postgres + pgvector if:
- You want to minimize infra sprawl and moving parts
- Your vector dataset is under 5 to 10 million records
- You rely on relational joins between vectors and other business data
- You have existing observability and DBA expertise for Postgres
Consider Pinecone if:
- Your Postgres instance needs massive, expensive vertical scaling just to keep the vector index in memory
- You don’t want to tune HNSW parameters,
mmapsettings, or vacuuming schedules for large vector tables - You need sub-millisecond similarity search at a scale where Postgres starts to struggle
That is what I would use to make that decision.
Most teams are probably nowhere near the scale where Pinecone becomes necessary. They have a few hundred thousand vectors, maybe a million or two. Postgres handles that without flinching. Adding a separate managed vector database at that point is just adding operational complexity for no measurable benefit.
The trap is thinking you need to “plan ahead” for scale you don’t have yet. You can always migrate later if you actually hit the ceiling. Moving from pgvector to Pinecone is a well-documented path. But moving from two services back to one because you overengineered your stack? That’s a conversation nobody wants to have.
Start with what you have. Add complexity when the numbers force you to, not when a vendor’s marketing page makes you nervous.
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
LangChain and LLM Routers, the Short Version
LangChain is important to know and understand in the age of agents. Also, LLM routing. They’re related but they’re not the same thing, and the distinction matters.
So lets break it down.
LangChain is the Plumbing
Out of the box, an LLM is a text-in, text-out engine. It only knows what it was trained on. That’s it. LangChain is an open-source framework that connects that engine to the outside world.
It gives you standardized tools to build pipelines:
- Models: Interfaces for talking to different LLMs (Gemini, Claude, OpenAI, whatever you’re using)
- Prompts: Templates for dynamically constructing instructions based on user input
- Memory: Letting the LLM remember past turns in a conversation
- Retrieval (RAG): Connecting the LLM to external databases, PDFs, or the internet so it can answer questions about your data
- Agents & Tools: Letting the LLM actually do things, like execute code, run a SQL query, or send an email
You could wire all of this up yourself, but LangChain gives you the standard pieces so you’re not reinventing the plumbing every time.
LLM Routers are the Traffic Controller
A router is an architectural pattern you build on top of that plumbing. Instead of sending every request through the same prompt to the same massive model, a router evaluates the request and directs it to the right destination. Simple concept, big impact.
Three reasons you’d want one:
- Cost: You don’t need a giant, expensive model to answer “Hello!” or look up a basic fact. Send simple queries to a smaller, cheaper model. Save the heavy model for complex reasoning.
- Specialization: Maybe you have one prompt for writing code and another for searching a company HR manual. The router makes sure the query hits the right expert system.
- Speed: Smaller models and direct database lookups are faster. Routing makes your whole application more responsive.
How Routing Actually Works
In LangChain, there are two main approaches:
Logical Routing uses a fast LLM to read the user’s prompt and categorize it. You tell the router LLM something like: “If the user asks about math, output MATH. If they ask about history, output HISTORY.” LangChain then branches to a specialized chain based on that output.
Semantic Routing skips the LLM entirely for the routing decision. It converts the user’s text into a vector (an array of numbers representing the meaning of the text) and compares it to predefined routes to find the closest match. This is significantly faster and cheaper than asking an LLM to make the call.
LangChain provides
RunnableBranchin LCEL (LangChain Expression Language, their declarative syntax for chaining components) for this, basically if/then/else logic for your AI pipelines. Worth digging into if you’re building with LangChain.Routing is what makes AI applications practical at scale. LangChain is one way to build it. They’re complementary, not interchangeable.
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
Your Brain vs. a Large Language Model
We don’t fully understand the human brain. That’s just how things are. But we know enough about its structure to make some genuinely interesting comparisons to how large language models work. So let’s walk through the major components of your brain and see where the parallels land.
The Neocortex and the Transformer
The neocortex is the outer layer of your brain, responsible for the higher-order stuff: sensory perception, spatial reasoning, language. The prefrontal cortex (PFC) sits within it as the orchestrator. It handles executive function, decision-making, and complex thought.
The LLM equivalent here is the transformer architecture itself. And the PFC’s role maps surprisingly well to the attention mechanism. The attention mechanism decides what information matters most given the current context, which is essentially what your prefrontal cortex does all day.
If you’ve worked with agentic AI systems, you’ve probably seen this pattern play out directly. You typically have an orchestration agent managing specialized sub-agents, each built for a specific task. That management layer is doing PFC work, deciding which agent to activate, what context to pass along, and how to synthesize the results.
The Hippocampus and Memory
The hippocampus is your storage unit. It’s critical for forming new memories and converting short-term experiences into long-term ones. Think of it as a buffer between what just happened and what you’ll remember later.
The LLM equivalent splits into two pieces. The model weights are your long-term memory, everything learned during training. The context window is your working memory, what the model can hold in its head right now for the current conversation.
LLMs don’t natively have long-term memory. The weights are baked in during training and that’s it. But memory systems get bolted on as part of the harness, and this is where retrieval-augmented generation (RAG) comes in. RAG lets the model pull in external data to contextualize its responses, which is functionally the same thing your hippocampus does when it retrieves a stored memory to help you make sense of something new.
Synapses and Parameters
Synapses are the gaps between neurons where signals pass, chemical or electrical. The strength of those connections determines how information flows through your brain. Stronger connections mean faster, more reliable signal paths.
This maps directly to model weights and parameters. Stronger connections between data points in the model mean those patterns carry more influence over the output. When we say a model has 170 billion parameters, we’re effectively describing the synaptic density of a digital brain. It’s not a perfect analogy, but it gives you an intuitive sense of scale.
Dopamine and RLHF
Your brain’s dopamine system is its reward circuit. It fires when an outcome is better than expected, reinforcing beneficial behaviors over harmful ones. It’s how you learn that some choices are worth repeating.
The LLM equivalent is reinforcement learning from human feedback, or RLHF. During training, humans rank the model’s responses. Good answers get a mathematical reward signal that makes similar outputs more likely in the future. Bad answers get penalized. This is the alignment problem in a nutshell: teaching the model what we find valuable and useful, the same way dopamine teaches your brain what’s worth pursuing.
This is also where the analogy breaks down the most. Dopamine is intrinsic. It’s wired into your survival. You don’t choose to feel rewarded when you eat, your brain just does that. RLHF is a proxy. The model isn’t learning what’s actually helpful, it’s learning what a secondary reward model scores as helpful. The result is a system that optimizes to appear useful rather than be useful. That’s why models can be confidently wrong or agree with you when you’re clearly mistaken. The reward signal says “the human liked that,” not “that was true.”
The Basal Ganglia and Routing
The basal ganglia is your gating mechanism. It’s a group of structures involved in motor control, habit formation, and deciding which thoughts or movements should surface and which should be suppressed. It’s basically your brain’s security and routing layer.
The LLM equivalent is the routing logic in mixture-of-experts (MoE) models. Every major provider uses some degree of MoE at this point. Different parts of the network activate depending on the task at hand, which is exactly what the basal ganglia does. System prompts play a similar role too, shaping how the model decides to respond given a particular input or situation.
So What?
None of these comparisons are perfect. The brain is biological, messy, and shaped by millions of years of evolution. LLMs are mathematical, deterministic (mostly), and shaped by a few years of engineering. But the structural parallels are hard to ignore. Attention mechanisms, memory systems, reward signals, gating logic. We keep arriving at similar architectural patterns, just built differently.
I don’t think that’s a coincidence, it tells us something about what intelligence requires, regardless of whether it’s running on neurons or GPUs.
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
NotebookLM Is Just RAG With a Nice UI
I’ve been watching AI YouTubers recommend NotebookLM integrations that involve authenticating your Claude instance with some random skill they built. “Download my thing, hook it up, trust me bro.” No details on how it works under the hood. No mention of why piping your credentials through someone else’s code might be a terrible idea. Let’s just gloss over that, I guess.
So here we are. Let me explain what NotebookLM actually is, because once you understand RAG, the magic disappears pretty quickly.
What Is RAG?
RAG stands for Retrieval Augmented Generation. It’s an AI framework that improves LLM accuracy by retrieving data from trusted sources before generating a response.
The LLM provides the reasoning and token generation. RAG provides specific, trusted context. Combining the two gives you general reasoning grounded in your actual data instead of whatever the model has or hallucinated from its training set.
The core pipeline looks like this:
- Take your trusted data (docs, PDFs, YouTube transcripts, whatever)
- Chunk it into pieces
- Create vector embeddings from those chunks
- Store the vectors in a database
- When you ask a question, embed the question into the same vector space
- Find the most similar chunks
- Feed those chunks into the LLM as context alongside your question
That’s it. That’s NotebookLM. Steps 1 through 6 are the retrieval half. Step 7 is where the LLM synthesizes an answer. The nice UI on top doesn’t change what’s happening underneath.
I Accidentally Built Half of It?
I was interested in the semantic embeddings portion of this pipeline and ended up building something I called Semantic Docs. It handles the retrieval half, steps 1 through 6.
You point it at a knowledge base, internal company docs, research papers, whatever you’re interested in. It chunks the content, creates vector embeddings, and stores them in a database. When you search, it creates a new embedding from your query, finds the most similar chunks, and returns those as search results.
The difference between Semantic Docs and NotebookLM is that last step. Semantic Docs gives you the relevant files and passages. It says “here’s where the answers live, go read it.” It doesn’t pipe everything through an LLM to generate a synthesized response. This is a choice, a deliberate choice, not a missing feature.
Why No Official API Is a Problem
NotebookLM doesn’t have an official API. People have reverse-engineered how it works, which means every integration you see is built on undocumented behavior that could break at any time. The AI YouTubers recommending these workflows are essentially saying “trust this unofficial thing with your data and credentials.” That should make you uncomfortable.
If you understand RAG, you can build the parts you actually need. The retrieval half is genuinely useful on its own, and you control the whole pipeline. No third-party authentication. No undocumented APIs. No wondering what happens to your data.
I’ll probably write more about RAG in the future. It’s a good topic and there’s a lot of noise to cut through. For now, just know that the next time someone tells you NotebookLM is magic, it’s really just vector search with a chat interface on top.
If you’re a developer, I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week. Or you can find me on Mastodon at @[email protected].
-
AI-Assisted vs AI-Agentic Coding
There are two ways to work (c0de) with AI tools right now. I think most people know the other one exists, but they haven’t taken the time to try it. You should know how to do both. And when to do both.
Assisted Mode
Everybody knows this one. You write some code, you get stuck, you ask a question.
How does date parsing work in Python? What’s this function do? Haven’t we built this already? I need some fucking Regex again.
The AI answers. You copy-paste or accept the suggestion. You keep going. You’re driving. The AI is in the passenger seat reading the map.
I mean, this is really useful. I’m not going to pretend it isn’t. It’s also just autocomplete with opinions. Fancy autocomplete. Smart autocomplete.
Great. You’re doing the thinking. You’re deciding what gets built and how to structure it and what order to do things in. You’re just asking for help on some of the blanks. That’s assisted mode.
Agentic Mode
This is different.
You describe what you want. You need to know how to describe what you want.
That is extremely important. Let me say that again. You need to know how to describe what you want.
You need to build an agent that understands how to interpret your description as what you want.
Sometimes it’s going to get it correct and sometimes it’s not. It’s going to go in a different direction than you wanted and you’re going to have to correct it. That’s the job now. You’re reviewing the output, the code, and how it’s producing the code. What are the gaps? You have to find the gaps and improve the agent so that it understands you better.
When I Use Which
I wish I had a clean rule for this. I don’t. That’s the vibes part.
Small or specific things can be assisted. Quick answers. Great. Easy. Move on.
Once you start wanting to touch multiple files, agentic. Major features like commands or parser changes or handler rewrites, recipes or tests. I’m not writing all that by hand. I can describe what I want way better than I can autocomplete it.
Bug fixes? Depends. If I already know where the bug is, assisted. If I don’t, agentic. Let the agent grep around and figure it out. It’s better at reading a whole codebase quickly than I am. Not better at understanding it. Better at reading it.
New features? Almost always agentic. I describe the feature, point it at similar code in the repo, and let it go.
Again, review is super important. Sometimes you have to send it back or start over or change major portions of it. And if you build a system that learns, it’ll get better along the way.
The Review Problem
Switching to agentic mode, your entire job is code review. All day, all the time, constant. That’s the human’s job. Code review.
Are you good at code review? You should get better at it. You need to get better at it.
This is not whether or not the tests pass. You need to identify possible issues and then describe tests that can check for those issues.
The nuanced bugs are the worst. And if those make it to production, you’re going to have problems.
Don’t skim the diff.
That should be the new motto. Read the code. Get better at code comprehension. It’s extremely important. You may be writing less code but you need to sure as shit understand what the code is doing and how it can be bad.
The Hybrid Reality
It’s totally fine to switch between modes depending on what you’re doing or your work session. Agentic can be way more impactful, but assisted mode is way better at helping you understand what the code is doing because you can select code blocks and easily ask questions about it.
So it’s not a toggle, it’s a spectrum. Now isn’t that funny? I’m on the spectrum of agentic development.
Where are you on the spectrum of agentic development?
So Which Is Better?
Neither. Both. It depends. Whatever, just build stuff.
Is assisted mode safer? Really? Like, does the human actually write better code this way? I don’t know. Agentic mode can be faster and you need to be super careful that it’s not gaslighting you into thinking it knows what it’s doing.
Build software for you. And when it makes sense, help out with the community stuff. Support open source.
If you’re a developer, I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week. Or you can find me on Mastodon at @[email protected].
-
Claude Opus 4.7 Is Here
Anthropic just announced Claude Opus 4.7 yesterday, and here is my take on the new model after reading the blog post and doing a bit of research on their rollout plans from previous models.
What’s New
The headline is a 13% improvement on a 93-task coding benchmark over Opus 4.6. Rakuten’s SWE-Bench saw 3x more production tasks resolved, which is the kind of real-world metric that actually matters. Benchmarks are one thing, but “can it handle my actual codebase” is another.
The big quality-of-life improvement is that Opus 4.7 is better at verifying its own output before telling you it’s done. If you’ve ever had a model confidently hand you broken code and say “there you go,” you know why this matters. It handles long-running tasks with more precision, and the instruction following is noticeably tighter.
There’s also a major vision upgrade. The new model accepts images up to 2,576 pixels on the long edge, which is more than 3x the resolution of previous Claude models. If you’re working with technical diagrams, architecture charts, or screenshots of code, that’s a real improvement.
When Can You Actually Use It?
For enterprise customers, Anthropic says Opus 4.7 is available from your cloud vendor: the API, Amazon Bedrock, Google Cloud Vertex AI, and Microsoft Foundry. But most of us aren’t using the API directly.
As of right now, Opus 4.7 is not yet available in Claude Code or the desktop app. It’s also not showing up in the model picker on claude.ai for Pro plan users. Anthropic’s announcement says “available today across all Claude products,” but that doesn’t seem to have fully rolled out yet for consumer plans.
Looking at previous releases, Opus 4.6 launched on February 5th and was accessible on claude.ai and the API the same day. Historically, Anthropic hasn’t gated new Opus models behind higher tiers, so there’s no reason to think Pro, Max, Team, and Enterprise won’t all get access. The question is just when. If past patterns hold, it should show up within a few days. Keep checking your model picker.
Claude Code Users
As of today, Claude Code on the stable release is still on Opus 4.6. I’m not sure if it’s available on the bleeding edge builds, but for most people it’s not there yet.
The announcement mentions a few Claude Code features coming with 4.7:
/ultrareviewis a new slash command for dedicated code review sessions. Pro and Max users get three free ultrareviews to try it out.- Auto mode has been extended to Max plan users, letting Claude make more decisions autonomously.
- The default effort level is being bumped to
xhigh(a new level betweenhighandmax), which means the model will spend more time reasoning through harder problems.
Once Opus 4.7 does show up in Claude Code, remember to check any custom agents or skills that have a model hardcoded in the frontmatter. If you’ve got
claude-opus-4-6specified in your.claude/commands/directory or agent configurations, those will keep using the old model until you update them.Anthropic also notes that Opus 4.7 follows instructions more literally than previous models. Prompts written for earlier models can sometimes produce unexpected results. So if something feels off after switching, it’s worth re-tuning your prompts.
The Tokenizer and Cost Changes
One thing to be aware of: the tokenizer has been updated. The same input text will produce 1.0 to 1.35x more tokens than before. That means your costs could go up slightly even at the same per-token pricing ($5/million input, $25/million output, unchanged from 4.6). Not a dealbreaker, but worth watching if you’re running high-volume workloads.
Pricing hasn’t changed, the coding improvements look useful, and important to know that the model ID is
claude-opus-4-7. Keep an eye on your model picker over the next few days. -
Agentic Development Trends: What's Changed in Early 2026
I’ve been following the agentic development space around Claude Code and similar tools and the last couple months have been interesting. Here’s what I’m seeing as we move through March and April 2026.
From Solo Agents to Coordinated Teams
The biggest shift is that more people are moving away from trying to build one agent that does everything. Instead, we’re seeing coordinated teams of specialized agents managed by an orchestrator, often running tasks in parallel. I think this is the more proper use of these systems, and it’s great to see the community arriving here.
If you’re curious about the different levels of working with agentic software development, I created an agentic maturity model on GitHub that goes into more detail on this progression.
Long-Running Autonomous Workflows
Early on, agents handled what were essentially one-shot tasks. Now in 2026, agents can be configured to work for days at a time, requiring only strategic oversight at key decision points. Doesn’t that sound fun? You’re still the bottleneck, but at least now you’re a strategic bottleneck.
Graph-Based Orchestration
Frameworks like LangGraph and AutoGen are converging on graph-based state management to handle the complex logic of multi-agent workflows. I think this makes sense when you consider the branching and conditional logic of real-world tasks could map naturally to graphs.
MCP Is Everywhere
MCP (Model Context Protocol) has become the industry standard for tool integration. All vendors fully support it, and there’s no sign of slowing down. Every week there are new MCP servers popping up for connecting agents to different services and tools.
Unified Agentic Stacks
The developer tooling is becoming more consistent. Cursor is becoming more like Claude Code, and Codex is becoming more like Claude Code. Maybe you see a pattern there… might tell you something about who’s setting the pace.
What is also noteable, people are experimenting with using different tools for different parts of the workflow. You might use Cursor to build the interface, Claude Code for the reasoning and main logic, and Codex for specific isolated tasks. Mix and match based on strengths.
Scheduled Agents and Routines
Claude Code recently released routines or scheduled or trigger-based automations that can run 24/7 on cloud infrastructure without needing your laptop. Microsoft with GitHub Copilot are working on similar capabilities? Cursor had something like this a while back too.
Security Gets Serious
Two things happening here. First, people are getting better at leveraging agents for security reviews and monitoring. Tasks that previously required highly specialized InfoSec expertise. You no longer need to be a hacker to find vulnerabilities; you can let your AI try to hack you.
However, the same capabilities that harden defenses can also be used for offensive attacks. We’re seeing a major push for security-first architecture as a requirement for all new applications, specifically to defend against the rise of agentic offensive attacks. Red team and blue team are both getting AI-pilled.
FinOps: Watching the Bill
Last on the list is financial operations. Inference costs now account for over half of AI cloud spending according to recent estimates. Organizations are prioritizing frameworks that offer explicit cost monitoring and cost-per-task alerts. Getting granular about how much you’re spending to solve specific problems and optimizing at the task level. I think that’s pretty interesting and something we’ll see a lot more tooling around.
The common thread across all of these trends is maturity. We’re past the “wow, an AI wrote code” phase and into “how do we make this reliable, secure, and cost-effective at scale.” That’s a good place to be.
-
What Is an AI Agent, Actually?
We need some actual definitions. The word “agent” is getting slapped onto every product and service, and marketers aren’t doing anybody favors as they SEO-optimize for the new agentic world we live in. There’s a huge range in what these things can actually do. Here is my attempt at clarity.
The Spectrum of AI Capabilities
Chatbot / Assistant — This is a single conversation with no persistent goals and no tool use. You ask it questions, it answers from a knowledge base. Think of the little chat widget on a product page that helps you find pricing info or troubleshoot a common issue. It talks with you, and that’s about it.
LLM with Tool Use — This is what you get when you open “agent mode” in your IDE. Your LLM can read files, run commands, edit code. A lot of IDE vendors call this an agent, but it’s not really one. It’s a language model that can use tools when you ask it to. The key difference: you are still driving. You give it a task, it does that task, you give it the next one.
Agent — Given a goal, it can plan and execute multi-step workflows autonomously. By “workflow” I mean a sequence of actions that depend on each other: read a file, decide what to change, make the edit, run the tests, fix what broke, repeat. It has reasoning, memory, and some degree of autonomy in completing an objective. You don’t hand it step-by-step instructions. You describe what you want done, and it figures out how to get there.
Sub-Agent — An agent that gets dispatched by another a command or “LLM with Tool Use” to handle a specific piece of a larger task. If you’ve used Claude Code or Cursor, you know what I’m talking about. The main chat coordinator kicks off a sub-agent to go research something, review code, or run tests in parallel while it keeps working on the bigger picture. The sub-agent has its own context and tools, but it reports back to the parent. It’s not a separate autonomous agent with its own goals. It’s more like delegating a subtask.
Multi-Agent System — Multiple independent agents coordinating together, either directly or through an orchestrator. The key difference from sub-agents: these agents have their own goals and specialties. They negotiate, hand off work, and make decisions independently. Think of a system where one agent monitors your infrastructure, another handles incident response, and a third writes the postmortem. Each Agent is operating autonomously but aware of the others.
So How Is Something Like OpenClaw Different From a Chatbot?
A chatbot is designed to talk with you, similar to how you’d just talk with an LLM directly. OpenClaw is designed to work for you. It has agency. It can take actions. It’s more than just a conversation.
Obviously, how much it can do depends on what skills and plugins you enable, and what degree of risk you’re comfortable with. But here’s the interesting part: it’s proactive. It has a heartbeat mechanism that keeps it running continuously in the background. It’ll automatically check on things or take action on a schedule you specify, without you having to prompt it.
A Few Misconceptions Worth Clearing Up
OpenClaw is just one specific framework for building and orchestrating agents, but the misconceptions around it apply broadly.
“Agents have to run locally.” That’s how OpenClaw works, sure. But in reality, the enterprise agents are running invisibly in the background all the time. Your agent doesn’t need to live on your laptop.
“Agents need a chat interface.” Because you can talk to an agent, people assume you must have a chat interface for it to be an agent. But by definition, agents don’t require a conversation. They can just run in the background doing things. No chat window needed.
“Sub-agents are just function calls.” This one trips up developers. When your agent spawns a sub-agent, it’s not the same as calling a function. The sub-agent gets its own context window, its own reasoning loop, its own tool access. It can make judgment calls the parent didn’t anticipate. That’s fundamentally different from passing arguments to a function and getting a return value.
Why Write This Down
I mainly wrote this for myself. I keep running into these terms and needing a mental model to put them in context, so as I’m thinking about building agentic systems and trying to decide what level of capability I actually need for a given problem. The process of writing it down makes those decisions somewhat easier.