Langchain

• LangChain and LLM Routers, the Short Version

  LangChain is important to know and understand in the age of agents. Also, LLM routing. They’re related but they’re not the same thing, and the distinction matters.

  So lets break it down.

  LangChain is the Plumbing

  Out of the box, an LLM is a text-in, text-out engine. It only knows what it was trained on. That’s it. LangChain is an open-source framework that connects that engine to the outside world.

  It gives you standardized tools to build pipelines:

  • Models: Interfaces for talking to different LLMs (Gemini, Claude, OpenAI, whatever you’re using)
  • Prompts: Templates for dynamically constructing instructions based on user input
  • Memory: Letting the LLM remember past turns in a conversation
  • Retrieval (RAG): Connecting the LLM to external databases, PDFs, or the internet so it can answer questions about your data
  • Agents & Tools: Letting the LLM actually do things, like execute code, run a SQL query, or send an email

  You could wire all of this up yourself, but LangChain gives you the standard pieces so you’re not reinventing the plumbing every time.

  LLM Routers are the Traffic Controller

  A router is an architectural pattern you build on top of that plumbing. Instead of sending every request through the same prompt to the same massive model, a router evaluates the request and directs it to the right destination. Simple concept, big impact.

  Three reasons you’d want one:

  • Cost: You don’t need a giant, expensive model to answer “Hello!” or look up a basic fact. Send simple queries to a smaller, cheaper model. Save the heavy model for complex reasoning.
  • Specialization: Maybe you have one prompt for writing code and another for searching a company HR manual. The router makes sure the query hits the right expert system.
  • Speed: Smaller models and direct database lookups are faster. Routing makes your whole application more responsive.

  How Routing Actually Works

  In LangChain, there are two main approaches:

  Logical Routing uses a fast LLM to read the user’s prompt and categorize it. You tell the router LLM something like: “If the user asks about math, output MATH. If they ask about history, output HISTORY.” LangChain then branches to a specialized chain based on that output.

  Semantic Routing skips the LLM entirely for the routing decision. It converts the user’s text into a vector (an array of numbers representing the meaning of the text) and compares it to predefined routes to find the closest match. This is significantly faster and cheaper than asking an LLM to make the call.

  LangChain provides RunnableBranch in LCEL (LangChain Expression Language, their declarative syntax for chaining components) for this, basically if/then/else logic for your AI pipelines. Worth digging into if you’re building with LangChain.

  Routing is what makes AI applications practical at scale. LangChain is one way to build it. They’re complementary, not interchangeable.

  I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].

  Apr 23, 2026 / AI / Programming / Langchain / LLM

  Enjoyed this?