Observability

• How I Would Build Observability for an Autonomous Agent

  I have not built a full production observability stack for an autonomous agent.

  I’ve built lots of small wrappers around existing coding harnesses. I have a pretty good idea how quickly their output can turn into a wall of model responses, tool calls, and subprocess logs. But I have not run LangChain across a Kubernetes cluster or operated an LLM router at scale.

  So this is not a postmortem. It’s a design exercise.

  If one of my wrappers became an always-on agent service tomorrow, what would I need to see when it failed? Where would I put that telemetry on Azure, Railway, or Cloudflare? And which parts of the design should stay the same no matter where it runs?

  Start With the Trace, Not the Platform

  An agent run is a distributed trace hiding inside a loop.

  There is a request that starts the work. The agent calls a model, the model requests a tool, the tool talks to another service, and the result goes back into the model. Repeat that enough times and a normal application log becomes hard to follow because the interesting question is not just, “What failed?” It is, “What sequence of decisions got us here?”

  I would use OpenTelemetry and give every run one root trace. Each model call and tool call becomes a child span. Structured logs carry the same trace and span IDs, so I can move from a failed run to the exact log event without searching timestamps and hoping I found the right one.

  The minimum useful event would look something like this:

  {
    "event": "agent.tool.completed",
    "agent_run_id": "run_01K0...",
    "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
    "span_id": "00f067aa0ba902b7",
    "step": 7,
    "tool": "github.get_issue",
    "duration_ms": 842,
    "status": "error",
    "error_type": "rate_limit"
  }
  

  That gives me four things I can debug:

  • The run: which request or scheduled job started the work?
  • The model: which provider and model answered, how long did it take, and how many tokens did it use?
  • The tool: what operation ran, how long did it take, and did it succeed?
  • The sequence: what happened immediately before the failure?

  I would keep an agent_run_id because it is useful in a UI and support ticket, but I would not use it as a replacement for a trace ID. The trace context needs to travel across HTTP calls, queues, model gateways, and anything else the agent touches.

  Do Not Log Everything

  The tempting version of agent observability records every prompt, response, tool argument, environment variable, and file body. It is also a convenient way to copy credentials and private data into the system with the broadest internal access.

  My default would be structural telemetry first. Model name, token counts, latency, tool name, result status, retry count, and safe error categories are useful without storing the full content.

  Prompt and tool payload capture would be an explicit policy, not a debug switch someone leaves enabled for six months. I would use an allowlist, redact before export, encrypt whatever remains, and give raw payloads a shorter retention period than ordinary metrics.

  This also answers where an AI gateway fits. The gateway is the natural place to record model latency, provider errors, and token usage. It cannot see the agent’s local decisions or tool calls, so it is one part of the trace, not the entire observability system.

  The Same Design in Three Places

  The trace model stays the same. The deployment choice changes how much infrastructure I have to own.

  Azure: The Integrated Option

  If the agent already lived in Azure, I would use the Azure Monitor OpenTelemetry distribution and send telemetry to Application Insights, backed by a Log Analytics workspace. That gives the system a managed place for traces, logs, metrics, exceptions, queries, and retention policies.

  Microsoft Foundry can also trace model calls, tool calls, intermediate steps, tokens, latency, and errors into Application Insights. I would treat that as an accelerator, not an excuse to skip the application-level design. Some of its agent tracing paths are still in preview, and content tracing can include sensitive prompts and outputs.

  Azure is the option I would choose for an organization already paying the Azure complexity tax. It has the most integrated telemetry path of these three and the governance controls a larger company is likely to ask for. For a personal agent prototype, it is probably more platform than I need.

  Railway: The Straightforward Container

  Railway is where I would start for a small agent that needs a normal process, a Docker image, background work, and the freedom to use ordinary libraries.

  I would instrument the application with OpenTelemetry and export traces over OTLP to an external observability backend. Railway’s built-in logs and container metrics are useful for deployment health, CPU, memory, disk, and network usage. They are not a distributed tracing backend for the agent itself.

  That split is fine. Railway runs the service, while the application owns its telemetry schema and exports it somewhere designed to query traces. I would also make sure the exporter flushes on SIGTERM, because a clean deployment is not helpful if the final spans disappear during shutdown.

  This is the least ceremonial option. It is also the one where I would have to make a separate decision about the telemetry backend and its retention cost.

  Cloudflare: The API-Oriented Agent

  Cloudflare gets interesting when the agent mostly calls models and HTTP APIs instead of running shell commands against a workspace.

  Workers can collect logs and traces automatically, and Cloudflare can export both in OpenTelemetry format to an external destination. I would still add application spans for the model and tool semantics because automatic request tracing cannot know which prompt, tool, or agent step matters to me. The OTLP export is also currently beta and does not export Worker metrics.

  For state, I would use a Durable Object when one run needs a single coordination point, D1 for relational records across runs, and R2 for larger artifacts or archived transcripts. If the agent needs retries, sleeps, or a human callback, Cloudflare Workflows is a better fit than trying to keep one HTTP request alive.

  I would not put a coding agent that needs arbitrary processes, a writable repository, and long CPU-heavy tool calls into a Worker just because the edge sounds nice. Cloudflare’s runtime has real CPU, memory, connection, and logging limits. For that workload, I would run the agent in a container and consider Cloudflare for the gateway or API edge instead.

  What I Would Pick

  For my first version, probably Railway. It matches the small wrappers I already build, lets the agent behave like a normal application, and leaves me free to try different tracing backends without moving the workload.

  I would pick Azure when the surrounding organization already uses Azure Monitor and needs one managed governance story. I would pick Cloudflare when the agent is an API orchestrator and its tools already live on the network.

  A mistake would be choosing a platform or dashboard first and calling that observability solved. I would start small with a trace per run or tool call with structured logging turned on, but no prompt content until there is a redaction or sanitization policy feature in place.

  I haven’t built this stack yet. But that is how I would start building it.

  Sources & References

  • OpenTelemetry Logs specification — trace and span correlation in structured logs.
  • Azure Monitor OpenTelemetry — supported telemetry signals and Application Insights setup.
  • Microsoft Foundry tracing — agent trace contents, storage, and data-handling considerations.
  • Railway third-party observability — application-side OpenTelemetry export and shutdown guidance.
  • Railway metrics — native container and service metrics.
  • Cloudflare Workers traces — automatic tracing and native retention.
  • Cloudflare OpenTelemetry export — OTLP trace and log export capabilities and current limitations.
  • Cloudflare Durable Objects — per-instance coordination and transactional storage.
  • Cloudflare Workflows — durable multi-step execution, retries, and external events.
  • Cloudflare D1 — relational storage for records shared across runs.
  • Cloudflare R2 — object storage for larger artifacts and archived transcripts.
  • Cloudflare Workers limits — CPU, memory, connection, and log limits.

  I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].

  Jul 29, 2026 AI Agents cloudflare Azure Observability Railway

  Enjoyed this?