Book a 30-min call
← cd ../blogs
$ cat posts/agent-stack-2026-chatbots-to-autonomous-systems.mdx

The Agent Stack in 2026: From Chatbots to Autonomous Systems

April 9, 2026 · updated August 8, 2026 · ImmovableTech Team

  • Agentic AI
  • MCP
  • Production AI

The chatbot era is over

Remember when “AI integration” meant bolting a ChatGPT wrapper onto your product? That was 2023. By 2024, the market figured out that chatbots without tools are just expensive autocomplete. The real value wasn’t in generating text — it was in AI that could do things: query databases, call APIs, modify files, trigger workflows.

That realisation kicked off the agentic revolution. And by early 2026, the dust has settled enough to see what the production agent stack actually looks like. It’s not one framework. It’s a layered architecture with protocols, orchestration engines, and operational tooling that looks surprisingly similar to how we built microservices a decade ago.

The stack, layer by layer

Layer 1: tool protocol — MCP

The Model Context Protocol is the HTTP of agents. It defines how an AI model discovers and calls external tools via a standard JSON-RPC interface. Anthropic created it, donated it to the Linux Foundation’s Agentic AI Foundation in December 2025, and every major AI lab supports it. It was cited at over 97 million monthly SDK downloads at the point of donation, and the core SDKs have grown several-fold since.

Before MCP, integrating a tool meant writing custom code for each model provider’s function-calling API. OpenAI had one format, Anthropic had another, Google had a third. MCP collapsed that into a single standard. You build one MCP server, and any MCP-compatible model can use it.

The practical impact: we built an MCP server for a client’s internal knowledge base once, and it worked immediately across models from three different vendors without any model-specific code.

Layer 2: agent communication — A2A

The Agent2Agent protocol, originally Google’s and donated to the Linux Foundation in June 2025, addresses a different problem: how do agents from different systems talk to each other? MCP is agent-to-tool. A2A is agent-to-agent.

The distinction matters at enterprise scale. Imagine a customer support agent that needs to check order status (handled by a logistics agent), verify payment (handled by a finance agent), and update the CRM (handled by a sales agent). Each agent might run in a different service, built by a different team, using a different model. A2A standardises how they negotiate tasks, share context and report results.

We haven’t deployed A2A in production, and it isn’t a maturity objection — it’s that none of our systems yet has a real cross-organisational agent boundary. Inside a single deployment, in-process state passing beats a network protocol on every axis that matters. A2A becomes the right answer when the agents belong to different teams or different companies.

Layer 3: orchestration — LangGraph, CrewAI, Microsoft Agent Framework

This is where the “thinking” happens. Orchestration frameworks define the control flow: which agent runs first, what happens when it fails, how state passes between agents, and when a human gets pulled into the loop.

We’ve used all three major frameworks. Here’s our honest assessment:

LangGraph — Our default for production. It models agents as nodes in a directed graph with explicit state transitions. You get deterministic control flow, built-in persistence, and the ability to replay failed runs from the exact point of failure. The downside: the learning curve is steep. You’re building state machines, not writing prompts.

CrewAI — Great for prototyping and simpler workflows. You can define agents with plain-text roles and let the framework handle coordination. But the implicit orchestration becomes a liability in production. When something goes wrong, it’s hard to determine why one agent was invoked instead of another.

Microsoft Agent Framework — Enterprise-focused and deeply integrated with Azure, consolidating what were previously Semantic Kernel and AutoGen into one supported path. We haven’t used it in production, but it’s the likely choice for clients already committed to the Microsoft stack, where the integration with existing identity and governance is worth more than framework ergonomics.

Layer 4: models — horses for courses

The “one model to rule them all” era is over. Production agent systems typically use two or three tiers:

  • A fast, cheap model for routing and classification
  • A frontier reasoning model for complex decisions and multi-step planning
  • A domain-specific or small model for specialised and edge-deployed tasks

We describe those as tiers rather than naming versions on purpose. Every specific model we would have named in this list when we first drafted it has since been superseded, and the architecture did not change when they were swapped — which is the point. Build the routing boundary, treat the model behind it as a replaceable part, and keep an eval set that tells you whether a swap made things worse.

The key metric isn’t accuracy per model — it’s cost per completed task. A system that routes 80% of requests to a fast model and 20% to a reasoning model costs a fraction of one that sends everything to the expensive model.

Layer 5: observability — you can’t debug what you can’t see

Agent systems are non-deterministic. The same input can produce different tool-calling sequences, different intermediate results, and different final outputs depending on model temperature, tool response time, and context window state.

Debugging this without observability is like debugging a distributed system without logs — technically possible, practically insane. LangSmith has become our standard. It traces every agent run end-to-end: model calls, tool invocations, intermediate reasoning, costs, and latencies.

The critical capability is eval-driven development. We maintain evaluation datasets for each agent system and run them on every deployment. If accuracy drops below a threshold, the deployment stops. This is the agent equivalent of unit tests, and it’s the single most important practice for shipping reliable agent systems.

The economics: FinOps for agents

Running agent fleets that make thousands of LLM calls daily gets expensive fast. The Plan-and-Execute pattern — where a capable model creates a strategy and a cheaper model executes it — cut our API costs by 60% on one project.

Other patterns that help:

  • Semantic caching: If the same question has been asked before, return the cached answer instead of re-running the agent. We use Redis with embedding-based similarity matching.
  • Token budgets per run: Each agent invocation has a maximum token spend. If the agent is spiralling (calling 40 tools instead of 4), the run terminates with a partial result.
  • Model tiering: Route requests based on complexity. A “what’s the weather?” question doesn’t need a frontier reasoning model.

The gap is engineering, not capability

The companies we work with have moved past “should we build agents?” and into “how do we build agents that don’t break in production?” That shift is the real story of the year, and it is not a story about model capability. Models are good enough. The gap is in orchestration that handles failures gracefully, observability that makes non-deterministic systems debuggable, cost controls that keep budgets predictable and evaluation frameworks that catch regressions before users do.

We are deliberately not quoting an analyst adoption forecast here, though there are several in circulation and they get cited constantly. We have no way to audit those numbers, and the argument does not need them: if you are running agents in production you already know whether the hard part is the model or the plumbing.

That’s the agent stack in 2026. It’s not one framework or one model. It’s a discipline — protocols (MCP, A2A), orchestration (LangGraph), operational tooling (LangSmith) and engineering practices (evals, FinOps, tiered routing) that together make autonomous systems reliable enough to trust in production.

What we’d do differently

We spent the first quarter of the year evaluating orchestration frameworks and roughly a fortnight building eval sets, and the ratio should have been the other way around. Framework choice turned out to be reversible — we migrated one system between frameworks in under a week — while the absence of a decent eval set meant we could not tell whether either version was actually better. The thing we treated as the big architectural decision was the cheap one.

References

We build production agent systems as part of our AI & Machine Learning Engineering practice. Talk to us about building agents that actually ship.