Book a 30-min call
← cd ../blogs
$ cat posts/mcp-in-production-building-agentic-ai-systems.mdx

MCP in Production: Building Agentic AI Systems That Actually Ship

April 17, 2026 · updated August 8, 2026 · ImmovableTech Team

  • MCP
  • Agentic AI
  • Production AI

The protocol that changed how we build AI

If you’ve been building AI systems for the past year, you’ve probably noticed a shift. The conversation moved from “which model should I use?” to “how do I connect my model to everything else?” That’s the gap MCP fills.

Model Context Protocol — originally Anthropic’s, donated to the Linux Foundation’s Agentic AI Foundation in December 2025 — has become the default way AI agents talk to tools, databases and data sources. At the donation it was cited at over 97 million monthly SDK downloads across roughly 10,000 active servers; by July 2026 the two core SDKs alone were running far past that, with the npm TypeScript SDK at roughly 156 million monthly downloads and PyPI’s mcp package at roughly 271 million. Client support spans ChatGPT, Claude, Cursor, Gemini, Microsoft Copilot and VS Code. It’s boring plumbing in the best possible sense.

We’ve shipped MCP-based systems to production for three clients so far. Here’s what we learned that doesn’t show up in the docs.

What MCP actually does (and doesn’t do)

MCP is a JSON-RPC protocol that standardises how an AI model calls external tools. Before MCP, every integration was custom: you’d write a function, register it with your framework, handle auth, parse the response, and pray the model called it correctly. With MCP, tool providers publish servers, and any MCP-compatible client can discover and call those tools.

What MCP doesn’t do: it doesn’t orchestrate. It doesn’t decide which tools to call, in what order, or what to do with the results. That’s where frameworks like LangGraph come in.

Put differently: MCP defines how tools expose themselves. LangGraph decides when and in what order to use them.

Our production stack

After three deployments, we’ve settled on a stack that works:

Agent Orchestration:  LangGraph (deterministic state machines)
Tool Protocol:        MCP (JSON-RPC over Streamable HTTP)
Models:               a small fast model for routing, a frontier
                      reasoning model for planning and hard decisions
Observability:        LangSmith (traces, evals, cost tracking)
Serving:              FastAPI + Redis queues

We’ve deliberately stopped naming specific model versions in our architecture docs. Over the life of these three deployments the specific models in those two slots changed four times, and every time we’d written a version number into a diagram it became a lie that someone later had to chase. What matters architecturally is that there are two tiers with different cost and latency profiles, and that the routing decision between them is ours rather than the vendor’s.

We tried CrewAI early on. It’s great for demos — you can spin up a multi-agent system in 20 lines. But in production, we needed explicit control over agent handoffs, retry logic, and state persistence. LangGraph’s graph-based approach gave us that. When an agent fails mid-pipeline, we know exactly which node failed, what state it was in, and can retry from that point.

The bits the docs skip

Tool registration is the easy part; schema design is not

Registering an MCP tool takes five minutes. Designing a tool schema that models actually use correctly? That takes days. We learned this the hard way on our restaurant intelligence platform.

Our first version of the NL-to-SQL tool had a parameter called query that accepted a free-form SQL string. The model generated valid SQL about 60% of the time. When we restructured the tool to accept structured parameters — table_name, columns, filters, group_by — accuracy jumped to 94%. The model was fine at reasoning; it was bad at writing raw SQL from scratch.

The lesson: design tool schemas for the model, not for a human developer. Models work better with constrained, structured inputs than open-ended strings.

Multi-agent ≠ better

Most systems that get built as multi-agent should have been single-agent. We’ve built multi-agent pipelines, and the honest truth is: they’re harder to debug, slower to execute, and only justified when tasks genuinely require different capabilities.

Our hallucination detection pipeline uses five agents because claim extraction, source retrieval, and factuality scoring are genuinely different tasks that benefit from specialisation. Our content generation system started with four agents and we merged it down to two — the “editor” and “reviewer” agents were doing overlapping work and adding 3 seconds of latency for no accuracy gain.

The rule of thumb: start with one agent. Add another only when you can prove (with evals) that splitting the task improves output quality enough to justify the added complexity and latency.

Observability is not optional

Non-deterministic systems are terrifying to operate without observability. When a traditional API returns a wrong answer, you check the logs and find the bug. When an agent returns a wrong answer, the “bug” might be a model hallucination, a tool returning unexpected data, a prompt that doesn’t handle an edge case, or a combination of all three.

LangSmith has been essential for us. We trace every agent run end-to-end: which tools were called, what the model “thought” (its chain-of-thought), what each tool returned, and how long each step took. When something goes wrong, we can replay the exact sequence and identify whether the issue was in the model’s reasoning, the tool’s response, or our prompt.

The cost tracking alone justified the tool. One client’s agent was making 40 tool calls per request instead of the expected 4-6, burning through their API budget. We only caught it because LangSmith showed the call pattern.

A2A: the other protocol you’ll need

MCP handles agent-to-tool communication. The Agent2Agent (A2A) protocol, originally Google’s and donated to the Linux Foundation in June 2025, handles agent-to-agent communication. They’re complementary, and both now sit under Linux Foundation governance.

We haven’t deployed A2A in production yet, and the reason is not the spec — it’s that none of our three systems has a genuine cross-organisational agent boundary. A2A earns its keep when Agent A and Agent B are operated by different teams or different companies and cannot share process state. Ours are all inside one deployment, where LangGraph’s node-to-node state passing is strictly better: typed, in-process, and debuggable in one trace.

That’s worth saying plainly because the default assumption in a lot of writing right now is that multi-agent means A2A. It doesn’t. Adopt the protocol when you have the organisational boundary that makes it necessary, not because your diagram has two boxes in it.

Update: the spec went stateless

We wrote this in April. In the MCP revision dated 28 July 2026, the protocol finalised its transition to a fully stateless architecture and introduced a formal feature lifecycle policy with a 12-month deprecation window. The authorisation changes in the same revision are incremental rather than an overhaul — the changelog lists them as minor: iss validation per RFC 9207 to close an authorisation-server mix-up hole, credentials bound to their issuer, and dynamic client registration deprecated in favour of Client ID Metadata Documents.

If you’re reading this as a build guide, the stateless change touches one of our assumptions materially. Our early servers held per-session state on the server side, and the stateless model means session context belongs with the client or in an explicit store you control. It is a better design — stateless servers scale horizontally without sticky routing, which was a real operational annoyance for us — but it is a migration, not a free upgrade, and we have written up what that migration actually involves separately.

The underrated item is the lifecycle policy. A formal deprecation window is what makes it defensible to build a production dependency on a protocol at all, and it matters more for planning than any single feature in the release.

What we’d do differently

If we started our first MCP project today instead of in late 2025:

  1. Remote, stateless MCP servers from day one. We started with local stdio-based servers and had to migrate to remote HTTP servers for production, then had to unpick server-side session state a second time. The ecosystem has converged on remote, OAuth-secured, stateless servers. Start there and save yourself both migrations.

  2. Structured tool outputs, not free text. Our early tools returned plain text descriptions. Models parse structured JSON far more reliably. Every tool should return typed, schema-validated responses.

  3. Cost budgets per agent run. We now set a maximum token spend per request. If the agent exceeds it, the run terminates gracefully with a partial result rather than spinning indefinitely. This should have been in place from the start.

  4. Evaluation datasets before architecture. We spent two weeks building an agent pipeline before we had a way to measure if it worked. Now we build the eval set first, always.

Where we’ve actually landed

MCP is real infrastructure, not a trend. The protocol is stable, the tooling is maturing fast, and the big labs are all-in. If you’re building AI systems that have to touch real databases, real APIs, real file systems, or real internal tools, MCP is the honest way to connect them.

But the protocol is just the plumbing. The hard work sits in schema design, orchestration, evaluation and observability. Those are engineering problems, not API problems — and they’re where production systems quietly succeed or quietly fall over.

References

We build MCP-based agentic systems as part of our AI & Machine Learning Engineering practice. Talk to us if you’re building something similar.