Book a 30-min call
← cd ../blogs
$ cat posts/agent-harness-build-vs-adopt.mdx

The agent harness: build the loop, adopt the sandbox

July 7, 2026 · ImmovableTech Team

  • Agentic AI
  • Production AI
  • Security

The layer everyone rewrites

Every team we have worked with in the past year has written an agent harness, and almost none of them decided to. It accumulated. Somebody added a retry, somebody else added a step limit because a run went to 200 tool calls overnight, somebody added a token budget after the first invoice, and six months later there is a thousand-line module nobody owns that sits between the model and everything the model can touch.

Microsoft’s own framing of the harness, from the walkthrough they published in June, is a good one: a loop around a language model wired up with tools, planning, memory, approvals and observability. That is accurate, and it is also why the build-versus-adopt question feels unresolvable. Those are not one component. They are at least four, with genuinely different answers, and teams keep trying to settle them with a single decision.

Our answer, after building this layer three times: own the loop, adopt the isolation, adopt the telemetry standard rather than a telemetry vendor, and accept that context assembly was never going to come from anyone else.

Note that this is a different question from which orchestration framework to use, which we have written about separately. Framework choice is largely reversible — we have migrated a system between two of them inside a week. The decomposition below is not, because it determines which parts of your system you are able to change at all.

Own the loop

The loop is small. Ours is under 300 lines including the type definitions, and the part that actually matters is about twenty.

from dataclasses import dataclass


@dataclass
class Budget:
    max_steps: int = 12
    max_input_tokens: int = 200_000
    steps: int = 0
    tokens: int = 0


def run(model, tools, messages, budget):
    """`model` handles provider retries; `dispatch` runs tools in the sandbox."""
    while budget.steps < budget.max_steps:
        budget.steps += 1
        reply = model(messages)
        budget.tokens += reply.input_tokens
        if budget.tokens > budget.max_input_tokens:
            return 'token_budget', messages
        if not reply.tool_calls:
            return 'done', messages + [reply.message]
        messages += [dispatch(tools, call) for call in reply.tool_calls]
    return 'step_budget', messages

Nothing there is clever. What it encodes is a set of product decisions: how many steps is too many, what happens when the budget runs out, whether a run that hits the ceiling returns a partial result or an error, and whether tool calls in one turn execute in parallel or in sequence. Every framework has an opinion on all four. The opinions are reasonable defaults and they are eventually wrong for you, because they were chosen for the median case and you are not the median case.

The specific place this bit us was budget exhaustion. Most harnesses treat hitting a limit as a failure and raise. For a document-processing pipeline that is correct. For an interactive assistant it is the worst possible behaviour — the user watched it work for forty seconds and got an exception. We wanted a partial result with an explicit reason attached, surfaced in the interface as “I ran out of budget, here is what I have”. Getting that out of a framework meant catching an exception thrown from inside a runner we did not control and reconstructing state we should have owned in the first place.

What owning the loop actually costs

This advice is easy to abuse, so here is the bill.

Owning the loop means owning retry semantics against every provider you use, and they do not agree. Rate-limit responses differ in whether they tell you how long to wait. Overload conditions come back with different status codes and different retryability. The thing that cost us most was streaming: reassembling tool-call arguments from partial deltas, where the argument JSON arrives in fragments across chunks and each provider fragments it differently. That took a fortnight, and it shipped with a bug. A stream that terminated mid-arguments left us with truncated JSON, our parser failed, our retry logic re-ran the turn, and the model re-issued a tool call that had in fact already executed. On a read tool that is invisible. On a write tool it is a duplicate record, which is how we found it.

That is a fortnight plus an incident, for a problem that every framework has already solved. If your agent is an internal batch job with one model provider and no streaming, you should not be paying that. We wrote a harness for exactly that kind of tool once and it was straightforwardly wasted effort — the control flow was “call the model, call one tool, stop”, and any framework’s default would have done.

Owning the loop pays when the control flow is the product. Our hallucination detection work has a stage ordering that no generic runner expresses, and where a step that fails validation reruns with narrowed context rather than escalating. That is the thing customers are buying. It should not live inside someone else’s abstraction.

Sandboxing and egress are not your engineering

This is the part where the build option is not a trade-off, it is a mistake. Isolation is specialist security work, the failure mode is a breach rather than a bug, and you will not discover you got it wrong from your metrics.

The concrete evidence for how hard this is sits in the Kubernetes Agent Sandbox project’s own default network policy, merged in March 2026. The secure-by-default posture blocks egress to RFC1918 private ranges, to the node metadata server and to internal cluster DNS — the three paths by which code running in a sandbox reaches laterally into your infrastructure, steals node credentials or enumerates internal services. The interesting detail is the compensating change that had to come with it: because blocking internal DNS also breaks CoreDNS resolution, the controller injects dnsPolicy: None with public resolvers into the sandbox pod, and it gates that injection so it does not clobber corporate proxy or air-gapped configurations.

Read that again as a build estimate. A correct egress policy broke name resolution, and the fix had to be conditional on deployment topology. That is one requirement out of dozens, and it was found by people who do this full time. We had a sandbox before we had that policy — a container per tool call, shared host kernel, no egress rules — and it was fine in development, which is precisely the problem. The kernel boundary and the network boundary were both notional.

The adopt side is now genuinely good. GKE Agent Sandbox reached general availability on 20 May 2026 with gVisor as the default isolation runtime, Kata Containers available as a pluggable alternative for a full VMM boundary, default-deny network policy, and warm pools that Google reports allocating at 300 sandboxes per second per cluster with 90% of allocations under 200 milliseconds. The isolation granularity is getting finer, too: Microsoft’s CodeAct work runs model-generated code in a fresh Hyperlight micro-VM per tool call, though that package was still alpha when they announced it at Build in June.

One caution on the benchmark that ships with CodeAct. On Microsoft’s own multi-step sample — computing order totals across many users, dozens of tool calls — collapsing the chain into a single generated program cut wall-clock from 27.81 to 13.23 seconds and tokens from 6,890 to 2,489. That is a vendor benchmark on one workload chosen to favour the pattern, and it measures orchestration overhead, not answer quality. It is a real effect and it is not a 52% speedup on your agent.

Adopt the standard, not the vendor

Observability should be adopted, but the thing to adopt is the wire format, not the backend. Emit OpenTelemetry GenAI semantic conventions and the backend stays a swap.

Be accurate about what you are adopting, because a lot of writing on this is wrong. The GenAI conventions are not stable. As of the current spec text they are marked Development across the whole set — model spans, agent spans, metrics, events and the MCP conventions alike. They also moved: OpenTelemetry semantic conventions v1.42.0, released 12 June 2026, deprecated every gen_ai.* attribute, metric, event and span in the core repository and relocated them to a dedicated semantic-conventions-genai repository, with instrumentations directed there for the schema URL to use.

Development status means attribute names can change under you. That is a real cost and it is still the right bet, because the alternative is a proprietary trace schema whose names can also change under you, with no migration path and no second implementation. Instrument to the convention, treat your dashboards and alerts as code that will break on a rename, and pin the schema URL you emit.

The gap worth closing is not instrumentation coverage. LangChain’s State of Agent Engineering, published 12 June 2026 from 1,340 responses collected in late 2025, found 89% of respondents had some form of agent observability and 62% had step-level tracing, rising to 94% and 71.5% among teams already in production — while only 52.4% ran offline evals. Nearly everyone is watching. Half as many are checking. Traces tell you what the agent did; they do not tell you whether it was right.

Context assembly is the part that is actually yours

The remaining component is where the engineering value sits, and no framework can supply it, because it is a function of your data.

Deciding what goes into the window on turn seven of a long run — which retrieved documents, which tool results kept in full versus summarised, which earlier turns dropped, which system instructions merged in what order — depends on your corpus, your access model and your latency budget. A framework can give you a compaction hook. It cannot tell you that summarising a tool result is safe for search output and destructive for a schema listing, because it does not know your schemas.

This is the same lesson we hit with tool schema design in our MCP work: the registration is five minutes and the design is weeks. Context assembly has the same shape. It looks like plumbing, it is where accuracy comes from, and it is the one part of the harness that is unambiguously worth your best engineers.

What we’d do differently

Separate the four decisions on day one. We treated “should we use a framework” as one question and answered it once, which meant we inherited a sandbox we did not evaluate and wrote streaming code we did not need. Ask it four times.

Adopt isolation before you need it. Our container-per-call sandbox never leaked, as far as we know, and “as far as we know” is the entire problem. Managed sandboxes with default-deny egress existed before we moved, and the migration was a week of work we could have done at the start.

Write the budget and termination semantics down before writing the loop. Both times we rewrote the loop, it was because the exit conditions were implicit. What a partial result contains, who sees the reason and whether a truncated run is billable are product decisions, and discovering them from an incident is expensive.

Pin the telemetry schema version explicitly. We emitted whatever our instrumentation defaulted to, across services that upgraded on different schedules, and ended up with two attribute generations in one trace store. With a Development-status convention, the version you emit is a deliberate choice, not a default.

References

We design and operate agent runtimes as part of our AI & Machine Learning Engineering practice. Talk to us if you are deciding how much of your harness to own.