Handling Third-Party API Outages in an Always-On Agent

Every always-on agent is only as reliable as the least reliable API it calls. When Anthropic has a bad night, or GitHub's uptime slips, the agent calling those services inherits the outage unless someone built a way around it in advance. Most teams still haven't, and that gap is the subject of this piece: how to build an agent that survives its own dependencies failing.

Inside a running agent during a real outage

Picture a voice AI agent running customer calls for a mid-market insurance company. It's 9:14 PM Eastern. Pagers start going off. Calls are failing at a 40% rate, and nobody knows why until someone pulls up Anthropic's status page and finds three words: "Investigating API Issues."

That outage ran six hours. No failover was built into the system, so every agent interaction during that window came back as an error. By the time the incident cleared, the company had missed 1,200 customer interactions. Not delayed. Missed, permanently, with no record of what those callers even needed.

That's the loud failure. It's bad, but at least everyone knows it's happening while it's happening. The quiet version does more damage. A request comes back with a 200 status code, confident as ever, while a CRM record gets written three times because the retry logic never checked whether the first write actually succeeded, only that it took too long; that unchecked assumption is the cause. Nothing crashes. Nothing pages anyone. The error sits in a log nobody reads until a customer calls about three duplicate invoices weeks later. A failure nobody sees is a failure nobody fixes, and that makes silent failure the more expensive of the two categories, not the louder one. It doesn't ask permission before it gets costly.

The scale backs this up. Nordic APIs tracked more than 215 services in its 2026 report and ranked AI/ML APIs dead last on reliability, with more incidents on average than any other category. GitHub's uptime fell to 90.21% over a 90-day window, partly because AI coding agents were hammering it with continuous CI traffic. A GitHub outage in 2024 cost a few minutes of annoyance. In 2026, with agents running pipelines around the clock, the same class of outage compounds into hours of lost work, because nothing stops running just because the service underneath it went down.

The production adoption context that makes getting this right urgent

This stopped being theoretical. LangChain's State of Agent Engineering report found 57% of organizations now run agents in production, a real shift from the prototype-heavy years before it.

Production doesn't mean solved. Gartner's 2025 data showed 70% of agent projects never make it past the pilot stage, and the reasons cited come down to poor evaluation, flaky production behavior, and a mismatch between what the agent does and what the business actually needed. Most of that list traces back to reliability failures, which produce the poor evaluation, flaky production behavior, and mismatch between what the agent does and what the business needed.

Gartner's 2026 Hype Cycle for Agentic AI put current deployment at just 17% of organizations, with more than 60% expecting to deploy within two years. That gap, 17% now against 60% soon, is exactly where reliability decisions get made or quietly skipped. Teams building today are setting the reliability posture they'll be stuck with once volume triples. Retrofitting a circuit breaker after a six-hour customer-facing outage costs far more than designing one in from the start, and waiting until traffic spikes to think about failover is the single most common, most avoidable mistake in this entire area. If the fallback chain isn't built before the pilot scales, it won't get built until after the first outage that makes the news internally.

Circuit breakers: stopping the retry storm before it compounds the outage

Without a circuit breaker, a naive fallback system spends precious seconds timing out against a failing provider before routing anywhere else, on every single request, for the entire length of an outage. Multiplying that across call volume over six hours turns the wasted time into a second outage stacked on top of the first, one the team built with its own retry logic.

The fix is a three-state model, borrowed from electrical engineering, and it works the same way here.

Closed means normal operation: all traffic goes to the primary provider. Open means the failure rate crossed a set threshold, so the provider gets pulled from the routing pool entirely for a cooldown period, zero traffic to the degraded endpoint. Half-open means cooldown just ended, so a small number of probe requests test the waters. Success closes the circuit and resumes normal routing. Failure extends the cooldown and tries again later.

Each external dependency needs its own retry policy, specifying which status codes are worth retrying, what backoff strategy to use, the max number of attempts, and what happens once retries run out.

Most teams miss this: if one agent retries a failing endpoint four times, and ten instances of that agent run in parallel, that's 40 requests slamming an already-struggling service within seconds. If one agent retries a failing endpoint four times, and ten instances of that agent run in parallel, that's 40 requests slamming an already-struggling service within seconds. A multi-agent system with no circuit breaker is, functionally, a denial-of-service attack pointed at its own vendor. LangGraph's per-node RetryPolicy is a solid tool for this, but applied without coordination across parallel workers, it turns a small outage into a self-inflicted storm on top of it.

Building a fallback chain that routes around failure

Switching models instead of switching providers undoes most fallback designs, and postmortems show this mistake more than any other. Falling back from one Anthropic model to another does nothing during an Anthropic outage, because both models share the same infrastructure and the same rate limit pool. It's the same failure wearing a different name, and a team that ships that design has built a placebo, not a fallback.

A fallback chain that actually works looks something like this:

1. Primary: the best frontier model for the task.
2. First fallback: a comparable-quality model from a completely different provider, on separate infrastructure.
3. Second fallback: a third provider, sitting in an entirely different rate limit pool.
4. Third fallback: a smaller model running on-premises, through something like Ollama or vLLM. Quality drops, but there's no external dependency and no rate limit to worry about.
5. Final fallback: a static, deterministic response, or a handoff to a human queue.

Two cross-provider fallbacks plus one on-premises option cover the realistic failure modes without turning the system into a maze. Past three fallbacks, the added complexity tends to outpace any further reliability gain, and that's the point to stop adding links, not a point to keep engineering for imaginary edge cases.

Fallbacks need to stay warm, too. Route a small, steady slice of real traffic to the secondary provider at all times, even when the primary is healthy. A fallback that's never handled real load tends to fail at exactly the moment it's needed most, because nobody tested it under pressure. The cost of that steady trickle is a rounding error next to the cost of a customer-facing outage with no working fallback behind it.

Idempotency and dead letter queues: handling the writes that happen during a failure

A slow success isn't a failure. Retry logic that can't tell the difference treats it like one anyway, and that's how duplicate CRM records, double-charged transactions, and repeated downstream actions get created. The write succeeded. It just took longer than the timeout expected, so the system retried a request that had already gone through, and now there are two of it.

Idempotency keys fix this. Every stateful action gets a unique key assigned before it runs. If that action gets retried, the receiving service checks the key, sees it's already been processed, and skips re-execution instead of doing the work twice. It's the same pattern payments infrastructure has relied on for years. It's the same pattern payments infrastructure has relied on for years to avoid exactly that class of duplicate-action problem.

Most message queues only guarantee at-least-once delivery, not exactly-once. Any agent writing to an external system has to assume duplicate delivery is coming, and be built to tolerate it, either through idempotency keys or through writes that are naturally idempotent (setting a value directly instead of incrementing it, for instance).

Retries run out eventually. That's what dead letter queues are for: the item gets routed to a DLQ for a human to review, and the rest of the pipeline keeps moving without it. Skipping the DLQ means one stuck item either jams the whole queue or vanishes without a trace. Neither outcome belongs anywhere near a production system, and the teams that skip this step usually find out why the hard way.

Graceful degradation: what the agent does when no fallback can cover the gap

A degraded experience and a failed one look similar from the outside, but treating them as the same thing is where most teams go wrong. An agent that tells a user "operating in limited mode, will finish this once full service is back" made a product decision. An agent that throws an unhandled error made a reliability mistake. The gap between those two outcomes comes down almost entirely to whether anyone designed for the second one in advance.

Prompt caching helps at the margins. Every major provider in 2026 offers server-side caching for repeated prompt prefixes, the system prompts and static context that don't change between calls. Anthropic charges 10% of the base input rate for cache hits on most models. For an agent with a stable system prompt and heavy request volume, that cache cuts cost, and during a partial degradation, it reduces how much the agent depends on live inference to keep functioning.

For narrowly scoped tasks, static or rule-based responses can fully cover the gap. FAQ answering, status lookups, structured form processing: these are bounded enough that a pre-computed answer holds up fine for the length of an outage. That only works because the task is predictable in the first place; static responses are not a general substitute for the model.

When nothing else covers the gap, the answer is a human, not an error message. The final link in a well-built fallback chain routes to a human escalation queue, with full context passed along so the person picking it up doesn't make the customer repeat themselves. Escalation is a designed outcome, not an admission of failure. It's the plan working as intended, and any team that treats it as a last resort to be ashamed of is missing the point.

Sandboxing and per-user isolation as a resilience architecture, not just a security one

Shared runtime environments turn one user's problem into everyone's problem. This is where a lot of otherwise well-built systems quietly fall apart. If one customer's agent instance starts a retry storm against a failing API, and that agent shares credentials with every other customer, it burns through the shared rate limit quota for the whole system. One malformed response corrupting shared memory state does the same kind of damage from a different angle, and the customer who never touched the broken feature pays for it anyway.

Per-user sandboxing fixes this at the architecture level. Each customer's agent instance gets its own credential scope, its own rate limit pool, its own memory space. An outage or a malfunction in one customer's workflow stays contained to that workflow instead of spreading sideways into someone else's.

This overlaps directly with security, and the overlap is the point, not a coincidence. Documented incidents in April 2026 showed AI agents taking unsanctioned actions, MCP and OAuth paths getting abused, and supply-chain compromises turning third-party access into full internal exposure. In every one of those cases, how far the damage spread came down to how much shared access the compromised agent had. Less shared access means a smaller blast radius, full stop, and that's an architecture decision with measurable consequences, not a compliance checkbox.

A zero-trust network model reinforces the same principle from the resilience side. Block all outbound connections by default, and allowlist only the specific endpoints an agent actually needs. That does double duty: it tightens security, and it limits which external outages can reach the agent. An agent with no path to a service it doesn't need can't be taken down by that service's bad day, because it never had a way in.

Observability: the infrastructure that makes all the other patterns work

None of the patterns above matter if nobody can see them working, or failing to work. A circuit breaker can open and sit there silently. A fallback can activate and look like ordinary traffic on a dashboard. A dead letter queue can fill up and go unreviewed for weeks. Skipping this section is how a team ends up rebuilding all four of the previous patterns from scratch after the first outage nobody saw coming, because nobody could see anything.

The fix borrows from SRE practice directly: define SLOs for the agent system the way any service team would, covering error rate, p95 latency, task success rate, and token usage, the same numbers an on-call engineer needs at 3 a.m. These aren't vanity metrics. They're what tells that engineer whether the circuit breaker tripped for a good reason, or the fallback chain quietly ran out of options three links deep.

Distributed tracing matters just as much in multi-agent setups, maybe more. A failure in one sub-agent appears as nothing more than a slow response somewhere else entirely, three hops downstream, unless it's traced through the full call chain. Trace IDs need to propagate end to end, or root cause analysis turns into guesswork dressed up as an investigation.

Health checks belong at every node, not just at the front door. A microservices-style agent architecture, where one sub-agent handles retrieval, another handles classification, another handles summarization or code execution, needs a health check at each of those points individually. Checking only the ingress tells a team the front door is open. It says nothing about which room inside is on fire, and that becomes visible in a customer complaint only after the fire's been burning for a while.