Handling API Errors and Retries for Claude, GPT, and Gemini

Learn how Claude, GPT, and Gemini surface API errors, handle retry headers, manage status codes, and configure SDK retry policies in production.

By ailog · Published Oct 30, 2026 · 5 min read · How we work
In short
  • Only retry transient status codes such as 429, 500, and 503; client errors like 400, 401, 402, and 403 will never succeed on replay.
  • Official Python SDKs differ in defaults: Claude retries twice, while Gemini retries up to four times across an initial 1-second delay up to 60 seconds.
  • Overload codes vary by provider: Claude emits 529 overloaded_error, OpenAI uses 503 service_unavailable_error, and Gemini surfaces 503 UNAVAILABLE.
  • Inspect the Retry-After response header whenever present, and apply exponential backoff with jitter if headers are omitted.

Understanding Provider Error Taxonomies

Production pipelines communicating with AI model backends must isolate transient communication hiccups from fatal operational faults. When an inference call fails, blindly resubmitting the payload creates resource exhaustion or compounds upstream downtime. Distinguishing between unfixable request formatting mistakes and temporary network congestion requires understanding each model platform’s status codes and JSON payload specifications.

Every vendor—Anthropic, OpenAI, and Google—structures error payloads around HTTP conventions, but specific status codes differ significantly for identical failure modes like model congestion or spend caps. Detailed rules on token usage and quotas can be explored in our breakdown on rate limits and usage tiers at Anthropic, OpenAI, and Google.

Provider Error Codes and Overload Handling Compared

While HTTP 400 and 401 universally designate request format problems and authentication errors, capacity shortfalls deviate across backends. Claude flags service capacity exhaustion using code 529 (overloaded_error). OpenAI denotes capacity crunches via 503 (service_unavailable_error with server_is_overloaded), while Gemini flags transient system unavailability via 503 (UNAVAILABLE).

Failure Scenario Claude (Anthropic) OpenAI (GPT) Gemini (Google)
Model / Server Overload 529 overloaded_error 503 service_unavailable_error (server_is_overloaded) 503 UNAVAILABLE
Rate / Token Limits 429 rate_limit_error 429 rate_limit_error (e.g. slow_down) 429 RESOURCE_EXHAUSTED
Depleted Credit / Billing 402 billing_error 429 credit_balance_exhausted 402 or 403 PERMISSION_DENIED (Payment needed)
Spend / Budget Cap Met 400 invalid_request_error or 429 429 organization_spend_limit_exceeded 403 PERMISSION_DENIED (Free tier restricted)
Request Entity Too Large 413 request_too_large 400 BadRequestError 400 Bad Request
Internal Backend Failure 500 api_error 500 InternalServerError 500 / 5xx Status Code

Spend limits present critical differences during parsing. When hitting an organization spend cap configured in your workspace, Anthropic returns an HTTP 400 invalid_request_error, though Claude Code workspace thresholds can return an HTTP 429. Tier spend-cap 429 errors omit the retry-after header and continue rejecting requests until access resets. OpenAI issues distinct 429 error codes: organization_spend_limit_exceeded, project_spend_limit_exceeded, and organization_usage_limit_exceeded. For Gemini, accounts falling into arrears show Payment needed and receive 403 PERMISSION_DENIED errors rather than standard 429 quota exhaustion.

Request Tracing and Payload Formats

Debugging failed inference attempts requires inspecting the machine-readable error bodies returned by each gateway. Claude returns an explicit JSON wrapper featuring the error classification, details, and tracking identifier.

{
  "type": "error",
  "error": {
    "type": "not_found_error",
    "message": "The requested resource could not be found."
  },
  "request_id": "req_011CSHoEeqs5C35K2UUqR7Fy"
}

Claude includes a request-id header on all transactions (such as req_018EeWyXxfu5pfWkrYcMdjWG), mirroring the body’s request_id. On Claude Platform on AWS, two keys appear: the primary AWS tracker x-amzn-requestid and the secondary Anthropic request-id. For OpenAI calls, errors provide parameters like error.type and error.code. If using WebSocket mode on the OpenAI Responses API, specific failures return string tags like previous_response_not_found or websocket_connection_limit_reached (which fires after 60 minutes).

Specific request validation errors also depend heavily on model generations. For example, Claude 4.6 and higher versions reject assistant message prefilling with a 400 invalid_request_error. Similarly, modifying thinking or redacted_thinking content blocks before sending turns back during tool execution returns a 400 error. On Gemini, parameters such as temperature, top_p, and top_k are ignored on Gemini 3.6+ and Gemini 4 models, returning deprecation warnings before becoming 400 errors in later updates.

SDK Retries and Backoff Strategies

Official client libraries come pre-configured with exponential backoff routines, though their boundaries and defaults diverge. The official Claude SDKs retry transient failures (connection issues, rate limits, and 5xx errors) twice by default, respecting retry-after header delays. This configuration can be adjusted across languages via client parameters such as max_retries in Python, maxRetries in TypeScript, .maxRetries() in Java, or option.WithMaxRetries() in Go.

Google’s python-genai client retries transient issues (429, 408, and 5xx errors) up to four times automatically. It starts with an initial wait time of approximately 1 second and expands exponentially up to a maximum delay of 60 seconds. When authoring custom retry loops, developers should introduce jitter to prevent synchronized retry waves across worker fleets.

Pacing traffic avoids sudden throttling spikes. When input volume approaches 1 million input tokens per minute (TPM) on OpenAI endpoints, traffic should increase by no more than 50% every 15 minutes to avoid a 429 response carrying the slow_down code. When analyzing total operational costs before provisioning capacity, run the figures through our /tools/llm-api-cost-calculator/ or verify unit rates on the /tools/llm-api-pricing/ reference page.

Connection Durations, Timeouts, and Streaming Errors

Standard HTTP requests that remain idle during generation risk silent termination. Cloud intermediary firewalls and load balancers drop idle sockets after variable periods. Anthropic enforces a 10-minute upper bound check inside its SDK for non-streaming Messages API calls and activates TCP keep-alive socket options.

For massive completions using models like Claude Sonnet 5.5 ($2 per 1M input tokens, $10 output) or Claude Opus 5.5 ($4 input, $20 output), Anthropic recommends the streaming Messages API or the Message Batches API. To inspect real-time events without parsing raw tokens, client SDKs offer accumulation helpers such as stream.get_final_message() in Python or stream.finalMessage() in TypeScript. Learn more about event contracts in our guide to streaming LLM responses across Claude, GPT, and Gemini.

When streaming over Server-Sent Events (SSE), failures can emerge well after the initial HTTP 200 OK header has been received. Middleware cannot catch these as standard HTTP status exceptions; your code must parse inbound mid-stream error frames directly from the message stream.

Implementation Checklist for Production Resilience

  • Check headers for Retry-After on all 429 and 503 responses, falling back to exponential backoff with randomized jitter if omitted.
  • Map provider-specific overload codes: 529 for Claude, 503 (server_is_overloaded) for OpenAI, and 503 (UNAVAILABLE) for Gemini.
  • Short-circuit and alert immediately on client errors (400, 401, 402, 403), because retrying without administrative intervention will always fail.
  • Preserve request IDs in logging: read the request-id header or _request_id property for Claude, or inspect both x-amzn-requestid and request-id on Claude Platform on AWS.
  • Avoid silent connection drops on operations over 10 minutes by switching to streaming APIs or batch endpoints.
  • Enforce token ramp-rate ceilings by keeping volume increases under 50% per 15-minute window once exceeding 1 million TPM on high-throughput GPT workflows.

Applying these checks ensures client applications survive upstream transient disruptions, preserve request tracing metadata, and halt immediately when facing non-recoverable configuration errors.

Sources
  1. Claude API errors
  2. Error codes
  3. https://ai.google.dev/gemini-api/docs/troubleshooting

Related