Long Context in Practice: Claude, GPT, and Gemini Context Windows

Compare context limits, request schemas, thinking token consumption, and state retention across Claude, GPT-6, and Gemini API architectures.

By ailog · Published Oct 19, 2026 · 7 min read · How we work
In short
  • Gemini 4 shares a unified 1,048,576 token pool across inputs, thinking, and outputs, while earlier versions cap outputs at 64K.
  • Claude models feature context windows of either 1M or 200k tokens, generating up to 128k output tokens with automatic thinking block handling.
  • OpenAI provides stateless continuation via the Responses API using previous response IDs, encrypted reasoning replay, or persistent Conversations objects.
  • Large input buffers risk context degradation and recall drops on multi-target queries despite high single-needle retrieval benchmarks.

Context Window Architecture and Token Retention Mechanisms

Every language model handles context as an active working memory rather than permanent storage. When making API calls, the active window bounds all text the system evaluates during inference, including past turns, instructions, system prompts, embedded media, and generated completion tokens. As conversations expand over sequential interactions, maintaining accurate recall becomes progressively difficult. Token volume increases can cause context rot, where the model misses nuanced details or displays diminished reasoning accuracy over complex inputs.

Anthropic, OpenAI, and Google structure this transient state through distinct architectural paradigms. In Anthropic models like Claude Sonnet 5.5 and Claude Opus 5.5, the context window accumulates previous user turns, assistant responses, and tool telemetry into an expanding buffer. To counter context exhaustion, developers can implement server-side compaction for Claude 4.6 and later releases, which condenses earlier conversational history automatically. Prompt cache prefixes also inhabit this capacity; while prompt caching reduces token fees, cached tokens continue to occupy physical context space.

Google designs the Gemini family around massive default input spans, moving from early systems that accepted 8,000, 32,000, or 128,000 tokens to architectures supporting 1 million tokens. With Gemini 4 (gemini-4-argon), Google introduced a unified capacity where input, reasoning, and output tokens draw from a single pool of 1,048,576 tokens. Conversely, OpenAI treats individual generation runs as stateless queries. Rather than requiring continuous raw prompt transmission, the Responses API allows teams to anchor context using persistent objects via the Conversations API or through chained requests linked by a previous response identifier.

Context Window Capacities and Allocation Limits Compared

Maximum input limits and output parameters vary widely across active model generations. For Claude, flagships such as Claude Fable 5.1, Claude Mythos 5.1, Claude Opus 5.5, Claude Sonnet 5.5, and Claude Haiku 5.5 deliver a 1M context window natively without requiring special headers. Each can produce up to 128k output tokens via max_tokens. Other models, such as Claude Sonnet 4.5, operate within a 200k context window. Document and visual quotas also vary: 1M models accept up to 600 PDF pages or images per request, whereas 200k models limit inputs to 100 media elements.

Model Context Window Maximum Output Tokens Thinking / Reasoning In Context
Claude Sonnet 5.5 1M tokens 128k tokens Preserved by default across turns
Claude Haiku 4.5 200k tokens Model context window limit Stripped automatically on subsequent turns
GPT-6 Astra Context window bound Model defined Encrypted reasoning replayed via output array
gpt-4o-2024-08-06 Model defined 16,384 tokens Not applicable
Gemini 4 (gemini-4-argon) 1,048,576 tokens (shared) Up to 1M tokens (long decode) Shared with input and output pool
Gemini 2.5 Pro 1M tokens 64K tokens Dedicated output budget

Gemini 4 alters output budgeting: if the sum of input tokens and max_output_tokens exceeds 1,048,576, the Gemini API automatically curtails the maximum output limit to fit the leftover context space. Previous releases, including Gemini 2.5 Pro, Gemini 2.5 Flash, Gemini 3.5 Flash, and Gemini 3.8 Flash, isolate input capacity from a dedicated 64K output cap. For Claude 4.5 and newer versions, providing a prompt where inputs plus max_tokens exceed total capacity causes the API to accept the run and halt output with stop_reason: "model_context_window_exceeded" when capacity is exhausted. If the prompt alone exceeds the total context window, the Claude API immediately returns a 400 invalid_request_error.

Multi-Turn State and Request Construction

Preserving multi-turn context without sending ballooning payloads requires distinct API designs across providers. In OpenAI systems, multi-turn conversations can be built by explicitly appending prior assistant and user items into the input array. However, to bypass passing full message arrays repeatedly, developers can chain turns using previous_response_id. When using this parameter, past inputs are still billed as input tokens, but the developer avoids manually marshalling complete historical dialog structures over HTTP.

response = client.responses.create(
    model="gpt-6-astra",
    input="tell me a joke",
)

second_response = client.responses.create(
    model="gpt-6-astra",
    previous_response_id=response.id,
    input=[{"role": "user", "content": "explain why this is funny."}],
)

For state persistence across distributed worker threads or multiple client devices, OpenAI exposes the Conversations API. Attaching a conversation ID persists state without the standard 30 days retention expiration applied to raw response items (which default to store: true). In the Responses API WebSocket mode, state is maintained in a low-latency connection-local cache where lanes keyed by stream_id manage lineage through previous_response_id.

In Claude architectures, context tracking can be managed natively by the model. Claude Sonnet 5, Claude Sonnet 4.6, Claude Sonnet 4.5, and Claude Haiku 4.5 feature native context awareness. The Anthropic API injects system parameters indicating remaining headroom throughout the execution cycle:

<budget:token_budget>200000</budget:token_budget>
<system_warning>Token usage: 35000/200000; 165000 remaining</system_warning>

Claude Opus 4.7, Claude Sonnet 5.5, Claude Haiku 5.5, Claude Fable 5.1, and Claude Mythos 5.1 do not receive these injected XML tags; developers using these releases can instead configure beta task budgets. To measure context before submission across providers, consult the token counting APIs to avoid unexpected HTTP 400 rejection errors.

Reasoning Tokens and Tool Payloads in Context

Advanced reasoning mechanisms consume context window allocations rapidly. In Claude, thinking tokens deduct from max_tokens and count directly toward context capacity. Whether earlier reasoning blocks persist in subsequent dialogue steps depends on the selected release. Models like Claude Opus 4.5, Claude Sonnet 4.6, Claude Sonnet 5.5, Claude Haiku 5.5, and Claude Fable 5.1 retain past thinking blocks by default, billing them as input tokens on subsequent requests. Conversely, earlier variants and Claude Haiku 4.5 automatically strip previous thinking blocks from incoming conversational histories.

When pairing thinking with tool execution in Claude, developers must return thinking blocks alongside the corresponding tool_result items. Claude verifies these thinking structures via cryptographic signatures, returning errors if the payload has been edited. Once the tool call cycle completes and a new user message is introduced, models that strip thinking drop those earlier reasoning blocks to conserve context space. For models that retain thinking, developers can purge historical reasoning blocks manually using thinking block clearing. Interleaved thinking allows Claude to reason between tool executions; this capability functions automatically with adaptive thinking, while Claude Opus 4.5 and Claude Sonnet 4.5 require the interleaved-thinking-2025-05-14 header.

For OpenAI reasoning models like GPT-6 Astra and GPT-5.5, context preservation requires replaying the complete output array in subsequent requests. This ensures encrypted reasoning items and the assistant message phase parameter stay intact. If an application drops the phase attribute, models such as GPT-5.5 may misinterpret intermediate updates as terminal responses. When persisting reasoning, developers can specify reasoning.context: "all_turns" to inject past reasoning steps into the subsequent sample.

Pricing, Retrieval Accuracy, and Context Rot

Deploying 1 million tokens unlocks in-context learning—such as Gemini learning translation from English to Kalamang using a 500-page reference grammar, a dictionary, and 400 parallel sentences. In code analysis, 1M tokens accommodates 50,000 lines of code at 80 characters per line. However, developers must consider retrieval accuracy degradation. While Gemini achieves ~99% accuracy on single-needle retrieval evaluations, recall decreases when queries target multiple separate needles across massive context buffers. Extracting 100 distinct items with 99% accuracy often requires splitting the task across 100 individual requests rather than querying a single monolithic prompt.

Provider pricing structures reflect the computational demands of long-context workloads. Baseline pricing verified on 2026-10-01 highlights substantial cost differences across model tiers:

Model Input / 1M Tokens Cached Input / 1M Tokens Output / 1M Tokens
Claude Sonnet 5.5 $2 $0.2 $10
Claude Haiku 4.5 $1 $0.1 $5
GPT-6 Astra $10 $1 $50
GPT-6.1 Sol $2 $0.1 $10
GPT-6 Luna $0.1 $0.01 $0.5
Gemini 3.8 Flash $0.75 $0.075 $3.75
Gemini 2.5 Flash $0.3 $0.03 $2.5
Gemini 2.5 Flash-Lite $0.1 $0.01 $0.4

Pricing mechanics require careful monitoring at scale. On Claude Haiku 5.5, prompts exceeding 100,000 tokens incur higher rates than the base tier. On Gemini Flash, utilizing context caching drops input expenses by ~4x compared to un-cached inputs. To calculate operational spend across workloads, consult the /tools/llm-api-cost-calculator/ or explore tier variations at /tools/llm-api-pricing/. Furthermore, placing queries at the end of the context window yields better performance than positioning questions at the start.

Implementation Checklist for Long Context Applications

Before scaling input prompts toward provider context ceilings, run through these implementation verification steps:

  • Query Placement: Position instructions and specific user queries at the end of the input context rather than the beginning.
  • Output Parameter Adjustments: In Gemini 4 (gemini-4-argon), verify that the combination of input tokens and max_output_tokens does not unintentionally constrict output generation within the 1,048,576 shared limit.
  • Reasoning Token Handling: Ensure thinking blocks are preserved intact with their cryptographic signatures during Claude tool result turns, or manage token growth via thinking block clearing.
  • OpenAI State Hygiene: Maintain the phase property and encrypted reasoning items across turns, or link turns via previous_response_id to prevent GPT-5.5 from misinterpreting intermediate outputs.
  • Cost Thresholds: Account for tiered pricing jumps, such as prompts exceeding 100,000 tokens on Claude Haiku 5.5, and leverage context caching for static document collections.
  • Graceful Overflow Handling: Handle stop_reason: “model_context_window_exceeded” in Claude 4.5 and newer models, or include the model-context-window-exceeded-2025-08-26 beta header on legacy endpoints.
Sources
  1. Context windows
  2. Conversation state
  3. https://ai.google.dev/gemini-api/docs/long-context

Related