Effective context engineering
for AI agents
Context is a critical but finite resource for AI agents. Building with language models is becoming less about finding the right words for your prompts, and more about answering the broader question: what configuration of context is most likely to generate our model's desired behavior?
Thinking in context, not in prompts
Context refers to the set of tokens included when sampling from a large-language model. The engineering problem is optimizing the utility of those tokens against the inherent constraints of LLMs in order to consistently achieve a desired outcome. Effectively wrangling LLMs often requires thinking in context - considering the holistic state available to the LLM at any given time and what potential behaviors that state might yield.
Curate, don't accumulate
The curation phase happens each time we decide what to pass to the model. Context is informative, yet tight - across system prompts, tools, examples, and message history.
Iterative by nature
In contrast to the discrete task of writing a prompt, context engineering is iterative. Agents operate over multiple turns of inference and longer time horizons, so the whole context state must be managed.
The managed state
System instructions, tools, Model Context Protocol (MCP) servers, external data, and message history all land in the same finite budget and compete for the model's attention.
Context engineering vs. prompt engineering
At Anthropic, context engineering is viewed as the natural progression of prompt engineering - not a replacement for it.
| Dimension | Prompt engineering | Context engineering |
|---|---|---|
| Scope | Methods for writing and organizing LLM instructions for optimal outcomes | Strategies for curating and maintaining the optimal set of tokens during inference, including everything that lands there beyond the prompt |
| Primary object | The system prompt | The entire context state: instructions, tools, MCP, external data, message history |
| Typical tasks | One-shot classification or text generation, optimized for a single interaction | Agents operating over multiple turns and extended time horizons |
| Cadence | Discrete - written once, then tuned | Iterative - curation happens at each decision about what to pass to the model |
Why the field moved
In the early days, prompting was the biggest component of AI engineering work because most use cases outside everyday chat required prompts optimized for one-shot tasks. As systems become more capable agents, that scope is no longer sufficient.
The working definition of an agent
The field has converged on a simple paradigm: LLMs autonomously using tools in a loop. As underlying models improve, the level of autonomy can scale - smarter models let agents navigate nuanced problem spaces and recover from errors.
Context is a finite resource with diminishing marginal returns
Despite their speed and ability to manage larger and larger volumes of data, LLMs - like humans - lose focus or experience confusion at a certain point. Studies on needle-in-a-haystack and context rot show that as the number of tokens in the context window increases, the model's ability to accurately recall information from that context decreases. This characteristic emerges across all models, though some degrade more gently than others.
An attention budget
Like humans with limited working memory capacity, LLMs have an "attention budget" they draw on when parsing large volumes of context. Every new token introduced depletes that budget by some amount.
n² pairwise attention
Transformers let every token attend to every other token across the entire context, producing n² pairwise relationships for n tokens. As context length grows, the ability to capture these relationships gets stretched thin.
Training distribution
Models develop attention patterns from training data where shorter sequences are more common, so they have less experience with - and fewer specialized parameters for - context-wide dependencies.
Grow the window, watch recall slip
Position encoding interpolation lets models handle longer sequences by adapting them to the originally trained smaller context, though with some degradation in token position understanding. These factors create a performance gradient rather than a hard cliff.
Models remain highly capable at longer contexts but may show reduced precision for information retrieval and long-range reasoning compared to their performance on shorter contexts. The curve above visualizes that gradient; the post's guidance is to treat context as scarce regardless of window size.
Bigger windows are not the answer
Waiting for larger context windows might seem like an obvious tactic, but for the foreseeable future, context windows of all sizes will be subject to context pollution and information relevance concerns - at least where the strongest agent performance is desired.
Therefore
Context must be treated as a finite resource with diminishing marginal returns, which increases the need to carefully curate the tokens available to the LLM. Thoughtful context engineering is essential for building capable agents.
Four components, one budget
Given a finite attention budget, good context engineering means finding the smallest possible set of high-signal tokens. Here is what that principle demands of each component.
System prompts
Extremely clear, using simple, direct language at the right altitude. Organize into distinct sections - <background_information>, <instructions>, ## Tool guidance, ## Output description - with XML tagging or Markdown headers, though exact formatting is becoming less important as models improve. Minimal does not necessarily mean short: you still need to give the agent sufficient information up front.
Tools
Tools let agents operate with their environment and pull in new context as they work, and they define the contract between an agent and its information/action space. They should be self-contained, robust to error, and extremely clear about intended use, with descriptive, unambiguous input parameters - returning token-efficient results while encouraging efficient agent behavior.
Examples
Few-shot prompting remains a strongly advised best practice. Rather than stuffing a laundry list of edge cases into a prompt to articulate every possible rule, curate a set of diverse, canonical examples that effectively portray expected behavior. For an LLM, examples are the "pictures" worth a thousand words.
Message history
The compounding component: every turn adds tokens that deplete the budget. It is the component long-horizon techniques exist to manage - through tool result clearing, compaction, and externalized memory.
Recommended sequence: start with a minimal prompt using the best available model to see how it performs on your task, then add clear instructions and examples to improve performance based on failure modes found during initial testing.
The tool-overlap test
One of the most common failure modes is a bloated tool set covering too much functionality or creating ambiguous decision points. If a human engineer can't definitively say which tool should be used in a given situation, an AI agent can't be expected to do better.
Minimal viable toolset
Curating a minimal viable set of tools for the agent also leads to more reliable maintenance and pruning of context over long interactions - a prerequisite for the long-horizon techniques below.
Just-in-time context retrieval
Rather than pre-processing all relevant data up front, agents built with the just-in-time approach maintain lightweight identifiers - file paths, stored queries, web links - and use those references to dynamically load data into context at runtime using tools.
| Pre-inference retrieval | Just-in-time retrieval | Hybrid | |
|---|---|---|---|
| How it works | Embedding-based retrieval surfaces important context before the agent reasons | Lightweight identifiers resolved into data at runtime via tools | Some data loaded up front for speed; further exploration pursued at the agent's discretion |
| Speed | Fastest first token; data already present | Slower - runtime exploration costs turns | Fast start, deeper answers on demand |
| Context hygiene | Risk of surfacing stale or irrelevant chunks | Only what is needed enters the window; bypasses stale indexing and complex syntax trees | Depends on where the boundary is drawn |
| Engineering load | Index, embedding, and refresh pipelines | Opinionated tool and heuristic design; without guidance agents waste context on dead-ends | Both, plus the boundary decision |
| Good fit | High-volume, well-understood corpora | Large, dynamic environments such as codebases | Less dynamic content - legal or finance work |
Names carry meaning
To an agent in a file system, test_utils.py inside a tests folder implies a different purpose than the same filename in src/core_logic/. Folder hierarchies, naming conventions, and timestamps are all signals that tell humans and agents how and when to use information.
Incremental discovery
Autonomous navigation lets agents discover context layer by layer: file sizes suggest complexity, naming conventions hint at purpose, timestamps proxy for relevance. Agents keep only what is necessary in working memory and use note-taking for persistence, staying focused on relevant subsets rather than drowning in exhaustive but potentially irrelevant information.
Claude Code
The model writes targeted queries, stores results, and leverages Bash commands like head and tail to analyze large databases without ever loading full data objects into context. CLAUDE.md files are dropped in up front while glob and grep primitives retrieve just-in-time.
The trade-off, stated plainly
Runtime exploration is slower than retrieving pre-computed data, and it requires thoughtful engineering to give the LLM the right tools and heuristics for navigating its information landscape. Given the pace of progress, "do the simplest thing that works" remains the best advice for teams building agents - and as models improve, agentic design trends toward letting intelligent models act intelligently with progressively less human curation.
Three techniques for tasks that outgrow the window
Long-horizon tasks require agents to maintain coherence, context, and goal-directed behavior over sequences of actions where the token count exceeds the context window - tasks spanning tens of minutes to multiple hours, like large codebase migrations or comprehensive research projects.
Summarize, then reinitiate
Compaction takes a conversation nearing the context window limit, summarizes its contents, and reinitiates a new context window with the summary. It is typically the first lever for long-term coherence: at its core it distills the window in a high-fidelity manner so the agent continues with minimal performance degradation.
- Claude Code passes message history to the model to summarize and compress the most critical details, preserving architectural decisions, unresolved bugs, and implementation details while discarding redundant tool outputs or messages.
- The agent then continues with that compressed context plus the five most recently accessed files - continuity without worrying about window limits.
- The art lies in what to keep versus discard: overly aggressive compaction can lose subtle but critical context whose importance only becomes apparent later.
- Tune the compaction prompt on complex agent traces - maximize recall first so every relevant piece is captured, then improve precision by eliminating superfluous content.
Agentic memory outside the window
The agent regularly writes notes persisted to memory outside the context window, and those notes get pulled back in at later times. This provides persistent memory with minimal overhead - like Claude Code creating a to-do list, or a custom agent maintaining a NOTES.md file - letting the agent track progress across complex tasks and preserve dependencies that would otherwise be lost across dozens of tool calls.
- Claude playing Pokémon demonstrates the pattern outside coding: the agent maintains precise tallies across thousands of game steps - "for the last 1,234 steps I've been training my Pokémon in Route 1, Pikachu has gained 8 levels toward the target of 10."
- Without any prompting about memory structure, it develops maps of explored regions, remembers unlocked achievements, and keeps strategic combat notes about which attacks work best against different opponents.
- After context resets it reads its own notes and continues multi-hour training sequences or dungeon explorations - coherence across summarization steps that would be impossible keeping everything in the window alone.
- Released in public beta with the Sonnet 4.5 launch as a file-based memory tool on the Claude Developer Platform, enabling knowledge bases that build up over time and project state maintained across sessions.
Separation of concerns via parallel specialists
Rather than one agent maintaining state across an entire project, specialized sub-agents handle focused tasks with clean context windows. The main agent coordinates with a high-level plan while sub-agents perform deep technical work or use tools to find relevant information.
Lead agent
Holds the plan. Focuses on synthesizing and analyzing results; detailed search context never enters its window.
This pattern, detailed in How we built our multi-agent research system, showed a substantial improvement over single-agent systems on complex research tasks.
Matching the technique to the task
The choice between these approaches depends on task characteristics - they are complements, not competitors.
Compaction
Maintains conversational flow for tasks requiring extensive back-and-forth.
Note-taking
Excels for iterative development with clear milestones.
Multi-agent architectures
Handle complex research and analysis where parallel exploration pays dividends.
Even as models continue to improve, the challenge of maintaining coherence across extended interactions will remain central to building more effective agents. Treating context as a precious, finite resource will remain central to building reliable, effective agents.
Less prescriptive over time
The techniques outlined will continue evolving as models improve. Smarter models already require less prescriptive engineering, allowing agents to operate with more autonomy.
Context stays scarce
Even as capabilities scale, treating context as a precious, finite resource remains central - the guiding principle does not change with window size.
Platform & cookbook
Get started with context engineering in the Claude Developer Platform, and consult the memory and context management cookbook for tips and best practices.