System prompts and context engineering patterns for agents

Last week I debugged a procurement review agent that broke in production. It looked like a senior analyst in the test environment. But once live, it started making up vendor numbers, treating $30,000 requests as $3,000 and approving them. The first reaction was that the model needed to be smarter. Time to swap it for a bigger one.

Upgrading wouldn't have fixed it. The problem was the system prompt itself: 1,700 lines mixing compliance rules, tool documentation, and role instructions together. The three constraints that mattered were buried around line 900. By the time the model reached them, its focus had already scattered.

System prompt structure and context engineering are separate problems.

Understanding context engineering

Context engineering is deciding, within the bounds of your context window, what information to include, how much of it, and where to put it, so the model gets exactly enough data for each reasoning step, no more and no less. Your system prompt is the most stable, most structured piece of that budget.

Most work goes into clarity: how to explain things better. But leverage lives in efficiency: how to pack information tighter. Attention is a finite resource, not infinite space.

The segmented system prompt template

A working system prompt breaks into four distinct sections in this order. Critical information goes first because models retain the beginning and end better than the middle.

Role and objective. One sentence. Who is this agent? Who does it work for? What counts as success? Avoid 'You are a helpful AI assistant', it's empty.

You are a procurement review agent for a manufacturing company's accounting department.
Your sole objective: evaluate each expense request using company policy and return "Approve / Reject / Escalate to human."
You do not execute payment. You make the judgment call and attach a one-line reason.

Hard constraints. This section goes wrong most often. The principle: constraints should be few, absolute, and verifiable. Five things you'll never do beats twenty things you'll try.

- Any amount over $50,000 routes to human review. No agent approvals above this threshold.
- Vendor ID must come back from lookup_vendor. Never generate one on your own.
- If information is incomplete, output "Escalate to human." Never guess.

Tool specifications. For each tool, cover three things: when to use it, what input it takes, what it returns. The trigger condition matters most. A common mistake is the model answering from memory instead of calling a tool.

lookup_vendor(vendor_name) → Returns vendor ID and credit status.
  Always call this before any step that requires a vendor_id.
check_budget(dept, amount) → Returns the department's remaining quarterly budget.

Few-shot examples. Include one to three complete walkthroughs, input, reasoning, output. Include at least one edge case: an amount at the threshold, or partial information. Few-shot shows the pattern of actual decisions, not just the format. One strong negative example beats ten lines of description.

Context window budgeting

Your context window is a fixed-size budget. With 200,000 tokens available, a typical allocation looks like this:

Execution history is the trickiest part. When a multi-step agent reaches step fifteen, if all fourteen prior tool results stay in context as-is, you waste tokens and bury the current information under old noise. Solution: every few steps, compress old history to a single-line summary. Keep full detail only for the last two or three steps. The same principle applies to RAG, retrieve ten results if needed, but don't include all ten. Rank by relevance, remove duplicates, keep the three or four best.

There's a related tradeoff: more few-shot examples make the agent more stable, but every call costs tokens for all of them. Early in production, load in extra examples to anchor quality. Once things stabilize, start removing examples that have become learned patterns and redirect those tokens to retrieval.

Why this matters

The procurement agent showed this plainly. No model swap was needed. Three changes fixed it: moved the three hard constraints to the top, locked the tool trigger conditions, and replaced the runaway execution history with a rolling summary. Same model. Error rate dropped from thirty percent to single digits. Two weeks after launch, the review team let it handle the first pass without hand-checking.

Version-control the system prompt and context budget like code. Measure every change against real-world pass rates, not how clean the demo looks. A polished demo is easy; the real test is whether it holds those three constraints at step fifteen with context nearly full. That's when it's genuinely ready for production.