Engineering the Context Window
Retrieval, memory, and the working set a model actually reasons over
The context window is the one surface you fully control, so most problems blamed on model quality are context-assembly problems.
Retrieval, memory, compaction and ordering are usually built by different people at different times, and then the result is judged as though the model were the variable. This book treats them as a single system for deciding what the model sees: an evidence pipeline feeding a budgeted, ordered window, with provenance, freshness and access control surviving all the way into the prompt. It also takes the long-context argument seriously, since long windows did kill lazy retrieval, and gives you the tests that show which parts of your own pipeline are now redundant.
What it makes operable
-
01
Decide retrieval versus long context on corpus size, freshness, access control and cost
-
02
Run lexical and vector retrieval together and fuse them by rank rather than score
-
03
Assemble a window under an explicit token budget, with decisive evidence out of the middle
-
04
Write a memory policy that holds up when the evidence is untrusted or poisoned
Contents
12 of 12 published
Every chapter is free to read in the browser, cites its own sources, and stands on its own if you came for one decision rather than the whole argument.
- 01 Context Is the Only Thing You Control The surface you own, as against the weights you do not. 11 min
- 02 What Long Context Fixed, and What It Did Not Mid-window accuracy loss and cost per query, measured on your own corpus. 10 min
- 03 Retrieval as an Evidence Pipeline The stages, contracts and failure points between a query and a supported claim. 10 min
- 04 Lexical and Vector Retrieval, Together Why exact identifiers still need BM25, and how to fuse two ranked lists. 10 min
- 05 Rerankers and the Cost of Being Sure Spending cross-encoder time only where it changes the evidence set. 10 min
- 06 Chunking Is a Document-Modelling Problem Segmenting on structure so a claim keeps the qualifier that bounds it. 10 min
- 07 Assembling the Window: Order, Budget, Compaction A token budget per section, an ordering rule, and compaction you can reverse. 10 min
- 08 Memory: What to Write, What to Forget A write policy for durable facts, with provenance and a TTL on every record. 10 min
- 09 Provenance, Freshness, and Access Control Inside Context Keeping who-may-see-this and how-old-is-this attached to the evidence itself. 10 min
- 10 Poisoned Memory and Untrusted Evidence Treating retrieved text, including your own memory, as attacker-controlled input. 10 min
- 11 Measuring Context Quality Independently of the Model Scoring the window so a bad answer can be blamed on the right stage. 10 min
- 12 Migrating a RAG System Into a Context System The order of operations for rebuilding a plateaued pipeline in place. 10 min
Evidence · 4 sources
Third-party sources behind the book's premise. Every figure in them belongs to the party that published it and is attributed to them in the text.
