AI & ML
Context Window
Quick Definition
The maximum number of tokens (words, characters, and punctuation) that a large language model can accept and reason across in a single conversational turn or API request.
Architectural Deep Dive
Every LLM has a finite memory span known as its context window (measured in tokens). While early models like GPT-3 had 2,048-token windows, modern 2026 models support from 128,000 to over 2,000,000 tokens. However, having a large context window does not guarantee perfect recall: models often suffer from 'Lost in the Middle' phenomena, where key facts buried in the middle of massive prompts are forgotten compared to facts at the beginning or end.
Why this matters in 2026: Determines whether an AI can read an entire codebase, legal deposition, or financial annual report in a single prompt.
Common Engineering Pitfalls
- •Stuffing entire databases into context instead of using RAG, resulting in high latency and steep token API bills.
- •Assuming 100% needle-in-a-haystack recall across the entire 1M token span without evaluation.