Overview
The context window is the AI's 'short-term memory.' It includes the user's prompt, previous messages in the conversation, and any retrieved documents (in RAG).
Measurement
Measured in tokens. If a conversation exceeds the context window, the model 'forgets' the earliest parts of the interaction.
Trends
Context windows are growing rapidly, from 4k tokens in early GPT-3 to 1M+ tokens in Gemini 1.5 Pro.