AI / Core OpenAI Codex Application Fundamentals Interview Questions
How do you handle context window management in long-running OpenAI applications?
Every OpenAI model has a finite context window - the maximum tokens (input + output) it can process in one call. In long-running applications (chat sessions, agentic workflows, document analysis), managing this window efficiently is critical for both functionality and cost.
from openai import OpenAI import tiktoken client = OpenAI() enc = tiktoken.encoding_for_model("gpt-5.5") # Context window strategy 1: Sliding window # Keep the most recent N turns, discarding the oldest def sliding_window(messages: list, max_tokens: int = 100_000) -> list: total = sum(len(enc.encode(str(m["content"]))) for m in messages) while total > max_tokens and len(messages) > 2: # Remove oldest user+assistant pair (keep system message) removed = messages.pop(1) total -= len(enc.encode(str(removed["content"]))) return messages # Context window strategy 2: Summarisation # Summarise old conversation instead of dropping it entirely async def summarise_history(messages: list[dict]) -> str: summary = await client.responses.create( model="gpt-5.4-mini", # cheap model for summarisation input=f"Summarise this conversation concisely:\n{messages}", ) return summary.output_text # Strategy 3: Responses API state management (avoids manual history) # Use previous_response_id - OpenAI manages the context server-side first = client.responses.create(model="gpt-5.5", input="Hello!", store=True) second = client.responses.create( model="gpt-5.5", input="What did I just say?", previous_response_id=first.id, # server retrieves context automatically ) # Strategy 4: use file_search for large static documents # Instead of pasting 100k-token documents into every prompt: # Upload once to a vector store, retrieve only relevant chunks vector_store_id = "vs_abc123" response = client.responses.create( model="gpt-5.5", tools=[{"type": "file_search", "vector_store_ids": [vector_store_id]}], input="What does section 4.2 of our API spec say about authentication?", # Only the relevant section is retrieved, not the entire doc )
| Strategy | Best for | Trade-off |
|---|---|---|
| Sliding window | Conversational UIs | May lose important early context |
| Summarisation | Long research sessions | Adds latency and cost for summariser call |
| previous_response_id | Responses API multi-turn | Requires store:true; server manages context |
| RAG / file_search | Large static documents | Retrieval may miss nuanced context |
More Related questions...