AI / Core OpenAI Codex Application Fundamentals Interview Questions
How do you handle context window management in long-running OpenAI applications?
Every OpenAI model has a finite context window - the maximum tokens (input + output) it can process in one call. In long-running applications (chat sessions, agentic workflows, document analysis), managing this window efficiently is critical for both functionality and cost.
from openai import OpenAI import tiktoken client = OpenAI() enc = tiktoken.encoding_for_model("gpt-5.5") # Context window strategy 1: Sliding window # Keep the most recent N turns, discarding the oldest def sliding_window(messages: list, max_tokens: int = 100_000) -> list: total = sum(len(enc.encode(str(m["content"]))) for m in messages) while total > max_tokens and len(messages) > 2: # Remove oldest user+assistant pair (keep system message) removed = messages.pop(1) total -= len(enc.encode(str(removed["content"]))) return messages # Context window strategy 2: Summarisation # Summarise old conversation instead of dropping it entirely async def summarise_history(messages: list[dict]) -> str: summary = await client.responses.create( model="gpt-5.4-mini", # cheap model for summarisation input=f"Summarise this conversation concisely:\n{messages}", ) return summary.output_text # Strategy 3: Responses API state management (avoids manual history) # Use previous_response_id - OpenAI manages the context server-side first = client.responses.create(model="gpt-5.5", input="Hello!", store=True) second = client.responses.create( model="gpt-5.5", input="What did I just say?", previous_response_id=first.id, # server retrieves context automatically ) # Strategy 4: use file_search for large static documents # Instead of pasting 100k-token documents into every prompt: # Upload once to a vector store, retrieve only relevant chunks vector_store_id = "vs_abc123" response = client.responses.create( model="gpt-5.5", tools=[{"type": "file_search", "vector_store_ids": [vector_store_id]}], input="What does section 4.2 of our API spec say about authentication?", # Only the relevant section is retrieved, not the entire doc )
| Strategy | Best for | Trade-off |
|---|---|---|
| Sliding window | Conversational UIs | May lose important early context |
| Summarisation | Long research sessions | Adds latency and cost for summariser call |
| previous_response_id | Responses API multi-turn | Requires store:true; server manages context |
| RAG / file_search | Large static documents | Retrieval may miss nuanced context |
Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!
Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.
Invest now!!! Get Free equity stock (US, UK only)!
Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.
The Robinhood app makes it easy to trade stocks, crypto and more.
Webull! Receive free stock by signing up using the link: Webull signup.
More Related questions...
