Python / Python Modern Generative AI and Agents Interview Questions
1. What are Large Language Models (LLMs) and how do they generate text?
Large Language Models (LLMs) are neural networks — almost universally transformer-based — trained on massive text corpora to learn the statistical patterns of language. At inference, they generate text autoregressively : given a sequence of input tokens, the model produces a probability distribut...
2. What is the Hugging Face Transformers pipeline API and how do you use it for common NLP and vision tasks?
The pipeline() function in Hugging Face Transformers is the highest-level API — it wraps model loading, tokenisation, inference, and post-processing into a single callable. It is the fastest way to get results from a pre-trained model and is ideal for prototyping and evaluation before committing ...
3. How does tokenisation work in Hugging Face and what are the key tokenizer concepts?
Tokenisation converts raw text into integer IDs that the model can process. Modern LLMs use subword tokenisation (BPE, WordPiece, or SentencePiece) rather than word or character tokenisation, balancing vocabulary size against the number of tokens per sentence. Each model family has its own tokeni...
4. What is the Auto-class pattern in Hugging Face and how do you run inference with a raw model?
The Auto* classes ( AutoTokenizer , AutoModel , AutoModelForSequenceClassification , etc.) are factory classes that read a model's config.json from the Hub and automatically instantiate the correct tokenizer or model architecture without you needing to know which specific class to use. This makes...
5. What is prompt engineering and what are the most effective techniques for getting better outputs from LLMs?
Prompt engineering is the practice of crafting inputs to LLMs to elicit more accurate, relevant, and reliable outputs without changing the model's weights. Since LLMs are sensitive to the exact phrasing, structure, and context of the prompt, small changes can dramatically affect output quality. C...
6. What is Retrieval-Augmented Generation (RAG) and why is it preferred over full fine-tuning for knowledge-intensive tasks?
Retrieval-Augmented Generation (RAG) augments an LLM's response by first retrieving relevant documents from an external knowledge source and injecting them into the prompt as context. Instead of relying solely on knowledge baked into model weights during training, the LLM reasons over dynamically...
7. What are vector databases and how do they enable semantic search in RAG pipelines?
Vector databases store numerical vector representations (embeddings) of documents and enable fast approximate nearest-neighbour (ANN) search — retrieving the vectors most similar to a query vector, typically measured by cosine similarity or inner product. This is the retrieval backbone of every R...
8. How do you build a complete RAG pipeline using LangChain?
LangChain provides composable abstractions for every component of a RAG pipeline — document loaders, text splitters, embedding models, vector stores, retrievers, and LLM chains — making it straightforward to assemble a production-quality system without boilerplate. The pipeline follows the standa...
9. What are the most important text splitting strategies in RAG, and how do chunk size and overlap affect retrieval quality?
Chunk size and overlap are the most impactful hyperparameters in a RAG pipeline — they directly affect both retrieval precision and answer quality. A chunk that is too small may contain only a fragment of a complete thought; a chunk that is too large may contain so much irrelevant content that th...
10. What are LangChain's core abstractions — Chains, Runnables, and the LangChain Expression Language?
LangChain's modern design (LangChain v0.2+) revolves around the Runnable interface: any component that can be invoked (prompts, LLMs, parsers, retrievers, custom functions) implements invoke() , stream() , and batch() . The LangChain Expression Language (LCEL) composes Runnables with the pipe ope...
11. How do you add conversation memory to an LLM application with LangChain?
LLMs are stateless — each API call is independent and the model has no memory of previous exchanges. Maintaining conversation context requires explicitly including past messages in the current prompt. LangChain provides memory abstractions that manage this history, automatically appending it to t...
12. What is an AI agent and how does function calling / tool use work in LLM-based agents?
An AI agent is a system where an LLM acts as a reasoning engine that decides what actions to take (calling tools, retrieving information, writing code) based on a goal, observes the results of those actions, and continues reasoning until the goal is met. Unlike a simple chain that executes a fixe...
13. What is the ReAct agent pattern and how does LangChain implement it?
ReAct (Reasoning + Acting) is an agent pattern where the LLM alternates between producing a Thought (internal reasoning about what to do next), an Action (calling a tool), and an Observation (the tool's result). This loop continues until the LLM produces a Final Answer . The key insight is that i...
14. How do you efficiently load large Hugging Face models for inference, including quantization and device placement?
Loading a 7B+ parameter model naively with from_pretrained() materialises the entire model in FP32 (~28 GB for 7B params), which exceeds most GPU memory budgets. Modern Hugging Face loading uses three key techniques: precision reduction (bfloat16 / float16), device mapping, and on-the-fly quantis...
15. How do you use Hugging Face's text-generation pipeline with open-source chat models like Mistral or Llama?
Open-source instruction-tuned models (Mistral-Instruct, Llama-3-Instruct, Qwen, Gemma) follow specific chat templates that structure the conversation into system, user, and assistant turns with special tokens. Using the correct template is critical — wrong formatting produces significantly degrad...
16. How do you use the Hugging Face Inference API and the InferenceClient for production deployments?
Running large models locally requires substantial GPU infrastructure. The Hugging Face Inference API offers serverless inference for thousands of public models — you send HTTP requests and receive predictions without managing any compute. The huggingface_hub library's InferenceClient provides a t...
17. What is LoRA and how does the Hugging Face PEFT library simplify fine-tuning large models?
Fine-tuning all parameters of a 7B model requires enormous compute and memory. LoRA (Low-Rank Adaptation) sidesteps this by keeping the original pretrained weights frozen and injecting small trainable rank decomposition matrices into each layer. For a weight matrix W ∈ ℝ^{d×k} , LoRA adds ΔW = BA...
18. How do you use the Hugging Face Datasets library for training and evaluation?
The datasets library provides a unified interface to thousands of NLP and computer vision datasets from the Hub, with built-in streaming, caching, and memory-mapped access via Apache Arrow. It integrates directly with the Transformers Trainer and works well with PyTorch DataLoader . from datasets...
19. How do you fine-tune a model using the Hugging Face Trainer API?
The Trainer class encapsulates the standard training loop — batching, gradient accumulation, mixed precision, evaluation, checkpointing, logging to TensorBoard/WandB — behind a clean API. Combined with TrainingArguments , it handles most production training concerns so you can focus on data prepa...
20. How do you evaluate LLM outputs for quality, factual accuracy, and hallucination?
Traditional NLP metrics like BLEU and ROUGE measure surface-level token overlap but correlate poorly with human quality judgments for open-ended generation. Modern LLM evaluation uses a combination of reference-based metrics, LLM-as-judge evaluation, and task-specific benchmarks. LLM Evaluation M...
21. How do you stream LLM responses token by token for a better user experience?
Without streaming, the user waits for the model to finish generating the entire response before seeing anything — for long outputs this can be 10–30 seconds of blank wait time. Streaming delivers each token to the user as it is generated, making the application feel dramatically more responsive. ...
22. How do you use multimodal models (vision-language) with Hugging Face for image understanding tasks?
Multimodal models like LLaVA, PaliGemma, and Idefics combine a vision encoder (typically a CLIP or SigLIP model) with an LLM, enabling reasoning over both images and text. They are used for image captioning, visual question answering (VQA), document understanding, and chart analysis. Loading them...
23. How do you reliably get structured JSON output from LLMs, and what tools does LangChain provide?
Getting LLMs to reliably return structured data (not just text) is essential for applications that need to parse and act on model outputs. Three complementary approaches exist: prompt-level instructions, API-level enforcement (JSON mode / structured outputs), and library-level output parsers with...
24. How do you compute semantic similarity between texts using Hugging Face and OpenAI embeddings?
Semantic similarity compares text meaning rather than surface words. This powers search engines, duplicate detection, recommendation systems, and the retrieval step in RAG. The standard approach embeds both texts into a high-dimensional vector space and measures the angle between them via cosine ...
25. What document loaders does LangChain provide, and how do you handle different file types in a RAG pipeline?
A RAG system is only as good as the documents it can ingest. LangChain provides over 100 document loaders for web pages, PDFs, Word files, databases, code repositories, spreadsheets, and cloud storage. Every loader returns a list of Document objects with page_content (the text) and metadata (sour...
26. What is the OpenAI Assistants API and how does it differ from the Chat Completions API?
The Assistants API (part of OpenAI's platform) provides a higher-level abstraction for building AI agents with persistent conversation threads, built-in tool use, and file handling — without managing state manually. Key concepts: an Assistant holds configuration (model, system prompt, tools); a T...
27. What is the Parent Document Retriever pattern and when does it improve RAG performance?
Standard RAG embeds large chunks (500–1000 tokens) to preserve context but stores them directly as the retrieved context. The trade-off: large chunks have better coherence but may score lower on retrieval similarity because their embedding averages out many ideas. Small chunks have precise embedd...
28. How do you manage, version, and reuse prompts in production LLM applications?
In production systems, prompts are first-class assets — they evolve through experimentation, need version control, and may be shared across teams. Hard-coding prompts in application code makes them difficult to update without deployment. Several strategies improve prompt management. # ââ Appr...
29. How do you generate and manipulate images using Hugging Face's Diffusers library?
The diffusers library provides a unified API for diffusion models including Stable Diffusion, SDXL, Flux, and ControlNet. Diffusion models generate images by progressively denoising random Gaussian noise, guided by a text prompt encoded by a text encoder (typically CLIP or T5). The DiffusionPipel...
30. How do you handle documents or conversations that exceed an LLM's context window?
Every LLM has a maximum context window (measured in tokens) — GPT-4o supports 128K tokens, Claude 3.5 Sonnet 200K, Llama 3.1 128K. Inputs exceeding this limit are either truncated (silently losing content) or raise an error. Several strategies handle long documents: Long Document Handling Strateg...
31. What is LangGraph and how does it differ from LangChain's AgentExecutor for building agents?
LangGraph is a framework for building stateful, multi-step agents as directed graphs where each node is a function (LLM call, tool call, or logic) and edges define the flow of control. Unlike LangChain's AgentExecutor (a simple Thought-Action-Observation loop), LangGraph gives you explicit contro...
32. What embedding models should you use for production RAG systems, and how do you choose between OpenAI and open-source options?
The embedding model is one of the most consequential choices in a RAG system — it determines retrieval quality, cost, latency, and whether data leaves your infrastructure. The right choice depends on your data volume, sensitivity, quality requirements, and deployment environment. Embedding Model ...
33. How do you add safety guardrails and input/output validation to LLM applications?
Production LLM applications need protection against prompt injection, jailbreaks, generation of harmful content, leaking of system prompts, and off-topic responses. Guardrails are validation and filtering layers applied before the LLM (input guards) and after (output guards). # ââ Input valid...
34. How do you manage LLM API costs and implement caching to reduce redundant calls?
LLM API costs can escalate quickly in production. For context, GPT-4o costs $5/1M input tokens and $15/1M output tokens — a system making 10,000 calls/day with 2,000 tokens each consumes $100+/day. Several strategies keep costs manageable: choosing the right model for the task, caching repeated q...
35. What is LlamaIndex and how does it compare to LangChain for RAG use cases?
LlamaIndex (formerly GPT Index) is a data framework specialised for connecting LLMs to diverse data sources. While LangChain is a general-purpose composable LLM framework covering agents, chains, memory, and RAG, LlamaIndex focuses almost exclusively on the data ingestion and indexing layer — pro...
36. What is the Hugging Face Hub and how do you push a trained model to share it?
The Hugging Face Hub is a platform hosting over 900,000 models, 200,000 datasets, and 300,000 Spaces (interactive apps). Every model on the Hub has a model card (README.md) documenting its architecture, training data, performance, intended uses, and limitations — following a community standard fo...
37. How do you build a demo web interface for an LLM application using Gradio?
Gradio is Hugging Face's rapid UI library for building interactive machine learning demos with a few lines of Python. It runs locally or deploys instantly to Hugging Face Spaces. For LLM applications, gr.ChatInterface provides a fully featured chat UI out of the box, while gr.Interface handles si...
38. How do you monitor and debug LLM applications in production using LangSmith?
LangSmith is LangChain's observability platform for LLM applications. It automatically traces every LLM call, chain step, and tool invocation, providing: full input/output logging, latency and cost breakdowns, error tracking, prompt version comparison, and human feedback collection. In production...
