AI / GenAI Essentials Interview questions

Generative AI refers to models that create new content - text, images, code, audio or video - instead of only labelling or predicting something about existing data. They learn the statistical patterns in huge training datasets and then produce fresh output that follows those patterns.

A classic machine learning model might say whether an email is spam. A generative model can write the email, summarise it, or draft a reply.

• Text: chatbots, summarisation, translation
• Code: completion, test generation, refactoring
• Images and audio: text-to-image, speech synthesis

Most of today's generative systems are built on large neural networks such as transformers and diffusion models.

A Large Language Model is a neural network, almost always a transformer, trained on very large text corpora to predict the next token in a sequence. That one simple objective, repeated over trillions of tokens, is what gives the model grammar, facts, reasoning patterns and coding skill.

"Large" refers to both the training data and the parameter count, which commonly runs from a few billion to hundreds of billions of weights.

At inference time the model reads your prompt, predicts one token, appends it, and repeats until it hits a stop condition. It does not look answers up unless you connect it to a retrieval or search tool.

A token is the basic unit of text a model reads and writes. It can be a whole word, part of a word, a punctuation mark or even a single byte, depending on the tokenizer.

As a rough rule for English, one token is about four characters or three quarters of a word. Other languages and source code often need more tokens for the same meaning.

Tokens matter because pricing, rate limits and context windows are all measured in tokens, not words. A long prompt with a long answer can use up both your budget and the available window quickly.

A prompt is the input you send to a generative model: the instructions, context and data it uses to produce a response. In chat APIs it is usually a list of messages with roles such as system, user and assistant.

A well-built prompt typically contains:

• a clear instruction or task
• relevant context or reference text
• the input data to work on
• the desired output format
• optional examples of good answers

The model has no memory beyond what is in the prompt, so anything it needs to know must be included there.

The context window is the maximum number of tokens a model can consider in a single request. It normally covers the prompt, any retrieved documents, the conversation history and the generated answer together.

If a model has a 128K-token window and your input uses 120K, only about 8K tokens remain for the reply. Go past the limit and the API either returns an error or older content has to be dropped, depending on how the application handles it.

A bigger window lets you pass whole documents at once, but longer inputs cost more and can slow responses.

Prompt engineering is the practice of designing and refining prompts so a model gives accurate, consistent and well-formatted output. It is mostly iterative testing: change the wording, compare results, keep what works.

Common habits that help:

• State the task and audience explicitly
• Provide examples of the exact output you want
• Ask for a specific format such as JSON or a table
• Break big tasks into smaller steps
• Tell the model what to do when it is unsure

It is cheaper than fine-tuning, which is why it is usually the first thing to try.

The most common prompting techniques differ in how much guidance you put into the prompt:

In practice these are combined, for example a role plus few-shot examples plus a requested output schema.

A hallucination is output that sounds fluent and confident but is false, made up or not supported by the provided source. Typical examples are invented citations, wrong dates, non-existent API functions or fabricated quotes.

The model is not lying. It generates the most plausible continuation, and plausible is not always true.

Hallucinations are especially risky in legal, medical and financial settings, so production systems add grounding (such as RAG), citations and human review. A good habit is to treat every factual claim from a model as a draft that needs a source, especially names, numbers, quotes and links.

Temperature controls how random the model's token choices are. Before sampling, the model's scores are divided by the temperature: low values sharpen the distribution so the top token wins almost every time, and high values flatten it so less likely tokens get a chance.

• Low (0 to 0.3): focused, repeatable - good for extraction, code, Q&A
• Medium (0.5 to 0.8): balanced writing
• High (1.0+): varied, creative, more risk of drift

Even at 0 some providers do not guarantee identical output on every run, so do not rely on temperature alone when you need strict repeatability. If the task has one correct answer, keep it low. If you want several different drafts to choose from, raise it. Values that are too high tend to produce rambling or off-topic text, which is why most production assistants stay in the low to medium range.

An embedding is a list of numbers (a vector) that represents the meaning of a piece of text, image or other data. Items with similar meaning end up close together in this vector space, even when they share no words.

For example, "reset my password" and "I can't log in" land near each other, while "quarterly revenue" lands far away. Typical embedding models output vectors with hundreds to a few thousand dimensions.

Embeddings power semantic search, recommendations, clustering and the retrieval step of RAG.

A vector database stores embeddings and quickly finds the ones nearest to a query vector. Instead of exact matching, it does approximate nearest neighbour (ANN) search using indexes such as HNSW or IVF, trading a tiny bit of accuracy for large speed gains.

Examples include Pinecone, Weaviate, Milvus and Chroma, and extensions like pgvector for PostgreSQL. Most also store metadata so you can filter by tenant, date or document type alongside similarity.

It is the usual storage layer behind RAG and semantic search.

RAG is a pattern where the system first retrieves relevant documents from an external knowledge source and then passes them to the LLM along with the question. The model answers using that supplied evidence instead of relying only on what it memorised during training.

1. Convert the user question to an embedding
2. Search a vector (or hybrid) index for the closest chunks
3. Insert those chunks into the prompt
4. Let the model generate an answer, ideally with citations

RAG lets you use private or fresh data without retraining, and it cuts hallucinations because the answer is grounded in retrieved text.

Fine-tuning means continuing the training of a pre-trained model on a smaller, task-specific dataset so its weights shift toward the behaviour you want. The data is usually prompt and ideal-response pairs.

It works well for teaching a consistent style, output format, tone or domain vocabulary, or for getting a small model to match a big one on a narrow task.

It is a poor way to add frequently changing facts. For that, retrieval is cheaper and easier to update. Parameter-efficient methods such as LoRA make fine-tuning affordable by training only a small set of extra weights.

A transformer is a neural network architecture introduced in the 2017 paper "Attention Is All You Need". It replaced recurrent networks by using self-attention, which lets every token look at every other token in the sequence at once.

Each transformer layer combines a multi-head attention block with a feed-forward network, wrapped in residual connections and normalisation. Positional information is added so the model knows token order.

Because tokens are processed in parallel, transformers train efficiently on GPUs, which is why they scaled into modern LLMs.

GenAI shows up wherever people write, read or search a lot. Common use cases include:

• Conversational assistants for support and internal helpdesks
• Summarisation of meetings, tickets and long reports
• Code assistance: completion, tests, explanation, migration
• Document Q&A using RAG over company knowledge
• Data extraction from invoices, contracts and emails into JSON
• Content drafting and translation
• Image and audio generation for marketing and media

The best early projects have a clear owner, measurable outcome and a human who can check the output.

A system prompt is a high-priority instruction set, written by the developer, that defines the model's role, tone, rules and constraints for the whole conversation. It sits in a separate message role from the user's text.

Typical contents are the persona ("you are a billing support assistant"), allowed and forbidden topics, answer format, and what to do when information is missing.

It guides behaviour but is not a security boundary. A determined user can sometimes override or extract it, so never place secrets or credentials inside. Keep it short, specific and testable, and version it like code so changes can be reviewed and rolled back.

Multimodal AI handles more than one type of data - such as text, images, audio and video - as input, output or both. A single model can, for instance, read a chart screenshot and answer questions about it in text.

Common patterns are image-to-text (describe or read a photo), text-to-image generation, speech-to-text, and text-to-speech.

Under the hood, each modality is converted into tokens or embeddings that live in a shared representation the model can reason over. Typical business uses include reading receipts and forms, describing product photos, transcribing calls and answering questions about diagrams. Cost and latency are usually higher than text-only calls, because images and audio expand into many tokens.

A foundation model is a large model pre-trained on broad, mostly unlabelled data so it can be adapted to many downstream tasks. LLMs, image models like Stable Diffusion, and embedding models such as CLIP are all examples.

Instead of training a new model per task, teams reuse one foundation model and adapt it with prompting, retrieval, fine-tuning or tools.

The expensive pre-training is done once by a few providers, while most organisations only pay for adaptation and inference. This is why choosing which foundation model to build on is one of the first architectural decisions in a GenAI project, since it affects cost, quality, licensing and data-handling terms.

An AI agent is an LLM-driven system that pursues a goal by deciding what to do next, calling tools, reading the results and repeating until the task is done. A plain chatbot answers once, while an agent acts in a loop.

The main ingredients are the model, a set of tools (search, code runner, APIs), instructions, and some memory of what has happened so far.

Because agents can take real actions, they need limits such as step budgets, permission scopes and human approval for risky steps.

Function calling lets a model request an action by returning a structured message - the function name plus JSON arguments - instead of plain text. You describe the available functions with a name, description and parameter schema, and the model picks one when it fits.

Your application, not the model, executes the function and sends the result back so the model can write the final answer.

{"name": "get_weather", "arguments": {"city": "Charlotte", "unit": "F"}}

This is how LLMs query databases, call APIs and trigger workflows safely.

A discriminative model learns the boundary between classes, estimating P(label | input). A generative model learns how the data itself is distributed, so it can sample new examples.

Generative models can also be used for classification by prompting, but a small discriminative model is often cheaper and more predictable for a fixed label set.

In self-attention each token is turned into three vectors: a query (what I am looking for), a key (what I offer) and a value (the information I carry). A token's query is compared with every key to get relevance scores.

1. Compute scores: query dot key for every pair of tokens
2. Scale by the square root of the key dimension
3. Apply softmax so scores become weights that sum to 1
4. Take a weighted sum of the value vectors

Attention(Q, K, V) = softmax(Q K^T / sqrt(d_k)) V

Multi-head attention runs this several times in parallel with different learned projections, letting heads specialise (syntax, coreference, position). In decoder LLMs a causal mask stops a token from seeing future tokens.

LLMs are trained to produce the most plausible next token, not to verify truth. A fluent wrong answer scores just as well as a right one during generation.

• Knowledge gaps: rare or niche facts were barely seen in training
• Stale data: the training cutoff hides newer events
• No calibrated uncertainty: the model does not reliably know what it does not know
• Ambiguous or leading prompts push it to fill in blanks
• Sampling randomness can drift into a wrong branch that later tokens then justify
• Long context: relevant details buried mid-prompt may be ignored

Grounding with retrieved sources, allowing "I don't know" and lowering temperature all reduce, but never fully remove, the problem.

RAG changes what the model sees at request time, while fine-tuning changes how the model behaves by updating its weights.

Start with prompting and RAG. Add fine-tuning when you need consistent behaviour that prompting cannot deliver. Many production systems use both.

A RAG system has two phases: an offline indexing phase and an online query phase.

flowchart LR
  subgraph Indexing
    A["Load documents"] --> B["Clean and chunk"]
    B --> C["Create embeddings"]
    C --> D[(Vector index)]
  end
  subgraph Query
    Q["User question"] --> E["Embed query"]
    E --> F["Retrieve top-k chunks"]
    D --> F
    F --> G["Rerank and filter"]
    G --> H["Build prompt"]
    H --> I["LLM generates answer"]
    I --> J["Answer with citations"]
  end

1. Index: load, clean, chunk, embed and store documents with metadata
2. Retrieve: embed the question and fetch the nearest chunks
3. Rerank (optional): a cross-encoder reorders results by true relevance
4. Augment: place the best chunks in the prompt with instructions to use only them
5. Generate: the LLM answers and cites sources

Quality problems usually come from the retrieval half, so evaluate retrieval separately from generation.

Chunking decides what each embedding represents and what text the model receives. It directly controls retrieval quality.

• Too large: one vector blends several topics, so matches get fuzzy and prompt tokens are wasted
• Too small: a chunk loses the surrounding meaning, such as which product a sentence refers to

Common strategies are fixed-size splits, recursive splitting on headings and paragraphs, and semantic splitting where topic changes mark the boundary. A 10 to 20 percent overlap helps keep sentences from being cut off mid-thought.

Attach metadata such as title, section and date to each chunk so you can filter and cite properly.

Fix retrieval before touching the generator, because the model can only be as good as the chunks it is given.

1. Improve chunking to follow headings and keep context together
2. Use hybrid search (BM25 plus vectors) for names, IDs and error codes
3. Add a reranker over the top 20 to 50 results
4. Rewrite queries: expand abbreviations, split multi-part questions
5. Filter by metadata such as product, language or date
6. Pick a better embedding model for your domain and language
7. Tune top-k: enough recall, not so many chunks that noise drowns the answer

Measure with a labelled question set using recall@k and mean reciprocal rank, so each change is judged by numbers instead of impressions.

Both control randomness, but they act at different points. Temperature reshapes the whole probability distribution. Top-p (nucleus sampling) cuts the candidate list to the smallest group of tokens whose probabilities add up to p, then samples from that group.

Providers usually advise adjusting one of them, not both, because their effects overlap and become hard to reason about. A practical starting point is to leave top-p at its default and tune temperature, then change one setting at a time while checking output on a fixed test set.

Most LLMs use subword tokenization, commonly Byte Pair Encoding (BPE). The tokenizer is trained before the model, and its vocabulary is fixed afterwards.

1. Start with single characters or bytes as the base vocabulary
2. Count the most frequent adjacent pair in the training text
3. Merge that pair into a new token
4. Repeat until the vocabulary reaches the target size, often 30K to 200K entries

At run time, text is split with the learned merges. Common words become one token, while rare words break into pieces, so there are no out-of-vocabulary failures.

The side effects matter in interviews: non-English text, numbers and code can use more tokens, and models can struggle with letter-level tasks like counting characters in a word.

The limit comes from compute, memory and training, not an arbitrary rule.

• Attention cost: standard self-attention compares every token with every other, so compute grows roughly with the square of the sequence length
• KV cache memory: stored keys and values grow with each token and with each concurrent request
• Training length: models perform best at lengths they were trained on, and position encodings may extrapolate poorly
• Cost and latency: longer inputs mean slower first tokens and higher bills

Techniques such as sparse or sliding-window attention, FlashAttention and position-scaling methods push limits higher. Even with huge windows, models can miss details buried in the middle of long inputs, so retrieval is still useful.

Chain-of-thought (CoT) prompting asks the model to write out intermediate steps before the final answer. Since each generated token becomes input for the next, the written steps work like scratch paper, letting the model carry partial results forward instead of jumping straight to a guess.

Q: A shop sells pens at $3 each. I buy 7 and pay with $50. Change?
Think step by step, then give the final answer.

It helps most on multi-step arithmetic, logic and planning problems, and does little for simple lookups.

Trade-offs: more output tokens, higher latency and cost, and the written reasoning is not guaranteed to be a faithful picture of what happened inside the model. Dedicated reasoning models do this thinking internally.

It depends on the API and your application. Some APIs return an error that the request is too long. Others, or chat frameworks you wrap around them, silently truncate the oldest messages, which can drop instructions or key facts without any warning.

Ways to handle it deliberately:

• Summarise older turns and keep the summary
• Sliding window: keep the system prompt plus the last N turns
• RAG: retrieve only relevant passages instead of pasting whole documents
• Map-reduce: process a long document in parts, then combine results
• Trim tool output before adding it to history

Count tokens before sending and reserve space for the reply.

Reinforcement Learning from Human Feedback (RLHF) nudges a model toward answers people prefer, such as helpful, honest and harmless ones.

1. Supervised fine-tuning: train on human-written ideal responses
2. Reward model: humans rank several outputs for the same prompt, and a model learns to predict those preferences
3. Policy optimisation: the LLM is tuned with reinforcement learning, commonly PPO, to maximise the reward score
4. A KL penalty keeps it from drifting too far from the original model and gaming the reward

Newer alternatives skip the separate reward model: DPO optimises directly on preference pairs, and RLAIF or constitutional methods use AI feedback guided by written principles.

A pre-trained base model often just continues your text. Instruction tuning, usually followed by preference tuning such as RLHF, is what turns it into a usable assistant.

Pre-training costs millions in compute and is done by a few labs. Fine-tuning is the stage most teams can afford.

LoRA (Low-Rank Adaptation) freezes the original weights and trains two small matrices per targeted layer whose product represents the weight update. Often well under 1 percent of the parameters are trained.

Choose LoRA when hardware is limited, you serve several customised variants, or you need quick experiments. Choose full fine-tuning when you have plenty of data and compute and need to shift the model deeply.

Function calling is a round trip between your app, the model and your tools. The model only asks for a call. Your code decides whether to run it.

sequenceDiagram
  participant U as User
  participant A as App
  participant M as LLM
  participant T as Tool/API
  U->>A: Weather in Charlotte?
  A->>M: Prompt + tool schemas
  M-->>A: Call get_weather(city=Charlotte)
  A->>T: Execute get_weather
  T-->>A: 72F, sunny
  A->>M: Tool result
  M-->>A: Final answer text
  A-->>U: It is 72F and sunny

1. Send the user message along with tool definitions (name, description, JSON schema)
2. The model replies with a tool call instead of text
3. Validate the arguments, then execute the function
4. Return the result to the model as a tool message
5. The model writes the final response, or requests another call

Always validate arguments and check permissions, since the model's output is untrusted input.

An agent repeats a think, act, observe cycle until it decides the goal is met or a limit is hit.

flowchart TD
  A["Goal from user"] --> B["Model plans next step"]
  B --> C{Needs a tool?}
  C -- Yes --> D["Call tool"]
  D --> E["Observe result"]
  E --> F["Update context and memory"]
  F --> B
  C -- No --> G["Produce final answer"]
  F --> H{Step or budget limit hit?}
  H -- Yes --> I["Stop and report"]

1. Receive the goal and instructions
2. The model reasons about the next action
3. If a tool is needed, it emits a call and the app runs it
4. The result is appended to context
5. Loop until a final answer, a failure, or a limit

Common failure modes are infinite loops, repeating failed calls, drifting from the goal and runaway cost. Mitigate with max steps, token budgets, timeouts, logging and human approval for irreversible actions.

Cosine similarity measures the angle between two vectors, ignoring their length. It ranges from -1 (opposite) through 0 (unrelated) to 1 (same direction). In semantic search it ranks stored embeddings by how closely their direction matches the query embedding.

import numpy as np

def cosine(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

Ignoring magnitude is useful because a long document and a short one on the same topic should still match. If embeddings are normalised to length 1, cosine similarity equals the dot product, which is faster and what many vector databases use internally. Always compare vectors produced by the same embedding model, since scores from different models are not comparable.

Use hybrid search when queries mix meaning with exact terms. Vector search is good at paraphrases but can miss precise strings, while keyword search (BM25) nails exact matches but misses synonyms.

The two ranked lists are merged, commonly with Reciprocal Rank Fusion, which scores each document by its position in each list. Many teams then rerank the merged top results.

It is a strong default for enterprise search over product names, ticket IDs, legal clauses and code.

Diffusion models learn to remove noise. During training, real images are gradually corrupted with Gaussian noise and a network learns to predict the noise added at each step.

1. Start from pure random noise
2. Encode the text prompt into an embedding
3. At each step, predict the noise and subtract a little of it, guided by the prompt
4. Repeat for tens of steps until a clean image appears

Classifier-free guidance strengthens how closely the image follows the prompt. Latent diffusion (used by Stable Diffusion) runs this process in a compressed latent space and decodes to pixels at the end, which makes it much cheaper than working on full-resolution pixels.

No single fix works, so layer several controls.

1. Ground answers with RAG and instruct the model to use only the supplied context
2. Require citations and check that quoted spans exist in the sources
3. Allow abstention: tell it to say "not found in the documents"
4. Lower temperature for factual tasks
5. Constrain output with JSON schemas or enumerated choices
6. Add a verification step, such as a second pass or rule-based checks on numbers and IDs
7. Route high-risk cases to human review
8. Evaluate continuously on a regression set with groundedness metrics

Choose the controls by risk: a marketing draft needs far less than a medical or financial answer.

Prompt injection is an attack where text the model reads contains instructions that override the developer's intent. Direct injection comes from the user ("ignore previous instructions"). Indirect injection hides in retrieved web pages, emails or documents the model processes.

The danger grows when the model can use tools: a poisoned document could tell an agent to send data to an attacker.

• Least privilege: give tools only the permissions the task needs
• Treat retrieved text as data, clearly separated from instructions
• Confirm sensitive or irreversible actions with the user
• Filter and validate inputs and outputs, including links and tool arguments
• Keep secrets out of prompts and sandbox code execution
• Log and monitor tool calls for odd behaviour

There is no complete fix today, so design assuming the model can be tricked.

Evaluate the whole system, not just the model, with a mix of offline and online checks.

1. Build a golden set of realistic inputs with expected answers or rubrics
2. Automated checks: exact match, schema validity, regex, unit tests for code outputs
3. LLM-as-judge with a clear rubric for tone, correctness or faithfulness, spot-checked against human labels
4. RAG metrics: context recall, context precision, answer faithfulness, answer relevance
5. Human review of samples, especially edge cases
6. Regression runs every time the prompt, model or retriever changes
7. Online monitoring: thumbs up or down, escalation rate, latency, cost

Classic overlap scores like BLEU or ROUGE are weak for open-ended answers, so use them sparingly.

Measure first. Track time to first token, tokens per second and cost per request, then fix the biggest contributor. Quality must be re-checked after every optimisation.

Neither wins everywhere. The right choice depends on task difficulty, volume, latency and data sensitivity.

A common pattern is a cascade: try the small model first, and escalate to the large one when confidence is low or the task is complex. Test both on your own evaluation set, since public benchmarks rarely match your workload. Also factor in operational effort, because self-hosting a small model adds deployment, scaling and monitoring work that a hosted API handles for you.

First decide whether the failure is in retrieval or generation. Log the retrieved chunks for a bad answer and read them.

1. Right chunk missing? It is a retrieval problem. Check chunk size, embedding model, hybrid search, metadata filters, top-k and query rewriting.
2. Right chunk present but ignored? It is a generation problem. Shorten the context, put key chunks first, tighten the instructions, or add a reranker.
3. Document never indexed? Check ingestion errors, parsing of PDFs and tables, and stale indexes.
4. Contradicting chunks? Deduplicate and prefer newer versions using metadata.
5. Answer invents details? Require citations and an explicit not-found response.

Turn each failure into a test case so the same bug cannot return unnoticed.

Most modern chat LLMs are decoder-only because one next-token objective scales cleanly and handles many tasks through prompting.

Encoder-only models are still popular for embeddings and rerankers, where understanding a full text matters more than generating one. Encoder-decoder models remain a good fit for tasks with a clear input-to-output mapping, such as translating a sentence or condensing a report into a short summary.

Quantization stores model weights (and sometimes activations) at lower numeric precision, for example 16-bit floats reduced to 8-bit or 4-bit integers. The goal is a smaller, faster model that fits on cheaper hardware.

The memory arithmetic is simple. A 7-billion-parameter model needs about 14 GB at 16-bit (2 bytes per weight) but only around 3.5 GB at 4-bit, plus runtime overhead.

The trade-off is a small, task-dependent quality drop, most visible at 4-bit and below. Always re-run your evaluation set after quantizing.

LLMs are probabilistic, so even a strong prompt cannot guarantee safe, on-topic, correctly formatted output every time. Guardrails are checks placed around the model to catch problems the prompt alone cannot prevent.

• Input side: detect prompt injection, redact PII, block off-topic or disallowed requests
• Output side: validate JSON schemas, run moderation, check groundedness against sources, filter leaked secrets
• Action side: limit tool permissions, cap spend, require approval for irreversible steps
• Operations: log decisions, alert on spikes and keep an audit trail

Treat them as defence in depth. Each layer is imperfect, but together they reduce incidents, protect users and make compliance reviews easier.

Function calling is a model capability: you pass tool schemas in the request and the model returns a structured call. MCP is an open protocol that standardises how an AI application discovers and uses tools, data resources and prompt templates from external servers.

They are complementary: an MCP client typically converts the server's tools into function-calling definitions for the model, then routes the model's calls back to the server. Remember that MCP servers still need authentication, scoped permissions and trust review.