AI / AI model fine tuning concepts Interview questions

Fine-tuning means taking an already pre-trained model and continuing its training on a smaller, task-specific dataset, so it adapts to a particular domain, style, or task. The starting point is the pre-trained weights, not random values.

The model already knows general patterns such as grammar, code structure, or visual features. That is why fine-tuning needs far less data and compute than training from scratch. A few hundred to a few thousand good examples is often enough for a focused task.

Typical uses: classifying support tickets, making an LLM reply in a company's tone, or extracting fields into a fixed JSON schema.

A foundation model is a large model trained on broad, mostly unlabeled data with a self-supervised objective, such as predicting the next token or a masked token. It learns general representations that can be adapted to many downstream tasks.

Three common families you will meet in interviews:

Fine-tuning, prompting, or RAG then specialize this general model for a real use case.

Transfer learning reuses knowledge learned on one task or dataset to improve performance on another. Fine-tuning is one form of it: you keep the learned weights and keep training them.

Two common styles exist:

Example: take an image model pre-trained on ImageNet, freeze it, and train a new 5-class head for plant diseases. If accuracy plateaus, unfreeze the top layers and train them at a low learning rate.

The main types differ by which weights change and what signal drives training:

• Full fine-tuning - every weight is updated.
• Parameter-efficient fine-tuning (PEFT) - LoRA, adapters, prefix and prompt tuning update a tiny subset.
• Supervised / instruction tuning - learns from prompt-response pairs.
• Preference tuning - RLHF or DPO learn from human or AI preference comparisons.
• Continued pretraining - next-token training on raw domain text.

Real projects often chain them, for example continued pretraining, then SFT with LoRA, then DPO.

SFT trains a model on labeled input-output pairs using the same cross-entropy loss as pre-training, but on curated examples. For chat LLMs, each example is a prompt plus the ideal response.

The model learns to imitate those responses. SFT is usually the first stage after pre-training and also the starting point for RLHF or DPO.

A typical JSONL record looks like this:

{"messages": [
  {"role": "user", "content": "Summarize: The server crashed at 2 AM..."},
  {"role": "assistant", "content": "A 2 AM server crash caused downtime; a patch fixed it."}
]}

Instruction tuning is SFT on a broad collection of (instruction, response) pairs across many tasks, so the model learns to follow natural-language instructions instead of just continuing text. FLAN and the SFT stage of InstructGPT are well-known examples.

A base model given "Translate to French: hello" may keep writing more English. An instruction-tuned model simply answers. Task variety matters because it helps the model generalize to instructions it never saw in training.

Datasets such as FLAN pack many tasks into instruction templates, for example summarization, sentiment, and question answering. Mixing task types and phrasings is what lets one model handle a request it has never seen verbatim.

PEFT adapts a model by training only a very small number of extra or selected parameters, often under 1-2% of the total, while the rest of the pre-trained weights stay frozen.

Why teams use it:

• Lower GPU memory, because no optimizer states are kept for frozen weights.
• Tiny checkpoints, megabytes instead of gigabytes.
• Many task adapters can share one base model.
• Less catastrophic forgetting.

Common methods: LoRA, adapters, prefix tuning, prompt tuning, IA3, and BitFit.

In interviews, mention the trade-off: PEFT can slightly trail full fine-tuning on very large domain shifts, but for most task adaptation the gap is small and the cost savings are large.

Low-Rank Adaptation (LoRA) freezes a pre-trained weight matrix W and learns its update as the product of two small matrices: ΔW = B x A. If W is d x k, then A is r x k and B is d x r, with the rank r much smaller than d and k.

The forward pass becomes h = Wx + (alpha / r) * B(Ax). A starts random and B starts at zero, so training begins from the exact behavior of the base model.

Example: a 4096 x 4096 layer has about 16.8M weights. At rank 8, LoRA trains only 65,536 of them (about 0.4%).

from peft import LoraConfig, get_peft_model

config = LoraConfig(r=8, lora_alpha=16,
                    target_modules=["q_proj", "v_proj"],
                    lora_dropout=0.05, task_type="CAUSAL_LM")
model = get_peft_model(model, config)

QLoRA fine-tunes LoRA adapters on top of a base model that is quantized to 4-bit. The 4-bit base stays frozen, the adapters are kept in 16-bit, and gradients flow through the dequantized weights into the adapters.

Three ideas make it work:

• NF4 - a 4-bit NormalFloat data type suited to normally distributed weights.
• Double quantization - quantizes the quantization constants too.
• Paged optimizers - move optimizer state to CPU RAM during memory spikes.

In the QLoRA paper (Dettmers et al., 2023), this let a 65B model be fine-tuned on a single 48 GB GPU.

The learning rate sets how large a step the optimizer takes on every update. In fine-tuning it must be smaller than in pre-training, because the weights are already good and big steps can wreck them.

Typical starting points: full fine-tuning of LLMs around 1e-5 to 5e-5, and LoRA around 1e-4 to 2e-4. LoRA tolerates more because its adapters start near zero. Most setups add a short warmup and then a cosine or linear decay.

If the loss spikes right after training starts, the rate is probably too high. If it barely moves after a few hundred steps, it is probably too low. A quick sweep over two or three values saves a lot of wasted GPU hours.

Catastrophic forgetting is when a model loses earlier abilities after being fine-tuned on a new task, because gradient updates overwrite weights that earlier skills depended on.

Example: a model fine-tuned heavily on legal contracts starts making mistakes on basic arithmetic or drops its multilingual ability.

Common mitigations: lower learning rate, fewer epochs, LoRA with a frozen base, mixing in general replay data, and regularization such as EWC.

A practical check is to run the fine-tuned model on held-out prompts that worked before fine-tuning. A big drop compared with the base model is the clearest evidence of forgetting.

Overfitting happens when the model memorizes training examples instead of learning the underlying pattern. Training loss keeps dropping, but validation loss flattens or rises. Small datasets and too many epochs make it likely.

Warning signs: the model repeats training answers word for word, and does poorly on paraphrased inputs.

Fixes: more diverse data, fewer epochs, early stopping, weight decay, dropout, or a lower LoRA rank.

Concrete example: a model trained for 10 epochs on 300 support replies may quote them nearly verbatim, yet fail when a customer words the same problem differently. Checking validation loss every 50-100 steps catches this early.

It depends on the tuning method. Each format carries only what that method needs:

Whatever the format, the data should match how the model will be used in production.

Every example should show the exact behavior you want in production, including tone, length, and edge cases such as refusals or "I don't know" answers.

Reinforcement Learning from Human Feedback aligns a model's outputs with human preferences. Humans rank several responses to the same prompt, a reward model learns to predict those rankings, and the language model is then optimized (commonly with PPO) to earn higher reward.

A KL penalty against the SFT model keeps the policy from drifting into odd outputs that merely game the reward model.

RLHF helped move raw instruction-following models toward the helpful chat behavior users expect, as described in the InstructGPT work. Its main downsides are labeling cost and complexity, since several models must be held in memory at once.

The validation set is data the model never trains on, used during training to monitor generalization. You use it to compare hyperparameters, pick the best checkpoint, and trigger early stopping.

It is not the same as the test set. The test set is touched once at the end for an unbiased final score. If you tune against the test set, it stops being unbiased. Common splits are 80/10/10, or 90/5/5 for large datasets.

Example: with 10,000 examples, a 90/5/5 split keeps 500 for validation and 500 for the final test. Evaluating on validation every 100 steps shows exactly when improvement stops.

An epoch is one full pass over the training set. Batch size is how many examples are processed before one weight update. Steps per epoch = dataset size / batch size.

Example: 10,000 examples with batch size 16 gives 625 steps per epoch.

LLM SFT usually runs just 1-3 epochs, because more passes encourage memorization. A larger batch gives steadier gradients but needs more memory, which is where gradient accumulation helps.

Rule of thumb: if validation loss starts climbing in epoch 2 or 3, stop there. On very small datasets, the best checkpoint may even come before the first epoch ends.

Perplexity is the exponential of the average negative log-likelihood per token. It tells you how surprised the model is by held-out text. Lower is better. A perplexity of 10 means the model is about as unsure as choosing uniformly among 10 tokens at each step.

Limits: it is only comparable with the same tokenizer and the same text, and it says nothing about helpfulness or factual accuracy. It suits continued pretraining well, and instruction-tuned models need task-based evaluation too.

The tokenizer converts text into the token IDs that the model's embedding layer expects. You must use the same tokenizer as the base model. Swapping it breaks the mapping between IDs and learned embeddings.

Also keep special tokens and the chat template consistent. If your training examples lack the EOS token, the model never learns where to stop. When you add new tokens, resize the embedding matrix and train it:

tokenizer.add_tokens(["<ticket_id>"])
model.resize_token_embeddings(len(tokenizer))

Padding is another tokenizer detail. Decoder models usually need pad_token set (often to the EOS token) and a consistent padding side, otherwise batches can be misaligned.

Prompt tuning learns a small set of continuous soft prompt embeddings that are prepended to the input embeddings, while the entire model stays frozen. Only those vectors are trained, for example 20-100 virtual tokens times the hidden size.

Soft prompts are not human-readable text. Lester et al. (2021) found the method gets closer to full fine-tuning as model size grows.

Prompt tuning is attractive because one frozen model can serve many tasks, each with its own tiny prompt file of a few kilobytes. Its weakness is sensitivity to initialization and weaker results on smaller models, so most practitioners reach for LoRA first.

Adapter tuning inserts small bottleneck modules inside each transformer layer and trains only those, with the base model frozen. Each adapter projects the hidden state down to a small dimension, applies a nonlinearity, projects back up, and adds a residual connection (Houlsby et al., 2019).

Unlike LoRA, adapters add extra layers that stay in the forward path, so they add a little inference latency and cannot be folded into the original weights.

Adapters were among the first PEFT methods and influenced LoRA. LoRA became more popular because it adds zero latency after merging and is easier to configure.

DPO trains a model directly from preference pairs (a chosen and a rejected response) using a classification-style loss. There is no separate reward model and no RL sampling loop.

The loss pushes up the log-probability margin of the chosen response over the rejected one, measured relative to a frozen reference model and scaled by beta. You need an SFT model as the reference and a dataset of preference pairs. It is simpler and usually more stable than PPO-based RLHF.

Fine-tune when prompting has plateaued and the gap is about behavior: a strict output format, a consistent tone, a specialized task, or a need for shorter prompts and lower latency from a smaller model. Always try better prompts and few-shot examples first, because they are cheap and reversible.

Fine-tuning changes the model's weights. RAG leaves the model alone and injects retrieved documents into the prompt at inference time.

They are not exclusive. Many systems fine-tune for tone and format, then use RAG for facts.

Decision shortcut: if the model needs to know something that changes or is private, use RAG. If it needs to behave differently, such as following a strict schema or house style, fine-tune. Many teams also fine-tune the model to use retrieved context faithfully.

A full update to a d x k matrix needs d*k parameters. LoRA replaces it with two factors, B (d x r) and A (r x k), which need only r*(d + k). This works because weight updates during adaptation tend to have low intrinsic rank.

Optimizer states (Adam keeps two per parameter) are held only for these small matrices, which is where most of the memory saving comes from.

In practice, rank 8-16 is a common starting point, and you raise it only if the adapter underfits.

It attacks each memory component. A 65B model in 16-bit needs about 130 GB for weights alone. In 4-bit NF4 that drops to roughly 33 GB, and double quantization shaves a bit more.

1. The base model is loaded in 4-bit and frozen.
2. Small 16-bit LoRA adapters are the only trainable weights, so gradients and Adam states are tiny.
3. Paged optimizers spill optimizer state to CPU RAM when GPU memory spikes.
4. Gradient checkpointing recomputes activations instead of storing them.

During the forward and backward pass, 4-bit blocks are dequantized to bf16 on the fly for the matrix multiplication.

The QLoRA paper reported that 4-bit NF4 with adapters matched 16-bit fine-tuning quality on the benchmarks it tested, so the memory saving did not cost accuracy there.

The loss only measures the new task. Gradient updates move shared weights to lower that loss, and nothing in the objective protects what older skills relied on. A high learning rate, many epochs, and narrow data all make the drift worse.

Interview tip: say that you measure forgetting explicitly, by running a fixed general evaluation set before and after fine-tuning, rather than assuming the mitigation worked.

Start from the published default for your method, then run a short sweep rather than guessing. Common grids are {1e-5, 2e-5, 5e-5} for full fine-tuning and {1e-4, 2e-4} for LoRA.

1. Pick a grid and run each value for a few hundred steps.
2. Use warmup (roughly 3-10% of steps) followed by cosine or linear decay.
3. Read the loss curve: spikes or NaN mean too high, a nearly flat curve means too low.
4. Re-check if you change batch size, since larger batches usually tolerate a higher rate.

Do not tune the rate on the test set. Use validation loss, and confirm the winner with the task metric, since the lowest loss does not always give the best answers.

Full fine-tuning updates every weight. PEFT trains a tiny add-on and leaves the base untouched.

Rule of thumb: begin with LoRA or QLoRA. Move to full fine-tuning only if a well-tuned adapter at a higher rank still underperforms on your validation metric and you can afford the extra memory, storage, and forgetting risk. Many production teams never need to make that jump.

RLHF runs in three stages: SFT, reward modeling, and policy optimization.

flowchart TD
    A["Pre-trained model"] --> B["SFT on human demonstrations"]
    B --> C["Sample several responses per prompt"]
    C --> D["Humans rank the responses"]
    D --> E["Train reward model on rankings"]
    E --> F["PPO optimizes policy with KL penalty"]
    F --> G["Aligned model"]

1. SFT gives a model that can answer in the right format.
2. Reward model is trained on human rankings to score any response.
3. PPO generates responses, scores them with the reward model, and updates the policy while a KL term keeps it near the SFT model.

RLHF needs a policy, a reference model, a reward model, and often a value model in memory at once, which is a big reason DPO became popular.

Both learn from preferences. RLHF does it with an explicit reward model and RL, while DPO folds the preference signal straight into a supervised-style loss.

Rule of thumb: pick DPO when you already have preference pairs and want a simple, stable pipeline. Pick online RL when the model must explore and improve beyond the examples in a fixed dataset, such as with verifiable rewards.

Most knowledge comes from pre-training. Fine-tuning mainly teaches format, style, and task behavior, so a small clean set can be enough. The LIMA study (2023) fine-tuned a 65B LLaMA model on about 1,000 carefully curated examples and got strong instruction-following.

Noise hurts directly. Inconsistent labels teach inconsistent behavior, wrong answers get imitated, and duplicates push the model toward memorization.

• Check answers for correctness and one consistent style.
• Keep prompts diverse.
• Deduplicate.
• Review the length distribution.

Example: 800 expert-written support replies in one consistent voice usually beat 20,000 scraped replies of mixed quality. Time spent on review and cleaning pays back more than collecting extra volume.

1. Define the target behavior and collect or write representative prompts.
2. Convert to chat-style JSONL with consistent roles.
3. Apply the base model's chat template so training matches inference.
4. Clean and deduplicate, then filter bad or off-topic samples.
5. Balance categories so one task does not dominate.
6. Split into train, validation, and test, grouping related items together.
7. Check token lengths against your max sequence length.

{"messages": [{"role": "system", "content": "You are a billing assistant."},
             {"role": "user", "content": "Why was I charged twice?"},
             {"role": "assistant", "content": "A duplicate charge usually reverses in 3-5 days..."}]}

Before training, print a few formatted examples exactly as the model will see them, including special tokens. Template bugs are one of the most common and most avoidable fine-tuning failures.

Use several layers, because no single score tells the whole story:

• Task metrics on a held-out test set.
• LLM-as-judge with a rubric, or pairwise comparison against a baseline.
• Human review of a sample, especially failure cases.
• Regression checks on general benchmarks to catch forgetting.
• Safety checks before release.

Always compare against the base model with a good prompt, not against nothing.

Keep the evaluation set fixed across experiments so scores stay comparable, and stratify it by category so you can see which kinds of requests improved and which got worse.

Log training and validation loss at regular intervals. When training loss keeps falling and validation loss turns upward, the model has started memorizing. Also track the task metric on the validation set and spot-check outputs for memorized phrases.

In Hugging Face Trainer you can automate the response:

args = TrainingArguments(
    eval_strategy="steps", eval_steps=100,
    save_strategy="steps", save_steps=100,
    load_best_model_at_end=True,
    metric_for_best_model="eval_loss")
trainer = Trainer(..., args=args,
    callbacks=[EarlyStoppingCallback(early_stopping_patience=3)])

Do not rely on training loss alone. A model can reach very low training loss simply by memorizing, so also run a few unseen prompts after each checkpoint and compare the answers with the base model. Keep several checkpoints so you can roll back to the best one.

In SFT you usually want the model to learn to produce the answer, not to predict the user's question. So the loss is computed only on response tokens. Otherwise capacity is spent modeling prompts, and with long prompts and short answers the useful gradient signal gets diluted.

In Hugging Face, labels set to -100 are ignored by the cross-entropy loss:

labels = input_ids.clone()
labels[:prompt_len] = -100

Training on the prompt too is sometimes tried with small datasets, but the evidence is mixed.

Check your framework's defaults. TRL's SFTTrainer offers options to train on completions or assistant messages only, while a plain Trainer computes loss on every token unless you set the labels yourself.

Rank (r) is the inner dimension of the two LoRA matrices and sets adapter capacity. Alpha is a scaling numerator, and the update is multiplied by alpha / r before being added to the frozen output.

A common recipe is r between 8 and 64, with alpha equal to r or 2r.

If a LoRA run underfits, raise rank or learning rate. If it overfits or drifts from base behavior, lower alpha or rank. Change one knob at a time so you can tell which one helped.

The original LoRA paper adapted only the attention query and value projections. QLoRA later found that adapting all linear layers is needed to match full fine-tuning quality.

For Llama-style models that means q_proj, k_proj, v_proj, o_proj plus the MLP layers gate_proj, up_proj, and down_proj. More targets mean more trainable parameters and memory, so choose based on your budget. PEFT can select them for you:

LoraConfig(r=16, lora_alpha=32, target_modules="all-linear",
           task_type="CAUSAL_LM")

Trade-off example: attention-only LoRA on a 7B model trains a few million parameters and runs fast. Adding the MLP layers multiplies that several times over, but it usually helps on tasks that need real new skills. Start with all linear layers at a modest rank, then trim if memory is tight.

It runs several small micro-batches forward and backward, summing gradients, and applies one optimizer step afterward. The effective batch size is micro-batch x accumulation steps x number of GPUs.

Example: micro-batch 4, accumulation 8, one GPU gives an effective batch of 32 without ever holding 32 sequences of activations in memory. The cost is time: each update takes longer.

TrainingArguments(per_device_train_batch_size=4,
                  gradient_accumulation_steps=8)

One caveat: some library versions have had bugs in how loss is averaged across micro-batches when sequence lengths differ, so keep your training library up to date. Also note that learning-rate schedules count optimizer steps, not micro-batches, so warmup length should be set accordingly.

Lower precision halves the memory for weights and activations and speeds up tensor-core math. bf16 keeps the same 8-bit exponent as fp32, so it handles the same value range and needs no loss scaling. fp16 has a narrower range and can overflow or underflow.

Master weights and Adam states are often kept in fp32. bf16 needs a GPU generation such as A100 or RTX 30-series and newer.

In Hugging Face you enable it with bf16=True in TrainingArguments. On older GPUs without bf16 support, fall back to fp16 with the built-in loss scaler.

Both train continuous vectors with a frozen model, but they inject them in different places.

Prefix tuning came first (Li and Liang, 2021) and was designed for generation tasks. Prompt tuning (Lester et al., 2021) simplifies it by touching only the input layer, so it has the fewest trainable parameters but depends more on model scale. Both keep the base frozen, so one model can serve many tasks by swapping small prefix files at request time.

Add up four parts: weights, gradients, optimizer states, and activations. For full fine-tuning with AdamW in mixed precision, a common rule is about 16 bytes per parameter: 2 for bf16 weights, 2 for gradients, 4 for an fp32 master copy, and 8 for the two Adam moments.

Activations scale with batch size and sequence length. Gradient checkpointing trades extra compute for much lower activation memory.

The loss spikes, diverges, or turns into NaN. Even when it stays finite, large steps erase pre-trained knowledge, so outputs can become gibberish or noticeably worse than the base model.

1. Lower the learning rate by 3-10x.
2. Add or lengthen warmup.
3. Enable gradient clipping, for example max_grad_norm=1.0.
4. Restart from the last healthy checkpoint rather than the corrupted one.

Recovery matters too. If the run already diverged, reload the last stable checkpoint instead of continuing, because a few bad updates can leave the model in a poor region that more training rarely repairs. Log the gradient norm so you can spot trouble before the loss visibly breaks.

Merging computes W' = W + (alpha / r) x B x A for every adapted layer, so inference needs no extra adapter computation. With PEFT it takes a few lines:

base = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=torch.bfloat16)
model = PeftModel.from_pretrained(base, "adapter_dir")
merged = model.merge_and_unload()
merged.save_pretrained("merged_model")

For QLoRA, load the base in bf16 or fp16 for the merge, not in 4-bit, to avoid rounding errors from merging into quantized weights. If you want to hot-swap adapters per task, keep them unmerged.

After merging, run a quick sanity test comparing merged outputs with the adapter-attached model on a few prompts. They should match closely, apart from tiny numerical differences.

Leakage inflates your scores and hides real weaknesses. Guard against it before training starts:

• Deduplicate first, including near-duplicates (MinHash or embedding similarity), then split.
• Split by group - by customer, document, conversation, or time - not by random row.
• Check for benchmark contamination against public test sets.
• Never tune hyperparameters on the test set.
• For synthetic data, make sure test prompts were not seeded from training items.

A common blind spot is time. For logs or tickets, train on older data and test on newer data, since random splits let the model peek at patterns from the future. Report the contamination checks you ran alongside the scores, so reviewers can trust the numbers.

Distillation trains a smaller student to mimic a larger teacher, either through the teacher's soft probabilities or through text the teacher generates. In LLM practice it is often just SFT of a small model on teacher outputs, so it is fine-tuning with a different data source.

Watch for two risks: the student inherits the teacher's mistakes, and the teacher's license may restrict using its outputs for training. Filter the generated data for quality before training.

Example: use a large model to write 20,000 answers to your prompts, filter them with a judge or rules, then fine-tune a 7B model on them. The small model is cheaper to serve and is often close to the teacher on that narrow task.

Work through the likely causes in order, cheapest checks first:

1. EOS token missing or wrong in training targets, so the model never learns to stop.
2. Chat template mismatch between training and inference.
3. Generation settings: very low temperature; try repetition_penalty=1.1 or no_repeat_ngram_size.
4. Low-diversity or duplicated training data.
5. Overtraining: compare an earlier checkpoint.

Typical case: a model fine-tuned on support emails loops the same sentence at the end of every reply, and the training file shows many examples with no EOS token. Adding it and retraining fixes the loop, while changing temperature only masks it. Rule out data problems before tweaking decoding parameters.

A healthy project is a loop, not a straight line.

flowchart TD
    A["Define task and success metric"] --> B["Baseline with prompting or RAG"]
    B --> C["Collect and clean data"]
    C --> D["Choose base model and method"]
    D --> E["Train and tune hyperparameters"]
    E --> F["Evaluate against baseline"]
    F --> G{Meets target?}
    G -- No --> C
    G -- Yes --> H[Deploy]
    H --> I["Monitor and retrain"]

Most failures trace back to the data step, which is why evaluation usually sends you back there rather than to the optimizer.

Track every run (data version, hyperparameters, metrics) so you can reproduce the best result and explain why one run beat another.

Research such as Qi et al. (2023) found that even benign fine-tuning can weaken a model's safety behavior. Treat safety as something you test, not something you inherit.

• Mix safety and refusal examples into the training data.
• Use a lower learning rate, LoRA, or fewer epochs to limit drift.
• Run a safety and red-team evaluation before and after training.
• Keep input and output moderation in front of the model.
• Review training data for harmful content, and limit who can access the weights.

Treat the safety check as a release gate. If the fine-tuned model refuses noticeably less often than the base on your red-team prompts, fix the data or training setup before shipping, not after.

Example: feeding 2B tokens of medical journals to teach terminology is continued pretraining. Teaching the model to answer patient-intake questions in a fixed format is SFT.

They are often combined: continue pretraining on domain text, run SFT for behavior, then apply DPO for preferences. Skipping continued pretraining is fine when the base model already handles the domain vocabulary well.

Cost scales with model size, tokens processed, and GPU time, so attack each:

• Start with the smallest base model that meets your metric.
• Use LoRA or QLoRA to fit cheaper GPUs.
• Deduplicate and trim data; quality beats volume.
• Cap sequence length and use sequence packing to avoid padding waste.
• Run 1-3 epochs with early stopping.
• Use spot instances with frequent checkpoints.
• Do hyperparameter sweeps on a data subset.

Measure cost per quality point, not raw cost. A run that costs half but loses five points on your metric may be more expensive once you count the failures it causes downstream. Track tokens trained and GPU-hours for every run so budgets stay predictable.