Prev Next

AI / LoRA trainer Interview questions

Last updated

1. What is LoRA? 2. What is a LoRA trainer? 3. What is low-rank decomposition in LoRA? 4. What is the rank (r) parameter in LoRA? 5. What is lora_alpha? 6. What is a LoRA adapter file? 7. What are the advantages of LoRA over full fine-tuning? 8. What are target modules in LoRA? 9. What is the Hugging Face PEFT library? 10. What is QLoRA? 11. What is LoRA dropout? 12. What are the popular LoRA trainers? 13. What is Kohya_ss? 14. How do you prepare a dataset for LoRA training? 15. What is a trigger word in LoRA training? 16. What learning rate should you use for LoRA training? 17. Define epoch, step, and batch size in LoRA training? 18. What does merging a LoRA into the base model mean? 19. Describe overfitting in LoRA training? 20. What is LoRA weight (strength) at inference time? 21. How does LoRA reduce the number of trainable parameters? 22. How do you choose the right LoRA rank? 23. What is the difference between LoRA and full fine-tuning? 24. What is the difference between LoRA and QLoRA? 25. Explain the internal working of LoRA's forward and backward pass? 26. Why is matrix B initialized to zero in LoRA? 27. Why does rsLoRA scale by alpha divided by the square root of r? 28. What is the difference between LoRA, LoHa, and LoKr? 29. How does DoRA differ from LoRA? 30. Why do we apply LoRA to all linear layers instead of only attention? 31. Explain the internal working of QLoRA? 32. How do you detect overfitting in a LoRA run? 33. How do you troubleshoot CUDA out of memory during LoRA fine-tuning of an LLM? 34. Why does gradient checkpointing help LoRA training? 35. What is the difference between LoRA, adapters, and prompt tuning? 36. How do you choose the learning rate scheduler and warmup for LoRA? 37. Explain the execution flow of a LoRA training run? 38. How does inference differ between merged and unmerged LoRA? 39. What happens when you merge multiple LoRAs? 40. How do you serve many LoRA adapters efficiently? 41. How does caption quality affect character and style LoRA training? 42. What is the difference between text encoder and UNet training in a diffusion LoRA? 43. How can you optimize LoRA training for low VRAM on diffusion models? 44. Which is better and why: bf16 or fp16 for LoRA training? 45. How do you evaluate a trained LoRA? 46. How does batch size affect LoRA training? 47. How do you troubleshoot a LoRA with no visible effect or burnt output? 48. How is LoRA used with DPO or other preference training? 49. Why doesn't LoRA always match full fine-tuning quality? 50. How do you troubleshoot NaN loss in LoRA training?

1. What is LoRA?

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method. It freezes the pretrained model weights and trains small low-rank matrices that are injected into selected layers. It was introduced by Microsoft researchers (Hu et al.) in 2021.

Instead of updating a full weight matrix W, LoRA learns an update ΔW = B × A, where the rank r is far smaller than the matrix dimensions. Only A and B are trained and saved, so an adapter is usually megabytes instead of gigabytes.

Because the base model is never modified, one base checkpoint can serve many tasks, each with its own small adapter.

Take quiz
What does LoRA train while the base model weights stay frozen?
A full lower-precision copy of the model
Only the tokenizer vocabulary
Small low-rank matrices added to selected layers
Only the final softmax layer
Why are LoRA adapter files so much smaller than the base model?
They zip-compress the base weights
They keep only the embedding layer
They store the dataset instead of weights
They store only the low-rank A and B matrices, not the full weights

2. What is a LoRA trainer?

A LoRA trainer is the tool or script that runs the fine-tuning loop for LoRA adapters. It loads the base model, injects adapter layers, feeds the dataset, optimizes only the adapter weights, and writes checkpoints you can load later.

A good trainer handles the boring but critical parts for you:

  • dataset loading, tokenization or image bucketing
  • precision, optimizer and scheduler setup
  • gradient accumulation and memory tricks
  • sample generation or evaluation during training
  • checkpoint saving in a format your inference stack understands

For LLMs, common choices are Hugging Face PEFT with TRL, Axolotl, Unsloth and LLaMA-Factory. For image models, people use Kohya_ss, OneTrainer, ai-toolkit and SimpleTuner.

Take quiz
What does a LoRA trainer actually optimize?
Only the adapter weights while the base stays frozen
Every weight in the base model
Only the tokenizer merges
The GPU driver settings
Which of these is a LoRA trainer for image-generation models?
TensorBoard
OneTrainer
ONNX Runtime
FAISS

3. What is low-rank decomposition in LoRA?

The rank of a matrix is the number of linearly independent rows or columns it has. A low-rank decomposition approximates a big matrix as the product of two thin ones.

LoRA applies this to the weight update, not the weights. For a layer with W of size d × k, the update ΔW is written as B (d × r) times A (r × k), with r much smaller than d and k.

Example: with d = k = 4096 and r = 8, a full update needs 16,777,216 values, but A and B together hold only 65,536.

This works because the LoRA paper's hypothesis is that fine-tuning updates have a low intrinsic rank, so a thin factorization captures most of what matters.

Take quiz
With d = k = 4096 and r = 8, how many values do A and B hold together?
16,777,216
32,768
65,536
8,192
Which hypothesis motivates LoRA?
Pretrained weights are mostly zeros
Every layer needs a full-rank update to adapt
Attention heads must be retrained from scratch
Fine-tuning weight updates have low intrinsic rank

4. What is the rank (r) parameter in LoRA?

The rank r is the inner dimension shared by the two LoRA matrices. It sets how much capacity the adapter has and how big the adapter file will be.

Typical values are 8, 16, 32 and 64 for LLMs. Image LoRAs often sit anywhere from 4 to 128 depending on the subject.

  • Higher r: more trainable parameters, more capacity, bigger file, more VRAM, higher overfitting risk.
  • Lower r: smaller and faster, but can underfit complex tasks.

Adapter size grows linearly with r, so doubling the rank doubles the adapter parameters.

Take quiz
What happens to adapter parameter count if you double r?
It roughly doubles
It stays the same
It is cut in half
It grows with the square of r
Which is a likely downside of a very high rank on a tiny dataset?
The base model forgets its tokenizer
Overfitting and a larger adapter file
Training becomes impossible on any GPU
The adapter can no longer be merged

5. What is lora_alpha?

lora_alpha is a scaling constant. The LoRA update is multiplied by alpha / r before it is added to the frozen layer output, so alpha controls how strongly the adapter speaks.

Example: r = 16 and lora_alpha = 32 gives a scale of 2. With alpha = 16 the scale would be 1.

A common starting point is alpha = r or alpha = 2r. The scaling lets you change the rank without retuning the learning rate from scratch. In Kohya this setting is called network_alpha, and a value lower than network_dim weakens the effective update.

from peft import LoraConfig
cfg = LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05,
                 target_modules="all-linear", task_type="CAUSAL_LM")

Take quiz
With r = 16 and lora_alpha = 32, the LoRA update is multiplied by:
0.5
16
2
32
What does lora_alpha act on?
The tokenizer vocabulary size
The number of training epochs
The dropout probability
The size of the update B x A before it is added to W

6. What is a LoRA adapter file?

A LoRA adapter file holds only the trained low-rank tensors, not the base model. In PEFT, a saved adapter is a folder with adapter_config.json and adapter_model.safetensors. Kohya-style image LoRAs are a single .safetensors file with lora_down and lora_up tensors plus metadata.

Size depends on rank and targeted layers. A 7B LLaMA-style model with r = 16 on all linear layers has about 40 million adapter parameters, roughly 80 MB in fp16.

The adapter must be loaded onto the same base model it was trained on. Its deltas only make sense relative to those exact weights and shapes.

Take quiz
Which pair of files makes up a saved PEFT adapter?
adapter_config.json and adapter_model.safetensors
tokenizer.json and vocab.txt
optimizer.pt and scheduler.pt
pytorch_model.bin and generation_config.json only
Why must a LoRA be loaded onto the base model it was trained on?
Adapters contain a hidden copy of the tokenizer
Its deltas are tied to that base model's weights and shapes
Safetensors refuses mixed models
The GPU cache only accepts one model name

7. What are the advantages of LoRA over full fine-tuning?

LoRA gives you most of the task adaptation at a fraction of the cost:

  • Fewer trainable parameters: often well under 1% of the model.
  • Lower memory: no gradients or optimizer states for frozen base weights.
  • Tiny checkpoints: megabytes per task instead of a full model copy.
  • Swappable: one base model, many adapters.
  • No added latency once the adapter is merged.
  • Less forgetting of the base model's general skills.

The trade-off is a lower capacity ceiling, which matters on very large or knowledge-heavy datasets.

Take quiz
Why does LoRA need less GPU memory than full fine-tuning?
It stores weights in 1-bit by default
It skips the forward pass
Frozen weights need no gradients or optimizer states
It trains only on CPU
Which LoRA property lets one base model serve many tasks?
Shared optimizer states between tasks
A single merged checkpoint for every task
Automatic prompt rewriting
Small swappable adapters per task

8. What are target modules in LoRA?

Target modules are the layers where LoRA injects adapters. In PEFT you list them by name, and any layer whose name matches gets a pair of A and B matrices.

In LLaMA-style models these are the attention projections q_proj, k_proj, v_proj, o_proj and the MLP projections gate_proj, up_proj, down_proj. The original LoRA paper adapted only the query and value projections. The QLoRA paper later showed that covering all linear layers matters more than rank for matching full fine-tuning.

In diffusion models the usual targets are the attention layers (to_q, to_k, to_v, to_out), sometimes plus feed-forward and convolution layers.

LoraConfig(target_modules=["q_proj", "v_proj"])      # minimal
LoraConfig(target_modules="all-linear")               # broad

Take quiz
Which setting in PEFT chooses the layers that receive adapters?
target_modules
task_type
lora_dropout
bias
What did the QLoRA paper find about target layers?
Only the embedding layer needs adapting
Adapting all linear layers matters more than raising rank
Query and value projections are always enough
Adapters should avoid MLP layers

9. What is the Hugging Face PEFT library?

PEFT is Hugging Face's library for parameter-efficient fine-tuning. It supports LoRA and variants such as DoRA, LoHa, LoKr and AdaLoRA, along with prefix tuning, prompt tuning and IA3.

The workflow is short. You wrap a Transformers model with get_peft_model, train it with your usual loop or Trainer, then save with save_pretrained, which writes only the adapter. Later you load it with PeftModel.from_pretrained, or call merge_and_unload to fold it into the base weights.

from peft import get_peft_model, LoraConfig
model = get_peft_model(base_model, LoraConfig(r=16, lora_alpha=32,
                                              target_modules="all-linear"))
model.print_trainable_parameters()

Take quiz
Which PEFT call wraps a base model with LoRA layers?
merge_and_unload
save_pretrained
get_peft_model
AutoTokenizer.from_pretrained
What does merge_and_unload return?
A smaller tokenizer
A quantized copy of the adapter only
A list of trainable parameter names
A plain model with adapter weights folded into the base

10. What is QLoRA?

QLoRA trains LoRA adapters on top of a base model that is frozen and quantized to 4 bits. It was introduced by Dettmers et al. in 2023 and made fine-tuning big models possible on a single GPU, for example a 65B model on one 48 GB card.

It combines three ideas:

  • NF4: a 4-bit data type designed for normally distributed weights.
  • Double quantization: quantizing the quantization constants to save more memory.
  • Paged optimizers: spilling optimizer states to CPU memory when GPU memory spikes.

The adapters themselves train in higher precision such as bf16, so the quality loss from quantizing the base is small.

Take quiz
In what format is the frozen base model stored during QLoRA?
4-bit NF4
8-bit integer only
fp32
Binary 1-bit
Which of these is a component of QLoRA?
Prefix tokens
Paged optimizers
Knowledge distillation
Beam search

11. What is LoRA dropout?

lora_dropout randomly zeroes parts of the input that flows into the adapter path, so the computation becomes B(A(dropout(x))). The frozen base path is unaffected, and dropout only runs in training mode.

It acts as a regularizer, mostly useful when the dataset is small and the adapter might memorize it. Values of 0.05 to 0.1 are common. Many recent recipes use 0 because the adapter already has few parameters and dropout can slow training.

Kohya-style trainers expose related options: network_dropout, rank_dropout and module_dropout.

Take quiz
Where is lora_dropout applied?
To the frozen base weights
To the tokenizer output
To the input of the adapter path only
To the final loss value
When is LoRA dropout mainly helpful?
With datasets of billions of tokens
Only at inference time
When training in 4-bit
With small datasets prone to memorization

The right trainer depends on whether you are tuning a language model or an image model.

Trainer Focus Notable trait
Hugging Face PEFT + TRL LLMs Library-level control, SFT and DPO trainers
Axolotl LLMs YAML configs, multi-GPU setups
Unsloth LLMs Optimized kernels, low memory on single GPUs
LLaMA-Factory LLMs CLI and web UI
Kohya_ss / sd-scripts Diffusion SD 1.5, SDXL and newer models, GUI available
OneTrainer Diffusion GUI with many model types
ai-toolkit Diffusion YAML configs, popular for Flux LoRAs
SimpleTuner Diffusion CLI, large-model friendly

Supported models change quickly, so check each project's README before starting.

13. What is Kohya_ss?

Kohya_ss is a GUI (maintained by bmaltais) around the kohya-ss/sd-scripts command-line scripts. It trains LoRA, DreamBooth, full fine-tunes and textual inversions for Stable Diffusion-family models.

The LoRA settings you touch most are network_dim (rank), network_alpha, learning_rate, unet_lr, text_encoder_lr, train_batch_size, max_train_epochs, resolution and the optimizer.

Datasets use folder names that start with the repeat count:

train/
  10_mychar/      # 10 repeats per epoch
    img01.png
    img01.txt   # caption

Each image usually has a matching .txt caption file, and the trainer writes the finished LoRA as a .safetensors file you can drop into ComfyUI or AUTOMATIC1111. Save a checkpoint every epoch so you can compare them later.

Take quiz
In the Kohya folder name 10_mychar, what does the 10 mean?
The LoRA rank
The batch size
Repeats of each image per epoch
The number of captions per image
Which Kohya parameter sets the LoRA rank?
network_alpha
unet_lr
max_train_epochs
network_dim

14. How do you prepare a dataset for LoRA training?

Good data beats clever hyperparameters. The steps depend on the domain, but the shape is the same:

  1. Collect relevant examples. Image concept LoRAs often work with roughly 15-50 varied images; LLM tasks usually need hundreds to thousands of clean examples.
  2. Clean by removing blurry, off-topic and near-duplicate items.
  3. Format consistently: image resolution and aspect buckets, or the model's chat template for LLMs.
  4. Label with captions or instruction-response pairs.
  5. Split off a small validation set if you want a real overfitting signal.

For chat LLMs, apply the same chat template at training and inference, and mask prompt tokens so loss only counts the response.

Take quiz
For a chat-style LLM LoRA, which preprocessing step is essential?
Using the model's chat template consistently
Converting all text to uppercase
Removing every system prompt
Shuffling characters inside words
Why remove near-duplicate images from a LoRA dataset?
They make captions longer
Duplicates get over-weighted and encourage memorization
They block latent caching
They change the base model's tokenizer

15. What is a trigger word in LoRA training?

A trigger word is a unique token or phrase placed in training captions so the LoRA ties the new concept to it. At inference you include that word in the prompt to activate the learned subject or style.

Pick a rare string, such as a made-up token, so it does not collide with a word the base model already understands. A trigger like castle would fight the existing meaning of the word.

Triggers are optional for style LoRAs, which are often meant to change everything. Models with natural-language text encoders such as Flux are also more forgiving, because descriptive captions carry most of the signal.

Take quiz
Why should a trigger word be a rare token?
Rare tokens train faster on CPU
Common words cannot be tokenized
A common word already has meaning in the base model
Rare tokens reduce the file size
Is a trigger word always required for a style LoRA?
Yes, training fails without it
Yes, but only for 4-bit models
No, because style LoRAs ignore captions completely
No, style LoRAs often work without one

16. What learning rate should you use for LoRA training?

LoRA trains few parameters from a clean start, so it tolerates and often needs a higher learning rate than full fine-tuning. Treat these as starting points, not rules:

  • LLM LoRA: about 1e-4 to 2e-4 with AdamW, plus warmup and a cosine or linear decay.
  • Full fine-tuning of the same model usually sits around 1e-5 or lower.
  • Diffusion LoRA: often around 1e-4 for the UNet or transformer, with a lower rate for the text encoder if it is trained at all.

Learning rate interacts with rank, alpha and batch size, so change one at a time. Too high gives burnt outputs or NaN loss; too low gives an adapter that seems to do nothing.

Take quiz
Why does LoRA usually use a higher learning rate than full fine-tuning?
Few parameters start from zero update and need larger steps to move
Frozen weights speed up the GPU clock
The loss function is different
Adam stops working at low rates
A typical starting LR for an LLM LoRA with AdamW is about:
1e-7
2e-4
1.0
5e-1

17. Define epoch, step, and batch size in LoRA training?

These three terms control how long and how coarsely training runs:

  • Batch size: samples processed per forward-backward pass.
  • Step: one optimizer update. With gradient accumulation, one step spans several batches.
  • Epoch: one full pass over the dataset, including any repeats.

Steps per epoch = (images × repeats) / (batch size × accumulation steps). Example: 30 images, 10 repeats and batch size 2 give 150 steps per epoch, so 10 epochs equal 1,500 steps.

Trainers like Kohya often let you cap training by epochs or by total steps, and checkpoints are usually saved per epoch.

Take quiz
20 images, 5 repeats, batch size 4, no accumulation. Steps per epoch?
100
5
25
20
When does one optimizer step happen with gradient accumulation of 4?
After every single sample
Once per epoch
Only at the final step of training
After four micro-batches have been processed

18. What does merging a LoRA into the base model mean?

Merging folds the adapter into the base weights using W' = W + (alpha / r) × B × A. The result is a normal checkpoint with no extra adapter layers, so inference runs at base-model speed.

The cost is flexibility. Once merged, you can no longer swap adapters or change the strength without keeping the original weights. PEFT can unmerge, but keeping a clean base copy is safer.

One caveat: merge into fp16 or bf16 weights, not into a 4-bit quantized base. Rounding the tiny delta into 4-bit values loses information, so QLoRA adapters are normally merged into a dequantized copy.

merged = model.merge_and_unload()
merged.save_pretrained("my-merged-model")

Take quiz
What formula describes a merged LoRA layer?
W' = W + (alpha / r) x B x A
W' = W x B x A
W' = A + B
W' = W / (alpha x r)
Where should a QLoRA adapter be merged for best fidelity?
Directly into the 4-bit weights
Into a higher-precision (fp16/bf16) copy of the base
Into the tokenizer
Into the optimizer state

19. Describe overfitting in LoRA training?

Overfitting means the adapter memorizes the training set instead of learning the general pattern. It is one of the most common ways a LoRA run goes wrong.

  • LLMs: training loss keeps falling while validation loss rises; outputs repeat training phrases.
  • Image models: samples copy training poses and backgrounds, ignore the prompt, or show burnt colors and artifacts.

Typical causes are too many epochs, a learning rate or rank that is too high, and a small, narrow dataset.

Fixes: stop earlier and pick an earlier checkpoint, lower the learning rate or rank, add dropout, diversify the data, and for image LoRAs add regularization images.

Take quiz
Which curve pattern signals overfitting in an LLM LoRA?
Both losses stay flat at zero
Validation loss falls faster than training loss forever
Training loss falls while validation loss rises
Loss alternates between NaN and zero
A character LoRA ignores your prompt and repeats training backgrounds. Most likely cause?
A missing tokenizer file
Too low a trigger word length
Using safetensors instead of pickle
Overfitting from too many steps or a narrow dataset

20. What is LoRA weight (strength) at inference time?

LoRA strength is a multiplier on the adapter's delta when it is applied. 1.0 is the trained strength, lower values blend the effect toward the base model, and values above 1.0 exaggerate it.

In AUTOMATIC1111 you write <lora:name:0.8> in the prompt. In ComfyUI it is a strength slider on the loader node. In diffusers you pass adapter weights:

pipe.load_lora_weights("path/to/lora", adapter_name="style")
pipe.set_adapters(["style"], adapter_weights=[0.8])

Backing off to 0.6-0.9 is a quick fix when an adapter is slightly overbaked, and it is how stacked LoRAs are balanced.

Take quiz
What does a LoRA strength of 0.5 do at inference?
Applies half of the adapter's delta
Doubles the adapter delta
Disables the base model
Halves the image resolution
In diffusers, which call sets per-adapter strengths?
pipe.merge_strength()
pipe.set_adapters(..., adapter_weights=[...])
pipe.enable_attention_slicing()
pipe.save_pretrained()

21. How does LoRA reduce the number of trainable parameters?

For a single weight matrix of size d × k, full fine-tuning trains d × k values. LoRA trains only r × (d + k), because it learns two thin matrices instead of one large update.

Layer (LLaMA-7B) Full update LoRA r=16
q_proj 4096 x 4096 16,777,216 131,072
gate_proj 4096 x 11008 45,088,768 241,664

Applied to every linear layer of a 7B model, r=16 adds about 40 million trainable parameters out of roughly 6.7 billion, or about 0.6%. The gap grows with larger layers because the savings scale with d × k / (d + k).

The savings are easiest to see by comparing the two shapes. A full update touches every cell of a 4096 x 4096 matrix, while LoRA only trains two strips that are 4096 wide and 16 thick. That is also why the adapter file is tiny: the saved tensors are those strips, not the matrix.

Take quiz
How many trainable values does LoRA need for one d x k matrix at rank r?
d x k x r
r x d x k
r x (d + k)
(d + k) / r
With r = 16 on a 4096 x 11008 layer, the LoRA parameter count is:
131,072
45,088,768
16,777,216
241,664

22. How do you choose the right LoRA rank?

Start modest and let results push you up. For most LLM tasks, r = 16 on all linear layers is a sound first run. Simple style or format changes often do fine with 8 or lower.

Raise the rank when training loss plateaus high, the task needs new skills, or the dataset is large. Lower it when you see overfitting on a small dataset or when file size matters.

The QLoRA paper found that covering all linear layers mattered far more than rank, so widen the target modules before you push rank very high. Keep alpha tied to rank (for example alpha = r or 2r), and compare two or three ranks, such as 8, 16 and 64, on a validation set before deciding.

Take quiz
Which is a sensible first LoRA run for a typical LLM task?
r = 16 on all linear layers
r = 4096 on the embedding layer only
r = 1 on one attention head
r = 0 with a high alpha
According to the QLoRA paper, what often matters more than a very high rank?
Using a bigger tokenizer
Applying LoRA to all linear layers
Training for fewer than ten steps
Disabling the optimizer

23. What is the difference between LoRA and full fine-tuning?

Aspect Full fine-tuning LoRA
Trainable parameters All weights Small adapter matrices (often under 1%)
GPU memory Gradients and optimizer states for the whole model Only for adapter weights, plus activations
Checkpoint size Full model copy Megabytes per task
Forgetting Higher risk Usually lower
Quality ceiling Highest, especially for large datasets Slightly lower on knowledge-heavy tasks
Deployment One model per task One base plus swappable adapters

Pick full fine-tuning when you have a large dataset, need new knowledge, and can afford the compute. Pick LoRA for most task-style adaptation, fast iteration and multi-tenant setups.

A practical rule: if you can describe the change as a behavior (tone, format, a tool-calling pattern, a visual style), LoRA is usually enough. If you are teaching the model a large body of new facts or a new language, expect to lean toward full fine-tuning or continued pretraining.

Take quiz
Which is a typical advantage of LoRA over full fine-tuning?
Always higher accuracy
No need for a base model at all
Much smaller per-task checkpoints
Training with no GPU memory
When is full fine-tuning the stronger choice?
Tiny style tweaks on 20 examples
Serving hundreds of tenants on one GPU
Quick hyperparameter experiments on a laptop
Large datasets that teach substantial new knowledge

24. What is the difference between LoRA and QLoRA?

Aspect LoRA QLoRA
Base model precision fp16 or bf16 4-bit NF4
Memory Lower than full fine-tuning Lower still, big models fit on one GPU
Speed Faster Slower due to on-the-fly dequantization
Adapter Trained in 16-bit Trained in 16-bit, same format
Quality Baseline Close to LoRA when configured well

The adapter itself is identical in both cases. The difference is how the frozen base is stored. Use QLoRA when VRAM is the constraint, and plain LoRA when you have the memory and want faster training.

Memory example: a 7B model in bf16 needs about 14 GB just for weights, while the 4-bit NF4 version needs roughly 4 GB. That gap is what lets QLoRA fit on consumer cards, at the price of slower steps and a small accuracy hit from quantization.

Take quiz
What is the main difference between LoRA and QLoRA?
QLoRA stores the frozen base model in 4-bit
QLoRA trains the base model instead of an adapter
QLoRA skips backpropagation
QLoRA uses a different tokenizer
Why is QLoRA usually slower than LoRA at the same size?
It uses fewer GPUs
The 4-bit weights must be dequantized during each forward pass
It trains more parameters
It disables gradient computation

25. Explain the internal working of LoRA's forward and backward pass?

In the forward pass, a LoRA layer computes two paths from the same input x and adds them:

h = W x  +  (alpha / r) * B( A( dropout(x) ) )

flowchart LR
  x["Input x"] --> W["Frozen W"]
  x --> A["A: r x k"]
  A --> B["B: d x r"]
  B --> S["Scale alpha/r"]
  W --> Add((+))
  S --> Add
  Add --> h["Output h"]

In the backward pass, W has requires_grad=False, so no weight gradient or optimizer state is created for it. Gradients still flow through W as activation gradients, because earlier layers' adapters need them. Only A and B receive weight gradients and get updated.

This is why activation memory stays high even though trainable parameters are tiny, and why gradient checkpointing remains useful.

Take quiz
In LoRA's backward pass, which weights receive gradients?
Every weight in the base model
Only the tokenizer embeddings
Only the A and B adapter matrices
Only the bias terms of layer norm
Why do gradients still flow through the frozen W?
W is secretly trained in fp32
The optimizer requires it for logging
To keep the dropout mask consistent
Earlier layers' adapters need the activation gradients

26. Why is matrix B initialized to zero in LoRA?

Initializing B to zero makes ΔW = B × A exactly zero at step 0. The adapted model starts identical to the pretrained one, so training begins from a known-good point instead of from a random perturbation.

A is initialized randomly (Kaiming-uniform style in PEFT), which avoids a symmetry problem. If both matrices were zero, neither would get a useful gradient.

At step 0 the gradient for A is zero because it depends on B, but B gets a real gradient immediately. After the first update B is non-zero, and then A starts learning as well.

In short, the zero start is a safety choice. Your first training steps cannot damage the pretrained behavior, and every change the adapter makes afterward is something the data asked for.

Take quiz
What is the main benefit of initializing B to zero?
The adapted model starts identical to the base model
It halves the adapter size
It removes the need for a learning rate
It forces rank 1
Why is A not initialized to zero as well?
A is not trainable
Both zero would leave neither matrix with useful gradients
Zero A breaks safetensors saving
A must always equal B

27. Why does rsLoRA scale by alpha divided by the square root of r?

Standard LoRA scales the update by alpha / r. As rank grows, that factor shrinks the update so much that higher ranks learn slowly or no better than low ones. The extra capacity is wasted.

rsLoRA (rank-stabilized LoRA, Kalajdzievski 2023) changes the factor to alpha / sqrt(r). Its analysis shows this keeps learning stable across ranks, so larger ranks can actually pay off.

In PEFT it is a flag:

LoraConfig(r=128, lora_alpha=16, use_rslora=True)

Use it if you plan to train with high ranks. At small ranks like 8 or 16, the difference is minor.

Example: at r = 256 with alpha = 16, the standard factor is 16/256 = 0.0625, while rsLoRA gives 16/16 = 1. That is the difference between an update that barely moves the model and one that trains at a healthy pace.

Take quiz
What scaling factor does rsLoRA use?
alpha x r
alpha / r squared
alpha / sqrt(r)
1 / (alpha + r)
What problem does rsLoRA address?
Slow tokenization
Out-of-memory errors in the optimizer
Adapter files that do not merge
Learning stalls as rank grows under alpha / r scaling

28. What is the difference between LoRA, LoHa, and LoKr?

LoHa and LoKr come from the LyCORIS family and are popular in image-model training. They replace the plain B × A product with a different factorization.

Method How the update is built Practical effect
LoRA Product of two low-rank matrices Simple, widely supported
LoHa Hadamard (element-wise) product of two low-rank products Effective rank up to r squared, more capacity per parameter
LoKr Kronecker product of small matrices Very compact, good for large layers

LoCon is a separate extension that also adapts convolution layers. Trade-off: LyCORIS formats need tool support at inference, while plain LoRA loads almost everywhere.

If you are unsure, start with plain LoRA. Move to LoHa or LoKr when you need more capacity at the same file size, such as a multi-concept or style-heavy image adapter, and confirm your inference tool can load the format first.

Take quiz
Which method builds its update with a Hadamard product?
LoHa
LoKr
Plain LoRA
QLoRA
What is a trade-off of LyCORIS methods compared with plain LoRA?
They cannot be trained on a GPU
They need explicit support in the inference tool
They only work for text models
They always produce larger files

29. How does DoRA differ from LoRA?

DoRA (Weight-Decomposed Low-Rank Adaptation, Liu et al. 2024) splits each pretrained weight into a magnitude and a direction. It applies the low-rank update to the direction and trains the magnitude as a separate vector.

The motivation is that full fine-tuning tends to change magnitude and direction in a more decoupled way than plain LoRA does. DoRA mimics that, and the paper reports gains especially at low ranks.

Costs: extra compute and memory per step, since the weight norm is recomputed. It still merges into W for inference, so there is no serving penalty.

LoraConfig(r=16, lora_alpha=32, use_dora=True)

When to try it: if plain LoRA at your affordable rank underfits, DoRA is a cheap next experiment before raising rank. If training time or memory is already tight, the extra overhead may not be worth the small gain.

Take quiz
What does DoRA decompose each weight into?
Mean and variance
Rows and columns
Magnitude and direction
Weights and tokenizer
What is a cost of DoRA compared with plain LoRA?
It cannot be merged for inference
It needs a bigger vocabulary
It removes the need for a base model
Extra compute and memory during training

30. Why do we apply LoRA to all linear layers instead of only attention?

The MLP blocks hold most of a transformer's parameters. In a LLaMA-7B layer, the three MLP matrices total about 135 million parameters, versus roughly 67 million for the four attention projections. Adapting only attention leaves about two thirds of the layer untouched.

The original LoRA paper used query and value projections to keep the parameter budget small. The QLoRA paper later found that applying adapters to all linear layers was needed to match full fine-tuning quality, and that rank mattered less.

The cost is more adapter parameters and a little more memory. In practice this is cheap, and target_modules="all-linear" is a common default for LLMs today.

Take quiz
Why does adapting only attention leave capacity unused?
MLP layers contain most of the layer's parameters
MLP layers cannot hold adapters
Attention has no weights
MLP layers are always frozen by PEFT
What shortcut selects every linear layer in PEFT?
target_modules="attn-only"
target_modules="all-linear"
use_all_layers=True
r="max"

31. Explain the internal working of QLoRA?

QLoRA keeps the base model in 4-bit and dequantizes each layer's weights on the fly for the matmul. The LoRA path runs in bf16 beside it, and only adapter weights get gradients.

flowchart LR
  Wq["4-bit NF4 weights"] --> D["Dequantize to bf16"]
  D --> M["Matmul with x"]
  X["Input x"] --> M
  X --> L["LoRA path in bf16"]
  M --> S((+))
  L --> S
  S --> Loss["Loss and backward"]
  Loss --> U["Update adapters only"]
  • NF4: quantization levels placed at quantiles of a normal distribution, applied block-wise with an absmax constant per block.
  • Double quantization: the per-block constants are themselves quantized, saving roughly 0.37 bits per parameter.
  • Paged optimizers: optimizer states use unified memory and spill to CPU RAM when GPU memory spikes.
from transformers import BitsAndBytesConfig
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_use_double_quant=True,
                         bnb_4bit_compute_dtype=torch.bfloat16)

The compute dtype matters too. Weights are stored in 4-bit, but matmuls run in bf16 (or fp16), so the adapter and activations keep enough precision for stable training. Setting the compute dtype to match your training precision avoids a silent slowdown from dtype casts.

Take quiz
What does NF4 stand for in QLoRA?
Near-Fixed 4-byte
Node Format 4
NormalFloat 4-bit
Normalized Fusion 4
What does double quantization quantize?
The LoRA rank values
The training labels
The learning-rate schedule
The per-block quantization constants

32. How do you detect overfitting in a LoRA run?

Detect it with signals that match the model type, because loss alone can mislead.

  • LLMs: hold out a validation set, log eval loss every N steps, and watch for it turning upward while training loss keeps falling. Enable early stopping or keep the best checkpoint.
  • Diffusion: loss is noisy and weakly informative. Generate a fixed-seed prompt grid at every checkpoint and compare them.
  • Out-of-distribution prompts: a healthy LoRA still follows prompts that were not in training. If it ignores them, it has memorized.

Also compare checkpoints side by side. The best one is often several epochs before the last.

TrainingArguments(eval_strategy="steps", eval_steps=100,
                  load_best_model_at_end=True, metric_for_best_model="eval_loss")

A simple habit is to save a checkpoint at every epoch and rank them with the same prompts afterward. That turns overfitting from a vague worry into a pick-the-best-checkpoint decision.

Take quiz
Why is loss a weak overfitting signal for diffusion LoRAs?
Diffusion loss is noisy and tells little about sample quality
Diffusion models have no loss
Loss is always exactly zero
It only exists after merging
Which setting keeps the best checkpoint for an LLM LoRA in Trainer?
save_total_limit=0
load_best_model_at_end=True
gradient_checkpointing=True
report_to="none"

33. How do you troubleshoot CUDA out of memory during LoRA fine-tuning of an LLM?

Work from the cheapest change to the most invasive one:

  1. Lower per_device_train_batch_size and raise gradient_accumulation_steps to keep the effective batch.
  2. Shorten max_seq_length. Activation memory grows with sequence length.
  3. Turn on gradient checkpointing.
  4. Switch to bf16 and an 8-bit or paged optimizer.
  5. Use FlashAttention or SDPA if the model supports it.
  6. Load the base in 4-bit (QLoRA).
  7. Reduce rank or the number of target modules as a last resort.

Check nvidia-smi for other processes first, and make sure evaluation batch size is not larger than training.

Example: a run that fails at batch 8 and length 2048 often succeeds at batch 2, accumulation 4 and length 1024 with checkpointing on. If answers in your data are short, packing or length-grouped batching also cuts wasted padding memory.

Take quiz
Which change reduces memory without changing the effective batch size?
Doubling max_seq_length
Switching to fp32
Smaller micro-batch plus higher gradient accumulation
Disabling bf16
Why shorten max_seq_length when you hit OOM?
It shrinks the base weights
It changes the LoRA rank
It removes the optimizer
Activation memory grows with sequence length

34. Why does gradient checkpointing help LoRA training?

With the base frozen, optimizer states and weight gradients are small, so activations become the dominant memory cost. Gradient checkpointing stores only some activations in the forward pass and recomputes the rest during backward.

The trade-off is compute: expect roughly 20-30% slower steps in exchange for large memory savings, which lets you use longer sequences or bigger batches.

With PEFT, a common gotcha is that checkpointed layers need inputs that require grad. Call enable_input_require_grads() or use prepare_model_for_kbit_training for QLoRA, otherwise you may get a warning that no gradients flow.

model.gradient_checkpointing_enable()
model.enable_input_require_grads()

Rule of thumb: turn it on when memory is the problem, leave it off when you have headroom, because the recomputation is pure time cost. With QLoRA the combination of 4-bit weights and checkpointing is what makes long-sequence training fit on one GPU.

Take quiz
Why does gradient checkpointing matter even though LoRA has few trainable parameters?
Activation memory still dominates
Adapters need fp64 precision
The tokenizer uses most memory
The optimizer stores the full base weights
What is the cost of gradient checkpointing?
It loses the adapter weights
Extra recomputation makes each step slower
It forces a smaller rank
It disables gradient accumulation

35. What is the difference between LoRA, adapters, and prompt tuning?

Method What is trained Inference cost Mergeable?
LoRA Low-rank matrices parallel to existing weights None once merged, small overhead if not Yes
Bottleneck adapters Small layers inserted in sequence inside blocks Added latency from extra layers No
Prompt / prefix tuning Learned virtual tokens prepended to the input Uses part of the context window No

LoRA wins on deployment simplicity because it can disappear into the weights. Prompt tuning is the lightest to store but tends to lag on smaller models and harder tasks. Bottleneck adapters are older and still useful, but their sequential layers always cost latency.

Memory and cost view: all three keep the base frozen, so they are cheap to train. The deciding factors are latency at serving time and how much task difficulty the method can absorb. LoRA offers the best balance for most teams, which is why tooling around it is the most mature.

Take quiz
Which method can be merged into the base weights with no inference overhead?
Prefix tuning
Bottleneck adapters
LoRA
Prompt tuning
What does prompt tuning consume at inference time?
Extra attention heads
A second copy of the model
Additional MLP layers
Part of the context window for virtual tokens

36. How do you choose the learning rate scheduler and warmup for LoRA?

For LLM LoRAs, cosine decay with 3-10% warmup is a safe default. Warmup protects the first steps, when B is zero and Adam's statistics are still unreliable. Cosine then lets the rate fall smoothly so the last steps polish instead of overshoot.

Linear decay behaves similarly. Short runs of a few hundred steps often do well with plain linear decay or even a constant rate with warmup.

Diffusion trainers commonly use constant, constant_with_warmup or cosine_with_restarts. Constant rates make it easier to compare checkpoints from different epochs, which is how many people pick the best one.

If loss spikes early, lengthen warmup or lower the peak rate before touching anything else.

A concrete example: 1,000 steps with a peak LR of 2e-4, 50 warmup steps and cosine decay to near zero is a reasonable baseline. If results look undertrained, raise steps before raising the peak rate, since a hot peak is the usual cause of burnt outputs.

Take quiz
Why use warmup at the start of a LoRA run?
Early steps are unstable while Adam statistics settle
It increases adapter rank
It freezes the base model
It reduces the dataset size
Why do many diffusion LoRA users prefer a constant scheduler?
It disables the optimizer
It makes checkpoints from different epochs easier to compare
It guarantees lower loss
It removes the need for captions

37. Explain the execution flow of a LoRA training run?

flowchart TD
  A["Load base model"] --> B["Optional 4-bit quantization"]
  B --> C["Inject LoRA adapters"]
  C --> D["Freeze base weights"]
  D --> E["Load and tokenize dataset"]
  E --> F["Forward pass"]
  F --> G["Compute loss"]
  G --> H["Backward: adapter gradients only"]
  H --> I["Optimizer step"]
  I --> J{Eval or save?}
  J -->|yes| K["Validate and checkpoint adapter"]
  J -->|no| F
  K --> F
  K --> L["Export adapter or merge"]
  1. Load the base model, quantizing it first for QLoRA.
  2. Inject adapters into the target modules and freeze everything else.
  3. Prepare the dataset: tokenize, apply templates, or cache image latents.
  4. Loop: forward, loss, backward for adapter weights, accumulate, optimizer step, scheduler step.
  5. Periodically evaluate and save adapter checkpoints.
  6. Export the adapter, or merge it into the base for deployment.

Only the adapter is written at each checkpoint, so saves are fast and cheap.

Take quiz
At which point in the flow are base weights frozen?
After the final checkpoint
Only during evaluation
Right after adapters are injected, before training starts
After merging
What gets written at each LoRA checkpoint?
The full base model
The raw training dataset
The GPU memory snapshot
Only the adapter weights and config

38. How does inference differ between merged and unmerged LoRA?

Aspect Unmerged Merged
Computation Wx plus a small B(A(x)) path per adapted layer Single matmul with W'
Latency Small overhead per adapted layer Same as base model
Swap adapters at runtime Yes No
Change strength Yes No, baked in
Multi-tenant serving Good fit Needs one model copy per adapter

Merge when you ship one fixed behavior and want maximum speed. Stay unmerged when many adapters share one base or you need to tune strength per request. With quantized bases, unmerged is also the safer option because it avoids rounding the delta into 4-bit weights.

Example: a company with 50 customer-specific adapters on one base model keeps them unmerged and attaches the right one per request. A team shipping a single product behavior merges once and serves a plain checkpoint everywhere.

Take quiz
Which setup lets you change LoRA strength per request?
Unmerged adapter
Merged model
Quantized base without adapters
Fully fine-tuned model
What is the latency of a merged LoRA compared with the base model?
Roughly double
Essentially the same
Ten times slower
Depends on the dataset size

39. What happens when you merge multiple LoRAs?

For linear merging, each adapter's delta is scaled and summed: W' = W + w1 × ΔW1 + w2 × ΔW2. All adapters must target the same base model.

The catch is interference. If two adapters push the same layers in conflicting directions, the result can lose both effects or produce artifacts. This is why stacked image LoRAs often need lower strengths, such as 0.5 each, and visual testing.

PEFT offers several combination methods in add_weighted_adapter, including linear, concatenation, SVD, TIES and DARE variants. TIES and DARE try to reduce conflicts by pruning small or opposing changes.

model.add_weighted_adapter(["a", "b"], weights=[0.6, 0.4],
                           adapter_name="mix", combination_type="ties",
                           density=0.5)

Test merges the same way you test a new adapter: fixed prompts, several weight combinations, and a check that each source adapter's behavior is still visible. Merging is cheap, so try a few ratios instead of trusting one.

Take quiz
What is the main risk of combining two LoRAs?
The base model is deleted
Rank is forced to 1
Their changes can interfere in shared layers
The tokenizer changes silently
Which methods reduce conflicts when combining adapters in PEFT?
Dropout and warmup
Gradient checkpointing and ZeRO
Beam search and top-k
TIES and DARE

40. How do you serve many LoRA adapters efficiently?

Keep one copy of the base model in GPU memory and attach adapters per request instead of deploying a full model for each tenant.

  • S-LoRA: research system that pages adapters between CPU and GPU memory.
  • vLLM: multi-LoRA serving with custom batched kernels.
  • LoRAX (Predibase): dynamically loads adapters on demand and batches across them.

The key trick is batching requests that use different adapters in one forward pass, using kernels such as Punica's grouped matmuls. Hot adapters stay in GPU memory; cold ones are loaded from disk or CPU with an LRU policy.

vllm serve base-model --enable-lora --max-loras 4 --max-lora-rank 64 \
  --lora-modules sql=/adapters/sql support=/adapters/support

Limits to plan for: maximum rank, number of resident adapters, and load latency on cache misses.

Monitoring matters here too. Track per-adapter latency, cache hit rate and the time to load an adapter from cold, because the slowest path decides your worst-case response time.

Take quiz
What do multi-LoRA servers share across requests?
One base model in GPU memory
One adapter for all tenants
One tokenizer per request
One optimizer state
Which vLLM flag turns on LoRA serving?
--merge-all
--enable-lora
--quantize-adapter
--tenant-mode

41. How does caption quality affect character and style LoRA training?

Captions tell the model what is explained by the text and what must be learned by the LoRA. Whatever you do not caption tends to get absorbed into the trigger word.

  • Character LoRA: describe clothing, pose, background and lighting, but not the character's fixed identity features. Then the trigger carries the face and body, and clothes stay promptable.
  • Style LoRA: describe the content (a cat on a sofa), not the style words. The style is what remains unexplained, so the adapter learns it.
  • Over-captioning the target feature stops the LoRA from learning it; under-captioning glues backgrounds and objects to the concept.

Tag-style captions suit SD 1.5 and SDXL anime models. Flux and SD3-class models, with T5 text encoders, respond better to short natural-language sentences.

Take quiz
In a style LoRA, what should captions avoid describing?
Objects in the image
The scene content
The style itself
Subject poses
What happens if you leave a character's outfit uncaptioned in every image?
The outfit is removed from the dataset
The text encoder is retrained
Rank is automatically reduced
The outfit gets glued to the trigger word

42. What is the difference between text encoder and UNet training in a diffusion LoRA?

The UNet (or transformer in newer models) does the denoising and holds most of the visual knowledge. The text encoder converts the prompt into embeddings.

Training the UNet LoRA teaches how to draw the concept. Training the text encoder LoRA helps the model map new words, such as a trigger token, to that concept. It can improve prompt response for new subjects, but it overfits easily, so trainers use a lower learning rate for it or skip it. Kohya exposes --network_train_unet_only for the latter.

For Flux and SD3-style models, people most commonly train only the transformer. The large T5 encoder is typically left frozen, which also allows caching its outputs to save VRAM.

A practical default: start with the UNet or transformer only. Add the text encoder when the model refuses to bind a new trigger word or ignores a new concept, and keep its learning rate well below the main one.

Take quiz
Which component does the actual denoising in an SD-style model?
The UNet or transformer
The text encoder
The VAE decoder only
The scheduler
Why is the text encoder often given a lower learning rate or left untrained?
It has no weights
It overfits easily
It does not accept captions
It must run in 1-bit mode

43. How can you optimize LoRA training for low VRAM on diffusion models?

Combine several techniques, ordered by how little they hurt quality:

  1. Cache latents so the VAE does not run every step, and cache text-encoder outputs if the encoder is frozen.
  2. Gradient checkpointing for large activation savings.
  3. bf16 mixed precision and an 8-bit optimizer such as AdamW8bit.
  4. Batch size 1 with gradient accumulation.
  5. Lower resolution or fewer aspect buckets.
  6. Quantize the base to fp8 or lower for large models, or use block swapping where the trainer supports it.
  7. Reduce rank only after the options above.

Flux-class models on 24 GB cards usually need most of these together. Check the specific trainer's low-VRAM config for tested presets.

Example: a Flux-class LoRA that will not fit on a 24 GB card at 1024 px can often run with cached latents, fp8 base weights, gradient checkpointing, batch size 1 and AdamW8bit. Add one change at a time so you know which one made it fit.

Take quiz
Why cache latents before diffusion LoRA training?
It increases batch size automatically
It raises the learning rate
The VAE does not have to run on every step
It removes captions
Which pairing keeps the effective batch size while saving memory?
Batch size 1 with accumulation of 1
Higher resolution with larger batch
fp32 with larger rank
Batch size 1 with gradient accumulation

44. Which is better and why: bf16 or fp16 for LoRA training?

bf16 is usually better on GPUs that support it (Ampere or newer). It has the same 8-bit exponent range as fp32, so values rarely overflow or underflow, and you do not need loss scaling.

Format Exponent / mantissa bits Practical effect
fp16 5 / 10 More precision, small range, needs loss scaling, prone to NaN
bf16 8 / 7 fp32-like range, less precision, stable without loss scaling

Use fp16 only on older GPUs without bf16 support, and enable gradient scaling. PEFT keeps adapter weights in fp32 by default when the base is half precision, which helps stability. If you still see NaN loss, check the learning rate and data before blaming the precision.

Example: if a fp16 run produces NaN around step 200 but the same config in bf16 trains cleanly, the exponent range was the issue. Keep bf16 as your default on any recent GPU and save fp16 for hardware that forces it.

Take quiz
Why is bf16 usually more stable than fp16?
It has the same exponent range as fp32
It uses 64-bit mantissa
It stores weights as integers
It needs no GPU
When is fp16 still the practical choice?
On every modern GPU
On older GPUs without bf16 support
When training with rank above 128
When the dataset is under 10 files

45. How do you evaluate a trained LoRA?

Evaluation should test both what the LoRA learned and what it might have broken.

  • LLMs: validation loss or perplexity on held-out data, task metrics (exact match, F1, pass@k), and an LLM-as-judge or human review for open-ended outputs.
  • Regression checks: run a general benchmark or prompt set before and after to catch forgetting.
  • Diffusion: generate a fixed-seed prompt grid across checkpoints and strengths, covering in-distribution prompts, out-of-distribution prompts and mixed styles.
  • Flexibility test: does it still follow pose, clothing and background prompts, or does it copy the training images?

Compare against the base model and against earlier checkpoints, not just against the final one. Keep the evaluation prompts fixed so differences come from the adapter and not from your choice of test.

Take quiz
Why run a general benchmark before and after LLM LoRA training?
To measure disk speed
To reduce rank
To detect forgetting of base capabilities
To compute the tokenizer size
What makes a diffusion LoRA evaluation grid comparable across checkpoints?
Random seeds each time
Different resolutions per image
A new base model per checkpoint
Fixed seeds and a fixed prompt set

46. How does batch size affect LoRA training?

Batch size sets how many samples contribute to each gradient. A larger batch gives a smoother gradient and allows a higher learning rate. A very small batch gives noisier updates, which can destabilize training or push the adapter to chase individual samples.

Memory is the constraint. When VRAM limits you to a batch of 1 or 2, use gradient accumulation to reach an effective batch of 8 or 16 without more memory. The cost is more time per optimizer step, not more memory.

When you change effective batch size, revisit the learning rate and the step count. The same number of epochs gives fewer optimizer steps with a larger batch, so an adapter that was well trained at batch 1 may look undertrained at batch 8 with the same settings.

Take quiz
How can you reach an effective batch of 16 on a GPU that fits only 2 samples?
Gradient accumulation over 8 micro-batches
Doubling the rank
Switching to fp64
Disabling the optimizer
What happens to steps per epoch when you raise effective batch size?
There are more optimizer steps
There are fewer optimizer steps per epoch
Steps stay constant
Epochs stop existing

47. How do you troubleshoot a LoRA with no visible effect or burnt output?

These are opposite symptoms with opposite fixes.

Symptom Likely causes What to try
No effect Wrong base model, adapter not actually loaded, mismatched target modules, trigger missing, LR too low, too few steps, strength too low Verify the loaded adapter keys, use the same base model, raise LR or steps, test at strength 1.0 with the trigger word
Burnt or overcooked LR too high, too many epochs, too much repetition, strength above 1 Lower LR, stop earlier, pick an earlier checkpoint, reduce strength, diversify data

Always test checkpoints from several epochs on a fixed prompt grid. Most of the time the answer is sitting in an earlier file you already saved.

Sanity trick: render the same prompt and seed with the adapter disabled and enabled. If the two images are identical, the adapter is not loaded or not reaching your layers, and no amount of hyperparameter tuning will help until that is fixed.

Take quiz
A LoRA shows no effect at all. Which check comes first?
Increase rank to 4096
Delete the validation set
That the adapter is loaded on the same base model it was trained on
Switch the scheduler to cosine
Which fix suits a burnt or overcooked LoRA?
Raise strength to 2.0
Add more repeats of the same images
Double the training epochs
Lower the learning rate or choose an earlier checkpoint

48. How is LoRA used with DPO or other preference training?

LoRA works well for preference optimization because it lowers memory. In DPO, the loss compares the policy's log-probabilities for chosen and rejected responses against a reference model. With LoRA you do not need a second model copy for that reference.

TRL's DPO trainer, when given a PEFT model and no explicit reference, computes reference log-probabilities by temporarily disabling the adapter, so the reference is the original base (or the SFT-merged model you started from). That saves roughly half the memory of a full-model setup.

A common pipeline is SFT with LoRA, merge, then train a fresh LoRA for DPO on the merged model. Use a much lower learning rate than SFT, since preference training is sensitive and can degrade fluency if pushed hard.

trainer = DPOTrainer(model=model, ref_model=None, args=cfg,
                     train_dataset=ds, peft_config=lora_cfg)

Take quiz
How does DPO with LoRA avoid keeping a separate reference model?
It disables the adapter to get reference outputs
It trains the reference separately on CPU
It duplicates the adapter twice
It skips the reference term entirely
How does the learning rate for DPO typically compare to SFT LoRA?
Ten times higher
Much lower
Identical in every case
Always zero

49. Why doesn't LoRA always match full fine-tuning quality?

LoRA constrains the update to a low-rank subspace. That is usually enough for style, format and task adaptation, but not always for tasks that need to absorb a lot of new information.

  • Capacity: when the dataset is large relative to the adapter's parameter count, LoRA can run out of room before full fine-tuning does.
  • Learns less, forgets less: studies comparing the two (for example Biderman et al., 2024) found LoRA underperformed on harder domains such as code and math continued training but preserved base abilities better.
  • Sensitivity: results depend on learning rate, rank and target layers more than people expect.
  • Different solutions: analyses of weight spectra show LoRA can introduce unusual new singular directions that full fine-tuning does not.

Practical advice: use all linear layers, tune the learning rate, and raise rank or switch to full fine-tuning if validation loss plateaus well above where full tuning gets.

Take quiz
When does LoRA most likely fall short of full fine-tuning?
When changing output format on small data
When swapping tone of voice
When the dataset is large and teaches substantial new knowledge
When adding a trigger word
What did comparisons of LoRA and full fine-tuning find about forgetting?
LoRA forgets everything
LoRA never changes the model
Full fine-tuning never forgets
LoRA tends to forget less of the base capabilities

50. How do you troubleshoot NaN loss in LoRA training?

NaN loss means a number overflowed or an invalid operation happened. Check the common culprits in this order:

  1. Learning rate too high. Cut it by half or more and lengthen warmup.
  2. fp16 overflow. Switch to bf16, or enable gradient scaling and clipping.
  3. Bad data. Empty samples or fully masked labels can produce a division by zero in the mean loss. Filter them out.
  4. Missing gradient clipping. Set max_grad_norm to 1.0.
  5. Unstable optimizer settings. 8-bit optimizers and very large batch changes can trigger it. Try standard AdamW to isolate the cause.
  6. Corrupt inputs. Check for NaN values in cached latents or embeddings.

Run a few hundred steps on a tiny subset with logging every step. If the NaN appears at a specific step, inspect the batch that arrived there.

Once training runs cleanly, keep gradient clipping on permanently. It costs nothing and prevents a single bad batch from poisoning the adapter weights.

Take quiz
Which data issue can produce NaN loss?
Samples whose labels are fully masked or empty
Having too many epochs
Using a long system prompt
Having captions in English
Which setting helps contain exploding gradients?
lora_alpha = 0
max_grad_norm = 1.0
warmup_steps = 0 with a higher LR
Disabling the validation set
«
»

Comments & Discussions