Practiced one out loud? Paste your answer into the free self-check and get a score in 30 seconds
What the interviewer is testing
Whether you know the mechanism (a frozen base plus a learned low-rank update), why it is cheap, what QLoRA adds, and the practical choices: rank, alpha, target modules and how to spot overfitting.
Answer framework
- LoRA: freeze the base weights and learn a low-rank update W + BA for selected matrices, with small rank r (often 8 to 64).
- Why it is cheap: trainable parameters drop by orders of magnitude, adapters are small files, and several adapters can share one base model.
- QLoRA: keep the frozen base in 4-bit and train LoRA adapters on top, so large models can be fine-tuned on a single GPU.
- Practical choices: learning rate, epochs, rank r and alpha, target modules, batch size and sequence length.
- Watch for overfitting: falling training loss with flat or rising validation loss; stop early and keep the best checkpoint.
Sample answer
LoRA freezes the base weights and learns a low-rank update for selected matrices: W + BA, where A and B have a small rank r, often 8 to 64. Trainable parameters drop by orders of magnitude, adapters are small files, and several adapters can share one base model.
QLoRA keeps the frozen base in 4-bit and trains LoRA adapters on top, which makes fine-tuning of large models feasible on a single GPU.
The hyperparameters that matter most are learning rate, number of epochs, rank r and alpha, which modules get adapters, batch size and sequence length. I apply LoRA to the attention projections at minimum; adding the MLP layers usually helps quality at more cost. If training loss falls while validation loss is flat or rising, I stop early and keep the best checkpoint.
Likely follow-ups
Common mistakes
- Saying LoRA updates all the weights, or that it changes the base model file.
- Confusing QLoRA (a 4-bit frozen base during training) with quantizing a model for serving.
- Not knowing which modules to target or what rank and alpha control.
- Ignoring validation loss and training until the adapter overfits.
- Treating fine-tuning as the way to add frequently changing facts; retrieval is better for that.
The 0 to 1 AI Engineer Interview Playbook covers the answer frameworks, system-design walkthroughs and an 8-week plan.
Get the playbook 40% off with code EARLYBF until Nov 3, 11:59 PM PTEARLYBF: 40% off until Nov 3Smaller start: Score your answers against 50 rubrics, $9. 50 AI engineer questions, one rubric each, PDF on Gumroad.
Interviewing for PM or TPM? See the full PM package, $59 on Gumroad: a resume system, a PM, TPM and PGM interview handbook, a 1:1 cheat sheet and a PM playbook. PM and TPM content, not AI engineer prep.