Practiced one out loud? Paste your answer into the free self-check and get a score in 30 seconds
What the interviewer is testing
Whether you understand scaling laws as a planning tool rather than trivia: how they relate model size, data and compute, and how inference cost shifts the choice away from the compute-optimal point.
Answer framework
- State the finding: loss falls smoothly, roughly as a power law, as you add parameters, training tokens and compute.
- Give the compute-optimal result: for a fixed training budget, grow parameters and tokens together, roughly 20 tokens per parameter (the 2022 Chinchilla paper).
- Say what it implied: many earlier large models were undertrained for their size.
- Bring in inference: a model served billions of times is cheaper to run if it is smaller, so teams train smaller models on far more tokens than compute-optimal.
- Mention the limit: lower loss does not map one-to-one to every downstream task, so you still evaluate on the tasks you care about.
Sample answer
Scaling laws say that pre-training loss decreases predictably, close to a power law, as you scale parameters, data and compute. That predictability is what makes them useful: you can run small experiments, fit the curve and estimate what a larger run will reach before spending the budget.
The compute-optimal result from the 2022 Chinchilla work is that for a fixed training budget you should scale parameters and training tokens together, at roughly 20 tokens per parameter. So a 10-billion-parameter model would want around 200 billion tokens. Several earlier models had many parameters but too few tokens.
In practice the decision is not only about training cost. If a model will serve a huge number of requests, every parameter costs money on every request, so it often pays to train a smaller model well past the compute-optimal token count. You spend more on training once and less on inference forever.
Likely follow-ups
Common mistakes
- Quoting the 20-tokens-per-parameter ratio as a law of nature rather than a compute-optimal estimate under specific assumptions.
- Ignoring inference cost when choosing model size.
- Assuming lower pre-training loss automatically means better results on your product task.
- Confusing parameter count with active parameters in mixture-of-experts models.
The 0 to 1 AI Engineer Interview Playbook covers the answer frameworks, system-design walkthroughs and an 8-week plan.
Get the playbook 40% off with code EARLYBF until Nov 3, 11:59 PM PTEARLYBF: 40% off until Nov 3Smaller start: Score your answers against 50 rubrics, $9. 50 AI engineer questions, one rubric each, PDF on Gumroad.
Interviewing for PM or TPM? See the full PM package, $59 on Gumroad: a resume system, a PM, TPM and PGM interview handbook, a 1:1 cheat sheet and a PM playbook. PM and TPM content, not AI engineer prep.
Related questions
Based on question #5 in the full question bank.