The round tests whether you can build a reliable product around a model you did not train. You treat the model as a remote dependency. It is paid per token, sometimes slow, and non-deterministic, so the same input can give different output.
Model design asks how to make the model better. System design asks what to do when it is wrong, slow, or down.
Designing the model
Designing the system
What you work on
Data, architecture, training loss
Latency and cost budgets
How you measure it
Offline benchmark scores
Evals on live traffic
What you ship
One artifact
Retrieval, state, guardrails, fallbacks
Interviewers listen for numbers: cost per request, a latency target, and a plan for failure. Saying "just call the API" without those is the usual reason people fail.
Rewriting in plainer words…
This answer doesn't lend itself to a diagram - it reads best . No credits were charged.
The same input can give different output. How does that change the way you test the system?Exact-match asserts break. Score a fixed eval set by pass rate, pin the model version, and note that low temperature narrows variance without removing it.
What you can say
This round tests whether I can build a reliable product around a model I didn't train, so I treat the model as a remote dependency.
That dependency is paid per token, sometimes slow, and non-deterministic, so the same input can come back with a different answer.
Model design asks how to make the model better through data, architecture and training loss, and it's judged on offline benchmark scores.
System design asks what to do when the model is wrong, slow or down, so I work in latency and cost budgets and measure with evals on live traffic.
Model design ships one artifact. What I ship is retrieval, state, guardrails and fallbacks around the model.
So I'd bring numbers: a cost per request, a latency target, and a plan for when the model fails. 'Just call the API' without those is the usual reason people fail.
Weak answers to avoid
Says 'just call the API' and stopsThat's the usual reason people fail. Give a cost per request, a latency target and what happens when the call fails or times out.
Talks about training and fine-tuning the modelThat answers the model design question. This round assumes a model you didn't train, so talk about budgets, retrieval, guardrails and fallbacks around it.
Assumes the model is always fast, up and rightThe model is a remote dependency that can be slow, down or wrong. Say what happens in each case, such as a timeout, a fallback, or a guardrail that catches bad output.
Measures quality only with offline benchmarksBenchmarks score the model in general. Interviewers want evals on your own live traffic, since that is where your product's failures show up.
Interview lens
Likely follow-ups, what you can say, and the weak answers to avoid.
