Model routing sends each request to the model that fits what the request needs. Small, cheap models handle easy work, and large models handle the hard cases. The router decides before the call, using rules, request length or a small classifier.
A cheap-first cascade decides after the fact instead. You always call the cheap model first and check its answer. You escalate to the stronger model only when that check fails.
async function cascade(prompt) {
const draft = await call('small-model', prompt);
if (passesCheck(draft)) return draft; // most traffic stops here
return call('large-model', prompt); // escalation pays twice
}
The tradeoff is that an escalated request pays for both calls and both latencies. A cascade saves money only while the escalation rate stays low. The check must also be cheap, because a check that calls a big model to grade defeats the point.
This answer doesn't lend itself to a diagram - it reads best . No credits were charged.
Why there's no diagram: “”
The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓
The diagram below the answer is the concept . Jump to it ↓