LLM Fundamentals · question

Question 1 of 58

What is a large language model, and how does it actually generate text?

beginner

Re-explain

A large language model (LLM) is a neural network trained on huge amounts of text to predict the next token.

The model reads the text so far, scores every vocabulary entry, and samples one token from the resulting probabilities.

tokens = encode(prompt)
while not done(tokens):
    logits = model(tokens)         # one score per vocabulary entry
    probs = softmax(logits[-1])    # next-token probabilities
    tokens.append(sample(probs))   # pick one, feed it back in
print(decode(tokens))

Autoregressive means each output becomes part of the next input.

The weights hold patterns, not a fact database, so fluent text is not proof of correctness.

Rewriting in plainer words…

This answer doesn't lend itself to a diagram - it reads best . No credits were charged.

Why there's no diagram: “”

The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓

The diagram below the answer is the concept . Jump to it ↓

Tailored explanation · switch back to · ·

What should the new diagram focus on?

Guided practice for LLM Fundamentals One question at a time. Answer out loud, get graded, see what you missed.

See it - interactive diagram

How an LLM writes text, one token per pass

tokens = encode(prompt)

while not done(tokens):

# one score per vocabulary entry

logits = model(tokens)

# next-token probabilities

probs = softmax(logits[-1])

# pick one, feed it back in

tokens.append(sample(probs))

print(decode(tokens))

tokensgrows by one per pass

ThecapitalofAustralia

prompt

probstop few of the vocabulary

empty

Console

1step

encode splits the prompt into tokens, which are words or pieces of words. These four tokens are all the model gets to read.

1 / 10

An LLM writes one token per pass and reads it back in, so each pick, even a wrong one, becomes input for the next pass.

Other questions have diagrams like this too. Sign in to ask for one where it's missing - it's free.

Create a free account

How well did you know this?

AI:

Saved in this browser - sign in to keep your review list.

How should your speech become text?

Listening… your words appear above as you speak - tap Stop when you're done.

Recording · cr - tap Stop & transcribe when you're done.

Transcribing with AI…

Voice:

Keep going - a few more words and AI can grade it.

Interview lens How interviewers actually use this question

Likely follow-ups

What exactly is a token, and why don't models work with whole words? A token is often a piece of a word. Whole-word vocabularies can't cover rare words, and single characters make sequences long and costly.
What does temperature change in the sampling step you described? Should say temperature rescales the logits before softmax. Low values push toward the top token, high values flatten the distribution.
If the output reads fluently, why can it still be flat wrong? Should tie it to the objective. The model picks likely tokens, and nothing in the loop checks them against a source of truth.
Your loop reruns the model on the whole sequence each step. How do real systems avoid that cost? Should name the KV cache, which keeps earlier tokens' keys and values so each step computes only the new token, trading memory for speed.
If pretraining only teaches next-token prediction, how does the model learn to follow instructions? Should separate pretraining on raw text from post-training, meaning instruction tuning and preference tuning like RLHF, which shapes how it responds.

What you can say

An LLM is a neural network trained on a huge amount of text to do one thing, predict the next token.
When it generates, it reads the text so far and produces a score for every entry in its vocabulary.
A softmax turns those scores into probabilities, and the model samples one token from them.
It appends that token and runs again, and that loop of each output becoming part of the next input is what autoregressive means.
One thing I'd stress is that the weights store patterns, not a database of facts, so fluent text is no proof the answer is correct.

Weak answers to avoid

Describes it as a database or search engine of facts The weights hold learned patterns, not stored records. Say that directly, and add that fluent output is no proof of correctness.
Says it writes whole words or sentences at once It emits one token per step, and tokens are often word pieces. Walk the loop: score the vocabulary, softmax, sample, append, repeat.
Claims it always picks the single most likely word Generation samples from a probability distribution, which is why one prompt can return different text. Name the sampling step.
Stays vague with 'it understands language' Talk of understanding gives no mechanism. Name the training objective, next-token prediction, and describe the generation loop step by step.

Interview lens

Likely follow-ups, what you can say, and the weak answers to avoid.

Want a quick review of the fundamentals? See the LLM Fundamentals cheatsheet.

From the blog: Jev explained: the AI model that returns decisions, not text

← Back to all LLM Fundamentals questions

Pro · $10/mo

51 of 58 LLM Fundamentals answers are in Pro.

Full answers, code samples, and AI explanations that go simpler or deeper. Cancel anytime.

Full answers + code
AI explanations, simpler or deeper
1,000 AI credits / month
Cancel anytime

Get Pro → See plans & pricing →