LearnThatStack Ace your next interview
Deep Learning & Neural Networks · question
Question 6 of 48

What does the learning rate control, and what goes wrong at either extreme?

beginner
← All Deep Learning & Neural Networks questions
Re-explain

The learning rate sets how far each weight moves along its gradient on every update. The gradient gives the direction. The learning rate gives the step size. It is usually the first thing to tune, because a wrong value ruins a run no matter how good the architecture is.

Set it too high and the steps overshoot the valley. Loss jumps around, or shoots to not-a-number as weights blow up. Set it too low and training crawls. You burn hours of compute and may stall on a flat stretch before reaching anything useful.

Most teams start around 0.001 with Adam, watch the first few hundred steps, then adjust by factors of ten. Decaying the rate later in training helps the weights settle instead of bouncing near the bottom.

Rewriting in plainer words…

This answer doesn't lend itself to a diagram - it reads best . No credits were charged.

Why there's no diagram: “”

The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓

The diagram below the answer is the concept . Jump to it ↓

Tailored explanation · switch back to · ·
What should the new diagram focus on?
How well did you know this?
AI:

Saved in this browser - sign in to keep your review list.

How should your speech become text?

Listening… your words appear above as you speak - tap Stop when you're done.

Recording · cr - tap Stop & transcribe when you're done.

Transcribing with AI…

Voice:

Keep going - a few more words and AI can grade it.

Interview lens

Likely follow-ups, what you can say, and the weak answers to avoid.

Sign in free to open it Free account - the lens opens as soon as you're back.

Want a quick review of the fundamentals? See the Deep Learning & Neural Networks cheatsheet.

← Back to all Deep Learning & Neural Networks questions
Pro · $10/mo

41 of 48 Deep Learning & Neural Networks answers are in Pro.

Full answers, code samples, and AI explanations that go simpler or deeper. Cancel anytime.

  • Full answers + code
  • AI explanations, simpler or deeper
  • 1,000 AI credits / month
  • Cancel anytime