Gradient descent finds good weights by repeatedly stepping downhill on the loss surface. Picture the loss as a landscape where every position is one setting of the weights. The gradient points in the direction of steepest increase, so you move the opposite way.
One iteration is three moves: run data through the network, measure the loss, then subtract a fraction of each gradient from its weight. Repeat over many passes and the weights drift toward a region where the loss stops improving.
Nothing about this guarantees the lowest possible point. You only ever see the slope directly under your feet, never the whole map. In practice that is acceptable, because a good-enough valley found in hours beats a perfect one you never reach.
Rewriting in plainer words…
This answer doesn't lend itself to a diagram - it reads best . No credits were charged.
