Nesterov Accelerated Gradient (NAG) — A Momentum Upgrade That Uses “Future” Gradients to Reduce Overshooting

Nesterov Accelerated Gradient (NAG) is an optimization method that improves classic Momentum-based gradient descent by computing the gradient not at the current point, but at a “predicted” point you’re about to move to. Instead of only accumulating velocity, NAG peeks ahead first, then corrects the direction—often yielding faster and more stable convergence.

02/25/2026