On the Link Between Optimization and Polynomials, Part 2
Momentum: when Chebyshev meets Chebyshev.
An analysis of momentum can be tightened using a combination Chebyshev polynomials of the first and second kind. Through this connection we'll derive one of the most iconic methods in optimization: Polyak momentum.