Most proofs in optimization consist in using inequalities for a particular function class in some creative way.
This is a cheatsheet with inequalities that I use most often. It considers class of functions that are convex,
strongly convex and $L$-smooth.
My friend Rémi Leblond has recently uploaded to ArXiv our preprint on an asynchronous version of the SAGA optimization algorithm.
The main contribution is to develop a parallel (fully asynchronous, no locks) variant of the SAGA algorighm. This is a stochastic variance-reduced method for general optimization, specially adapted for problems …
TL;DR: I describe a method for hyperparameter optimization by gradient descent.
Most machine …