Leap in Second-Order Optimization: Shampoo Runtime Boosted 40%

A Google-led research team has introduced a new method for optimizing neural network parameters that is faster than all common first-order methods on complex problems.