Dynamics of learning in singular models: SGD, natural gradient, and resolution of stratifolds:

In this video we will revisit the double cone model, a toy model to examine leaning
dynamics in singular statistical models
From there we propose to use stratifolds : a generalization of smooth manifolds - to describe the parameter space of singular statistical models and use the resolution of the stratifold to speed up convergence and avoid attractor behavior near the singularity.

stratifolds: