The balance between feature learning and collapse in dynamical systems-based generative models
This theoretical paper uses dynamical systems and operator theory to analyze gradient flow for a general class of stochastic differential equations used in continuous-time generative models. It links mode collapse to low-rank biases and identifies two mechanisms: collapsed models as stable fixed points, and sequential feature learning in which smaller modes take progressively longer to learn. The authors frame these findings as a trade-off between feature learning and collapse and propose an operator view toward a unified learning theory for models such as normalizing flows and diffusion.
It offers a theoretical explanation of mode collapse in continuous-time generative models, giving researchers a framework for understanding training dynamics and representation learning.
Comments & Questions (0)
No comments yet
Be the first to comment!