Natural PC adds lateral circuits for local natural-gradient learning
Researchers introduce Natural PC, which augments predictive coding with a lateral circuit that learns activity correlations determining the Fisher information matrix and implicitly computes its inverse. The approach enables exact natural-gradient learning locally in space and time without expensive matrix inversion. Across tested image reconstruction benchmarks, Natural PC reports state-of-the-art sample and compute efficiency among predictive coding methods. It also scales to ImageNet and is claimed to match backpropagation trained with AdamW and second-order optimizers at an order of magnitude lower compute and with two orders of magnitude fewer training samples.
It presents a concrete research advance that could make predictive coding more competitive with backpropagation in sample and compute efficiency.
Comments & Questions (0)
No comments yet
Be the first to comment!