Research
Natural PC adds lateral circuits for local natural-gradient learning
Researchers introduce Natural PC, which augments predictive coding with a lateral circuit that learns activity correlations determining the Fisher information matrix and implicitly computes its inverse. The approach enables exact natural-gradient learning locally in space and time without expensive matrix inversion. Across tested image reconstruction benchmarks, Natural PC reports state-of-the-art sample and compute efficiency among predictive coding methods. It also scales to ImageNet and is claimed to match backpropagation trained with AdamW and second-order optimizers at an order of magnitude lower compute and with two orders of magnitude fewer training samples.
predictive codingnatural gradientneural network training