arXiv:2611.07920v1 Announce Type: new
Abstract: A child who learns to add small numbers adds numbers of any length. LSTMs and Transformers trained on short sequences fail on longer ones. We trace the difference to what a learner reads when it decides. A position, a count or a time keeps taking new values as the input grows: it is a coordinate, and a learner that reads one meets in every long input values its experience never covered. Whether two things are the same has the same two answers at every size: it is a coincidence, and a learner that reads only coincidences finds in a long input nothing that short inputs lacked, so finite experience fixes its rule for every size. The memory carries the size of the input and the rule stays finite, and this determines the memory a generalizing brain can have: a memory that grows while what the learner reads of it stays finite has to look the same from every place, so it is made of cycles, and the past is reached by asking what happened when the cycles last stood where they stand now. The cycles are the grid modules, the cells that detect their return the place cells, the records the hippocampal index; a prefrontal controller moves the cycles and never sees them. Reading only coincidences costs no power: what finite experience fixes in such a learner is what Turing machines compute in step with their input, which is why a circuit that evolved to find the way home can learn arithmetic and grammar. In a network of neurons this machine of cycles is the dynamics seen through its symmetry, and symbols are its invariants. Trained on strings up to 40 symbols, the machine is at least 99% correct, on average over lengths 41 to 500, on nine of fifteen benchmark tasks, five of them tasks that no LSTM or Transformer solves; given new cycles after training it reverses strings of 15,015 symbols without error and fails beyond that on the curve the theory predicts; and its controllers run unchanged on neurons.
arxiv
Spatial Memory as a Universal Computer: How Finite Brains Generalize Without Bound
This paper proposes that generalization beyond training length arises when a learner reads only coincidences (invariant relations) rather than coordinates (position, count, time). It argues that such a learner's memory must be cyclic, linking to grid cells, place cells, and the hippocampal index, and that this architecture achieves Turing-complete computation. The authors report that their machine trained on strings up to 40 symbols achieves at least 99% accuracy on lengths 41-500 for nine of fifteen benchmark tasks, including five that LSTM/Transformer cannot solve, and can reverse strings of 15,015 symbols when given new cycles.
Offers a novel theoretical framework connecting neuroscience to AI generalization, with concrete benchmark results showing advantages over standard sequence models.
Comments & Questions (0)
No comments yet
Be the first to comment!