Loop Scaling, Specialization, and Parallel Inference for Transformers
This arXiv study examines looping in dense decoder-only transformers, where weights are reused to make models deeper without increasing parameter count. It finds that looping the full transformer gives the strongest qual...