Scaling Laws for SAE Training Data
This paper investigates how the number of unique activations affects sparse autoencoder (SAE) training when the total training budget is fixed. It finds that quality improves rapidly with buffer size but then plateaus, w...