LeCat: Measuring LLM Category Learning from Pretraining Data
LeCat is a two-part framework for analyzing how LLMs learn category knowledge from pretraining data, distinguishing memorization, generalization, and flexible use. Experiments on open-source models and a controlled 68M m...