Research
Towards Reliable Scaling Laws for Tabular Foundation Models
The paper investigates whether scaling laws for TabPFNv2-style tabular foundation models reliably extrapolate to larger compute budgets and generalize across evaluation settings. It finds differences between synthetic and real-world scaling behavior and shows that accurately fitting a scaling law does not necessarily yield reliable prescriptions on unseen datasets. Selecting configurations using too few evaluation datasets leads to systematically over-optimistic performance estimates, and compute-optimal scaling requires expanding the range of synthetic pretraining dataset sizes alongside model size and training compute. The insights are used to guide training of more efficient TabPFNv2-like models with the open-source nanoTabPFN pipeline.
TabPFNscaling lawstabular data