arXiv:2611.18315v1 Announce Type: new
Abstract: Web platforms record user and item activity in relational databases whose schemas differ from platform to platform, which makes it difficult to transfer predictive models between them. Relational foundation models address this through cross-database pretraining, yet recent improvements have relied on increasingly large pretraining corpora. We ask whether a better architecture, together with features specific to the target database, can deliver comparable gains from far less pretraining data. We introduce MoREGT, a mixture-of-relational-experts graph transformer that represents both rows and cells as tokens, adds the log-ages of rows and the names of foreign-key relationships to attention, and routes cells and rows to experts by column metadata rather than by values. A label-blind LLM agent complements the model: it reads the target database's schema and column statistics and adds validated single-row derived columns, without access to labels or model scores. On 21 RelBench tasks with leave-one-database-out pretraining on six databases, the MoREGT backbone improves on the Relational Transformer pretrained on the same databases by 2.25 points of mean AUROC and 5.83 points of mean nMAE. The agent's columns raise mean AUROC from 78.45 to 79.63 and lower mean nMAE from 27.91 to 26.98, the highest means on both metrics among the compared methods, including RT-J, which pretrains on about 400 times as many databases.
arxiv
MoREGT: Agent-Guided Mixture-of-Relational-Experts Boosts Cross-Database Generalization
The paper introduces MoREGT, a mixture-of-relational-experts graph transformer for cross-database generalization, which represents rows and cells as tokens and uses column metadata for expert routing. A label-blind LLM agent adds validated derived columns from target database schema and statistics. On 21 RelBench tasks, MoREGT outperforms the Relational Transformer pretrained on the same databases and achieves higher mean AUROC and lower nMAE than RT-J, which pretrains on about 400 times more databases.
This presents a concrete architectural and agent-based method that improves cross-database generalization with less pretraining data, offering practical value for relational learning.
Comments & Questions (0)
No comments yet
Be the first to comment!