RL Post-Training Improves Ranking but Hurts Recall in Generative Recommenders
A new arXiv paper studies generative recommenders that decode a user's next item semantic ID with a language model and are post-trained with reinforcement learning after supervised fine-tuning. Across five datasets, the authors find RL post-training raises hit@K for small K but lowers hit@K when K is in the hundreds, meaning the model becomes a better ranker and a worse retriever. They trace the loss to negative advantages assigned to sampled rollout candidates, since only one next item is logged per user, and propose out-of-rollout anchoring, an added loss term that preserves the SFT model's ordering of items outside the rollout. The remedy restores candidate-list recall to SFT levels on all five datasets while keeping top-of-list gains, and also works under other RL configurations.
Why it matters: It exposes a concrete, previously unmeasured failure mode of RL post-training in recommender systems and offers a simple, tested fix that practitioners can apply.