RecoGEM: A Serving System for Production-Scale Generative Recommendation
RecoGEM is a production serving system for generative recommendation, where item identifiers are generated autoregressively instead of scoring candidate items. It uses a runtime-engine decoupling architecture to handle GR-specific responsibilities such as candidate-state transitions and live artifact updates, while the execution engine optimizes self-attention, candidate selection, and execution planning for short decoding sequences and large candidate populations. The system has been deployed for over nine months across production lines serving hundreds of millions of users and more than ten GR model families. On OneModel workloads, it reports over 1,000 QPS per serving instance, P99 latency below 35 ms, and a 2.88x throughput improvement over representative baselines.
Why it matters: It offers concrete architecture and production performance details for serving generative recommendation models at industrial scale.