Back to AI News
arxiv

Discrete diffusion world model cuts inference cost while improving data efficiency

This arXiv paper proposes a two-stage discrete diffusion approach for world models, replacing next-token prediction with iterative masked predictions performed by a lightweight refinement transformer. Building on an existing autoregressive world model, it reports a 79x speed-up with a modest quality trade-off. It also shows that consistent gameplay can be reproduced using a dataset of approximately 28 hours of gameplay. The work targets real-time interactive play and ideation, where inference speed and data efficiency are critical.

AI Brief

It offers a concrete architectural step toward real-time, data-efficient world models, which matters for interactive generation and creative tooling.

arXiv:2611.22497v1 Announce Type: new Abstract: State-of-the-art world models have been used to produce sequences of gameplay that accord with provided user-input actions. However, high quality, consistent gameplay generation often comes at the cost of inference speed, making real-time interactive play and ideation challenging. The substantial quantities of data used to train state-of-the-art models have also made custom training of these models appear impractical, limiting many of their potential uses in creative fields. In this work we demonstrate a major step towards efficient world models: achieving both fast high-quality inference, and improved data efficiency. This is done by replacing the next-token prediction paradigm with a two-stage discrete diffusion-based approach, introducing a lightweight refinement transformer which carries out iterative masked predictions. Building on an autoregressive world model capable of generating highly consistent and complex sequences over minutes (Kanervisto et al., 2025), we achieve substantial model speed-up (79x) with a modest quality trade-off. Additionally, we demonstrate that this model can be trained to reproduce high-quality, consistent gameplay using a dataset containing approximately one day (28 hours) of gameplay data. Our results demonstrate progress towards removing barriers to the use of world models in settings where real-time inference and data efficiency are essential.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!