Back to AI News
arxiv

Forward-Value Discretization Speeds Up Stochastic Diffusion Sampling

This arXiv paper analyzes why higher-order discretization does not generally improve the convergence order of stochastic diffusion samplers, establishing matching upper and worst-case lower bounds. It introduces forward-value discretization, a training-free framework that evaluates the model at a lookahead estimate built from previous predictions. The resulting F-DDPM adjusts the lookahead for subsequent noise injection and reuses prior model predictions and injected noise, with first-order convergence guarantees at one model evaluation per iteration. Across CIFAR-10, ImageNet, FFHQ, and LSUN, F-DDPM reports lower FID than evaluated stochastic baselines at matched NFE budgets.

AI Brief

It offers a practical, training-free sampling method that narrows the efficiency gap between stochastic and deterministic diffusion sampling.

arXiv:2611.14328v1 Announce Type: cross Abstract: Stochastic diffusion samplers often require substantially more neural function evaluations (NFEs) than deterministic samplers to achieve comparable sample quality. Although higher-order discretization accelerates deterministic sampling, its benefits for stochastic sampling remains unclear. We establish matching upper and worst-case lower bounds showing that stochastic samplers with higher-order approximations remain first-order convergent, even with exact score functions. Thus, increasing the approximation order does not generally improve the convergence order. Motivated by this limitation, we introduce forward-value discretization, a training-free framework that evaluates the data/noise prediction model at a lookahead estimate constructed from previous predictions. For deterministic sampling, the resulting Forward DPMSolver (F-DPMSolver), improves sample quality at low NFE budgets through its DDIM-based variant, despite remaining first-order convergent. For stochastic sampling, we develop Forward DDPM (F-DDPM), which evaluates the model at a cleaner lookahead point adjusted for subsequent noise injection. This estimate reuses both the previous model prediction and the previously injected noise. Both variants admit first-order convergence guarantees with one model evaluation per iteration. Across CIFAR-10, ImageNet, FFHQ, and LSUN, F-DDPM consistently achieves lower FID than the evaluated stochastic baselines at matched NFE budgets. With 1.5 times as many NFEs as deterministic baselines, it achieves competitive sample quality, narrowing the practical efficiency gap between stochastic and deterministic sampling. Code is available at https://github.com/Na-Li66/F-DDPM.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!