arXiv:2611.26437v1 Announce Type: cross
Abstract: Frontier text-to-image (T2I) foundation models predominantly rely on multi-billion parameter Diffusion Transformers (DiTs) pretrained across massive compute clusters, creating an accessibility barrier for academic laboratories. When training DiTs from scratch on single consumer GPUs (e.g., an NVIDIA RTX 4090 with 24\,GB VRAM), practitioners encounter three primary bottlenecks: quadratic attention sequence costs, Just-In-Time (JIT) compilation graph breaks induced by variable aspect-ratio bucketing, and an autoencoder throughput wall where frozen Variational Autoencoder (VAE) operations dominate step latency. In this work, we introduce \textbf{Asuka-FM}, an end-to-end system and architectural framework demonstrating that competitive dual-stream DiTs ($\sim$206M parameters) can be pretrained from scratch on a single consumer GPU in $\sim$160 GPU hours for under \$55. Asuka-FM integrates four core designs: (i) an AdaLN-free dual-stream DiT backbone reducing parameter count; (ii) structured sequence sparsification via Sparse-Dense Residual Fusion (SPRINT) to cut attention FLOPs by over 50\%; (iii) continuous aspect-normalized Rotary Position Embeddings (RoPE) coupled with shifted square crop training, guaranteeing static tensor shapes while unlocking zero-shot virtual camera control; and (iv) a pixel adaptation phase utilizing a lightweight (10M parameter) Patch Detailer Head operating under velocity regression ($v$-prediction). Evaluated on open-domain benchmarks without benchmark-specific fine-tuning, Asuka-FM achieves GenEval 0.588 / DPG 78.96 scores at 256px and GenEval 0.609 / DPG 78.51 scores at 512px, setting a new Pareto-optimal efficiency standard for democratic foundation modeling.
arxiv
Asuka-FM: Efficient Text-to-Image Pretraining on a Single Consumer GPU
Asuka-FM is an end-to-end system and architectural framework for pretraining dual-stream Diffusion Transformers from scratch on a single consumer GPU. It combines an AdaLN-free backbone, Sparse-Dense Residual Fusion to cut attention FLOPs by over 50%, aspect-normalized RoPE with shifted square crop training, and a lightweight Patch Detailer Head for pixel adaptation. The authors report pretraining a ~206M-parameter model in ~160 GPU hours for under $55, with GenEval and DPG scores at 256px and 512px without benchmark-specific fine-tuning.
It provides a concrete low-cost recipe and benchmark results for training text-to-image diffusion transformers without cluster-scale compute.
Comments & Questions (0)
No comments yet
Be the first to comment!