HiUni: A Unified Multimodal Model Sharing One Backbone for Understanding and Generation
HiUni is a unified multimodal model that reuses the same language-aligned ViT and dense Transformer backbone for both visual understanding and image generation, confining task-specific computation to the output side. To handle pixel-space optimization and the loss of fine spatial detail in pretrained ViT features, it introduces a Pixel Diffusion Head and a lightweight HiGate pathway. Experiments report an average understanding score similar to the pretrained VLM while achieving competitive image generation and editing performance. The authors argue a single dense backbone can support both capabilities without task-specific backbone computation.