Live intelligence

Latest AI updates

2 updates
Research

HiUni: A Unified Multimodal Model Sharing One Backbone for Understanding and Generation

HiUni is a unified multimodal model that reuses the same language-aligned ViT and dense Transformer backbone for both visual understanding and image generation, confining task-specific computation to the output side. To handle pixel-space optimization and the loss of fine spatial detail in pretrained ViT features, it introduces a Pixel Diffusion Head and a lightweight HiGate pathway. Experiments report an average understanding score similar to the pretrained VLM while achieving competitive image generation and editing performance. The authors argue a single dense backbone can support both capabilities without task-specific backbone computation.

unified multimodal modelsvision-language modelsimage generation
Research

MoE Routing Compared: DeepSeek, Qwen, Mixtral, Llama 4, gpt-oss

This article compares routing mechanisms across 10 open Mixture-of-Experts (MoE) models, detailing expert counts, top-k values, shared experts, gating functions, and load-balancing methods. It highlights trends like growing expert counts (from 8 to 384) and variations in top-k (from 1 to 8). The comparison is based on verified config.json files and technical reports, offering a practical reference for model deployment.

Mixture of ExpertsModel ArchitectureOpen Source Models