Live intelligence

Latest AI updates

1 updates
Models

Lara Mini Base: A Multilingual MoE Base Model Trained on 6T Tokens

Lara Mini Base is a 7.2B-parameter mixture-of-experts model with 0.6B active parameters, trained from scratch on 6T tokens spanning English and 44 other languages, covering all 24 official EU languages. It supports 128k-token contexts and is released under the CC BY-NC-SA 4.0 license. The authors report it achieves the highest average score among open base models with up to 2B active parameters on English, multilingual, math, and code benchmarks, with the largest margins on multilingual and math tasks. They also plan to release T-HellaSwag, a machine-translated commonsense reasoning benchmark in 110 language variants.

multilingual language modelsmixture-of-expertsefficient inference