Live intelligence

Latest AI updates

2 updates
Models

Lara Mini Base: A Multilingual MoE Base Model Trained on 6T Tokens

Lara Mini Base is a 7.2B-parameter mixture-of-experts model with 0.6B active parameters, trained from scratch on 6T tokens spanning English and 44 other languages, covering all 24 official EU languages. It supports 128k-token contexts and is released under the CC BY-NC-SA 4.0 license. The authors report it achieves the highest average score among open base models with up to 2B active parameters on English, multilingual, math, and code benchmarks, with the largest margins on multilingual and math tasks. They also plan to release T-HellaSwag, a machine-translated commonsense reasoning benchmark in 110 language variants.

multilingual language modelsmixture-of-expertsefficient inference
Research

CardioLM: A multilingual suite of small language models for the cardiology domain

Researchers introduce CardioCCC, a corpus of 508 cardiology case reports originally in Spanish and extended to Czech, Dutch, English, Italian, Romanian and Swedish via machine translation, annotation projection and clinician review, annotated for diseases, symptoms, procedures and medications. The accompanying CardioLM suite comprises CardioBERTa encoders built by continued pretraining on cardiology and biomedical corpora, plus CardioNER named-entity-recognition models fine-tuned on CardioCCC. Cardiology-specific pretraining cut pseudo-perplexity by up to a factor of eight but shifted strict span F1 by at most 0.04, while jointly trained multilingual XLM-RoBERTa-large models gave the best strict F1 in nearly every language and entity type, beating the strongest monolingual system by up to 0.13 F1. Clinicians rated CardioNER well above a rule- and dictionary-based baseline, with mean ratings rising from 26% to 73%, though many residual errors were boundary-related, especially for medications.

CardiologyClinical NLPMultilingual language models