Live intelligence

Latest AI updates

2 updates
Research

Parameter-Efficient Conciseness Adaptation: Differentiable Length Control for Concise Clinical Reasoning in Smaller Language Models

The paper introduces PECA, a method that adds a differentiable conciseness term to QLoRA supervised fine-tuning for smaller language models. Instead of relying on preference data, reward models, or proprietary teachers, PECA raises end-of-sequence probability once generation exceeds a token budget. On a 1.7-billion-parameter model fine-tuned on the medical-o1-reasoning-SFT corpus, PECA reduced mean response length by 72.8% while improving multiple-choice diagnostic accuracy from 40.8% to 44.0%. The authors also describe a sweep over conciseness strength that traces an accuracy-to-length frontier.

small language modelsclinical NLPparameter-efficient fine-tuning
Research

CardioLM: A multilingual suite of small language models for the cardiology domain

Researchers introduce CardioCCC, a corpus of 508 cardiology case reports originally in Spanish and extended to Czech, Dutch, English, Italian, Romanian and Swedish via machine translation, annotation projection and clinician review, annotated for diseases, symptoms, procedures and medications. The accompanying CardioLM suite comprises CardioBERTa encoders built by continued pretraining on cardiology and biomedical corpora, plus CardioNER named-entity-recognition models fine-tuned on CardioCCC. Cardiology-specific pretraining cut pseudo-perplexity by up to a factor of eight but shifted strict span F1 by at most 0.04, while jointly trained multilingual XLM-RoBERTa-large models gave the best strict F1 in nearly every language and entity type, beating the strongest monolingual system by up to 0.13 F1. Clinicians rated CardioNER well above a rule- and dictionary-based baseline, with mean ratings rising from 26% to 73%, though many residual errors were boundary-related, especially for medications.

CardiologyClinical NLPMultilingual language models