Live intelligence

Latest AI updates

5 updates

Research Editor selected

NaijaFacts: Benchmark Reveals AI Knowledge Gap on Nigerian Facts

NaijaFacts is a new benchmark of factual and statistical questions about Nigeria, authored by native speakers and available in five languages. Testing eight models shows a dominant knowledge effect: English-query accuracy is only 20% for Nigeria versus 49-50% for US/UK, while language effects are smaller. The gap persists across all 54 African countries, with models scoring 96% on culture and 98% on politics but only 26% on statistics.

Why it matters: Provides concrete evidence of a major AI knowledge gap for African facts, with actionable insights for multilingual and knowledge-aware model development.

arxiv18h agoScore 85

AI evaluationMultilingual NLPKnowledge representation

Research Editor selected

Reasoner-Checked Crosswalks for the Information Exchange Standard: From Pairs to a Certified Hub

This paper formalizes ontology alignment validation for a national digital twin ecosystem, applying coherence and conservativity checks to crosswalks from the Information Exchange Standard (IES) into thirteen ontologies. Pairwise certification catches many violations but fails to compose, as merging three building crosswalks into a hub causes unsatisfiable classes due to a homonym. The authors show that a hub, not a pair, is the correct unit for certification, and they identify an inconsistency between two published standards that must be resolved.

Why it matters: Provides a concrete, logic-based method for validating ontology alignments and demonstrates a critical compositionality failure, offering practical guidance for digital twin interoperability.

arxiv18h agoScore 85

ontology alignmentdigital twinknowledge representation

Research Editor selected

Detecting Suppressed Knowledge in LLMs via Activation Fragmentation

This paper introduces a framework that measures the geometric coherence of hidden-state differences between concept-specific queries and anonymized paraphrases to detect whether an LLM genuinely knows a concept. It shows that genuine knowledge forms tight clusters while ignorance causes fragmentation, and that this geometry correlates with adversarial recovery after unlearning across multiple models and attack types. The method offers a single-phase, attack-free proxy for unlearning robustness that captures knowledge inaccessible to output-level metrics.

Why it matters: Provides a novel, attack-free method to evaluate LLM knowledge and unlearning robustness, with empirical validation across multiple models and attacks.

arxiv1d agoScore 85

LLM interpretabilityknowledge representationmachine unlearning

Research

Toward Knowledge-Centric AI: Explicit Knowledge Layer Across Agents and Generations

The paper argues that AI agents should be built around an explicit, continuously evolving knowledge layer independent of any single agent, rather than trapping knowledge in model parameters or context windows. It identifies internalization as the bottleneck, proposing knowledge foundation models for shared neural capacity and a research agenda for reuse, cost efficiency, auditability, and continuity.

arxiv2d agoScore 78

AI agentsknowledge representationmachine learning research

Research Editor selected

Chess Transformer Study Shows Lower-Skill Play Retains Internal Knowledge

Researchers analyzed Maia-2, a chess transformer, to investigate whether simulating lower-skill players reflects reduced knowledge or altered use of retained knowledge. They found probe accuracy for 172 chess concepts stayed constant across skill levels, and fine-tuning only the output layer on high-skill data boosted low-skill best-move accuracy from 14.4% to 82.8%. The results suggest weaker behavior can understate a model's internal knowledge.

Why it matters: Provides concrete evidence on how skill conditioning affects model internals, with implications for AI evaluation and user simulation.

arxiv2d agoScore 85

AI interpretabilityuser simulationchess AI