Benchmarking LLMs on Heterogeneous Cognitive and Biomedical Knowledge Graphs
The study benchmarks GPT-5.1, Gemini 2.0 Pro, Llama 3, Qwen, and BioGPT across DrugBank, NeuroSynth, Cognitive Atlas, Brain Map, and GDPKG using a multitier query dataset under graph-grounded and graph-agnostic conditions. It reports that general-purpose models generally outperform domain-specific models on multi-hop and high-level reasoning, although performance declines as query complexity increases. Knowledge-graph grounding reduces hallucination rates across all evaluated models, while domain-specific models do not consistently outperform general-purpose models on specialized tasks without grounding.
It provides concrete evidence on how knowledge-graph grounding affects multi-hop reasoning and hallucination across general and biomedical LLMs.
Comments & Questions (0)
No comments yet
Be the first to comment!