Back to AI News
arxiv

PARIS: A multi-agent LLM framework for end-to-end, PRISMA-guided systematic literature reviews with pre-synthesis corpus quality assessment

PARIS is an open-source framework in which a conductor agent running in Claude Code orchestrates 15 specialized LLM agents across a systematic review workflow, from research scoping to manuscript draft export. Agents coordinate through persisted artifacts validated by the conductor, while retrieval, bibliometric analysis and scoring are delegated to deterministic Python utilities; three human checkpoints gate inclusion, corpus adequacy and synthesis outline. The paper introduces QualityScore, a 0-100 pre-synthesis index combining network, bibliometric, semantic and process sub-scores, and reports two case studies plus audit findings on failure modes. The authors state that output quality has so far been judged only by an internal reviewer agent and that independent validation against human reviews is the next step.

AI Brief

It offers a concrete open-source multi-agent design and corpus-quality metric for automating PRISMA-guided literature reviews, while clearly documenting current limitations.

arXiv:2611.12222v1 Announce Type: new Abstract: Systematic literature reviews (SRs) remain slow and labor-intensive, and most automation tools address single stages of the workflow. We present PARIS (PRISMA Agentic Research Intelligence System), an open-source framework in which a conductor agent running in Claude Code orchestrates 15 specialized large language model (LLM) agents, from scoping a research question to exporting a manuscript draft. Agents are specified declaratively, coordinate exclusively through persisted artifacts that the conductor validates before every transition, and delegate retrieval, bibliometric analysis and scoring to deterministic Python utilities. Three human checkpoints gate study inclusion, corpus adequacy and the synthesis outline. We also introduce QualityScore, a pre-synthesis index that aggregates normalized network, bibliometric, semantic and process sub-scores into a 0-100 grade. Two case studies illustrate both: AI chatbots and university students' learning (388 records, 100 included) and AI-assisted SR methods (131 records, 123 included, QualityScore 69.1 recomputed with a proxy network sub-score). They show end-to-end feasibility across domains and the diagnostic use of QualityScore: in the first run, corpus improvements chosen at the corpus-adequacy checkpoint raised the score from 74.6 to 81.9 and the number of pre-2023 studies from 1 to 19. Audits of both runs exposed failure modes: incomplete retrieval, counts that drifted between agents, unrecorded or delegated checkpoint decisions and a grade that depends on how citation data are handled. PARIS fully supports 8 of the 27 PRISMA 2020 items and partially supports 11. Output quality has so far been judged only by an internal reviewer agent; independent validation against human reviews is the next step.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!