Research
Re-Search probes LLM research ideation with a Twenty Questions-style game
The paper introduces Re-Search, an evaluation that adapts Twenty Questions to research ideation: a model is given only a published paper's research problem and must rediscover its core idea through yes-or-no questions. Built from 200 ICML 2026 papers, it tracks how a model's hypothesis evolves under binary oracle feedback, isolating ideation from execution. Across six recent LLMs, nearly all progress occurred within the first 30 of 100 questions, after which most models plateaued. Controlled interventions indicate the bottleneck is generation rather than judgment: models can recognize good ideas or questions when offered but often cannot produce them independently.
research ideationLLM evaluationautoresearch systems