arXiv:2611.17239v1 Announce Type: cross
Abstract: In interactive systems where users engage with LLMs, users are adaptive and, after observing LLM responses, can change their prompting strategy, as well as the type and difficulty of subsequent tasks. Thus model responses can influence the set of prompts on which the model is subsequently evaluated. We study the implications of this user adaptivity and prompt endogeneity for the evaluation of LLMs. We define two classes of estimands for comparing models: the common-distribution effect, which compares model performance on the same distribution of prompts, and the induced-distribution effect, which compares each model on the prompt distribution generated through user interaction. We show that offline benchmark evaluation and live A/B testing target different quantities, with the standard difference-in-means estimators for the two methods converging to the common-distribution effect and the induced-distribution effect, respectively. This distinction can affect decision-making: the two comparisons can rank models differently and lead to discrepancies in deployment recommendations. We characterize when such ranking reversals occur and provide a sufficient condition under which the rankings are guaranteed to agree. This result provides a practical heuristic for assessing whether offline evaluation is sufficient to reproduce the decision suggested by a live experiment. Third, we show, in a stylized linear model of interactions, that user adaptivity can increase variance in estimation. Finally, we discuss implications of user adaptivity for the interpretation of experiments and show that adaptivity creates a common mechanism leading to common evaluation challenges, such as reward hacking and novelty effects. Together, this work highlights the necessity of measuring both model performance as well as changes in the user behavior that the model induces.
arxiv
When Both Sides Adapt: The Challenges of Evaluating Interactive LLM Systems
This paper examines how user adaptivity complicates evaluation of interactive LLM systems, since users change their prompts and tasks after observing model responses. It distinguishes between common-distribution and induced-distribution effects, showing offline benchmarks and live A/B tests target different quantities and can rank models differently. The authors characterize when ranking reversals occur and provide a sufficient condition for agreement, plus discuss variance increases and links to reward hacking and novelty effects.
Provides a concrete conceptual framework for a growing practical problem: evaluating LLMs in adaptive, interactive settings where offline benchmarks may mislead deployment decisions.
Comments & Questions (0)
No comments yet
Be the first to comment!