When Both Sides Adapt: The Challenges of Evaluating Interactive LLM Systems
This paper examines how user adaptivity complicates evaluation of interactive LLM systems, since users change their prompts and tasks after observing model responses. It distinguishes between common-distribution and indu...