Research
Study of 489K LLM Conversations Reveals Gap Between Grounding Claims and Inspectable Evidence
An analysis of 489,283 assistant-user response pairs from ShareChat and WildChat finds that broad attribution language appears in 31.9% of responses, while explicit URLs appear in only 1.33%, highlighting a gap between claims of grounding and preserved inspectable support. The study examines how response cues like attribution and uncertainty relate to users' accepting, checking, or challenging behaviors, finding that uncertainty is the only cue with a small statistically distinguishable association favoring checking. Scrutiny is concentrated in certain conversations, and its turn-to-turn persistence diminishes when conversation-level differences are accounted for.
Large language modelsUser trustEvidence grounding