Kierston & KierstonBench: Simulated City Benchmark for AI in Crime Investigation
The paper introduces Kierston, a city-scale simulation of Allentown, PA, and KierstonBench, a benchmark for evaluating language models in crime investigation and surveillance tasks. It generates realistic records from a common behavioral timeline and tests models on 32 task families. Evaluation of 16 models shows most fail, while a few frontier models solve most tasks but still fall for deceptions.
Why it matters: Provides a novel, concrete benchmark for a critical AI application area, with clear evaluation results and practical implications.