A.S.E 2.0: How Repository Context Impacts the Security of LLM-Generated Code
This paper introduces CASE-Bench, a benchmark of 145 repository-level tasks derived from real CVEs across 51 projects, to study how repository context affects the security of LLM-generated code. Experiments with 13 context construction methodshol and six LLMs show that functionality-oriented context improves functional pass rates more than secure pass rates, and that withholding safeguards or inserting insecure examples significantly increases exploitable code. Coding agents observe relevant safeguards in only 27-36% of trajectories, highlighting the need for security-aware context construction.
Why it matters: Provides concrete, recent evidence and a new benchmark on a critical but understudied aspect of LLM code security, offering actionable insights for developers and researchers.