Research
Fairness Audits Expire: LLM Bias Shifts Across Model Versions, 13-Month Study Finds
Researchers repeated a fairness audit of provider-served language models in a venture screening task three times over 13 months, totaling 248,400 API calls across five providers. The same model called under the same identifier a year later showed a significantly different bias pattern, and successor models carried the same bias as their predecessors but in different size and form. Provider documentation showed models are changed or retired at the providers' discretion, with notice periods ranging from zero to six months. The authors propose "evidence anchoring" theory and recommend auditing model classes rather than single models, version records for responses, and re-audits triggered by model changes.
fairness auditingLLM biasAI regulation