arXiv:2610.41866v1 Announce Type: cross
Abstract: Job applicants increasingly use large language models (LLMs) to write cover letters, while employers deploy LLMs to screen them. In this all-AI setting, we ask whether the \emph{choice of model} affects hiring outcomes independently of candidate qualifications. We introduce a multi-model auditing framework pairing eleven LLM writer agents with nine API-hosted evaluator agents across ten job descriptions and fifty candidates per job. Four findings emerge. First, we identify a generative preference hierarchy: a consistent global ranking of writer models by evaluator acceptance, in which locally hosted models are systematically disadvantaged regardless of their license, compounded by self-preference bias where evaluators favour their own outputs. Second, evaluator models differ substantially in how much bias they introduce based on applicants' tool choice, offering actionable guidance for fairer screening design. Third, this hierarchy has measurable competitive consequences: in the most adverse pairing, approximately two (1.8) out of twenty-five qualified candidates are filtered out purely due to model mismatch. Fourth, a common mechanism explains the patterns above: writer preference and self-preference trace to a combination of how much each writer leans on specific stylistic cues and how strongly evaluators, often unanimously, reward or penalise those same cues. A further check finds little evidence that an unmeasured "writer fingerprint" remains once these same features are accounted for. We discuss implications for algorithmic fairness in employment, regulation of AI-assisted screening, and the design of auditing frameworks that can be reapplied as the model landscape evolves.
arxiv
AI evaluators favor job applications written by certain AI models, independent of candidate quality
A new study introduces a multi-model auditing framework that pairs eleven LLM writer agents with nine evaluator agents across ten job descriptions and fifty candidates per job. It finds a consistent generative preference hierarchy where locally hosted models are systematically disadvantaged, compounded by self-preference bias. In the most adverse pairing, about 1.8 out of 25 qualified candidates are filtered out purely due to model mismatch. The study offers actionable guidance for fairer screening design and discusses implications for algorithmic fairness and regulation.
This research provides concrete evidence of AI model bias in hiring, a high-impact area with practical implications for fairness and regulation.
Comments & Questions (0)
No comments yet
Be the first to comment!