Back to AI News
arxiv

DrawOffer: Evidence-Grounded Longitudinal User-State Modeling for Career Support

DrawOffer is a system that separates language-model observation extraction from deterministic inference over source-linked records to maintain uncertain ability estimates and revisable issues in career-support assistants. Evaluated on 102 fictional personas with 580 scored checkpoints, it passes 169/210 checkpoints on test personas with issue precision of 0.875 and recall of 0.636. Ablation studies show that removing answer-evidence requirements reduces passes and precision, highlighting the importance of evidence sufficiency. The work is an exploratory systems contribution with limitations in generalization and reproducibility.

AI Brief

Offers a concrete, evidence-grounded approach to improving reliability of AI career assistants, with clear evaluation metrics and ablation studies.

arXiv:2611.02793v1 Announce Type: new Abstract: Career-support assistants must distinguish user claims from evidence of current ability and withhold judgments when evidence is insufficient. DrawOffer separates language-model observation extraction from deterministic inference over source-linked records to maintain uncertain ability estimates, experience assessments, and revisable issues. We evaluate 102 fictional personas with 580 scored checkpoints, a historical full-context and Mem0 comparison, and three auxiliary knowledge-tracing datasets. On 37 test personas, DrawOffer passes 169/210 checkpoints; checkpoint-defined issue precision and recall are 0.875 and 0.636. Across three development simulations, 87.0% of checkpoint verdicts agree, including persistent failures. A cache-only development ablation replacing answer-evidence requirements with confidence-only settlement reduces majority-vote passes from 306 to 293 of 370 and issue precision from 0.927 to 0.725. Equalizing positive source weights improves predictive metrics without changing total passes; removing decay gives mixed results. These findings support an exploratory systems contribution centered on evidence sufficiency. Developer-authored scenarios, unmatched baseline models, historical test exposure, and incomplete public artifacts limit generalization and independent reproduction.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!