arXiv:2611.04834v1 Announce Type: new
Abstract: Post-training updates change what large language models (LLMs) can answer, but may leave their answerability judgments misaligned. We study this knowledge-judgment desynchronization through functional feeling of knowing (FoK), which supports selective generation, retrieval triggering, and model routing without implying subjective experience. We propose two trainable channels: a knowledge channel that changes answerability and a FoK channel that judges the current knowledge state. Three stages train them: on-policy supervised fine-tuning (OP-SFT) updates knowledge from corrected on-policy answers; knowledge-delta FoK training (K$\Delta$-FT) adapts judgments across intermediate knowledge states; and FoK-guided policy optimization (FGPO) trains the subsequent answer or explanation. Factual-QA experiments, cross-dataset evaluation, and three sequential updates show that explicit FoK alignment improves judgment quality on updated knowledge. MMLU and ARC-Easy separately assess capability preservation.
arxiv
Feeling of Knowing in Large Language Models
This paper introduces a functional 'feeling of knowing' (FoK) mechanism for LLMs to align answerability judgments with post-training knowledge updates. It proposes two trainable channels—knowledge and FoK—trained via three stages: on-policy supervised fine-tuning, knowledge-delta FoK training, and FoK-guided policy optimization. Experiments on factual QA and sequential updates show improved judgment quality without sacrificing capability on MMLU and ARC-Easy.
Provides a concrete method to address knowledge-judgment desynchronization in LLMs, with experimental validation, useful for model alignment and reliability.
Comments & Questions (0)
No comments yet
Be the first to comment!