Token Identity Brier Flow Improves Continued Training of Continuous Language Flows
Researchers propose Token Identity Brier Flow (TIBF), which applies squared supervision directly to token identities when continuing pretrained language flows, combining network scores with token-matching scores under a Gaussian observation model and using the Brier loss for token probabilities. They show that squared error in predicted token probabilities bounds the squared errors of continuous denoising predictions and flow velocities, and provide a risk decomposition separating observation uncertainty from posterior fitting error. Under matched continuation budgets with frozen embeddings, TIBF achieves Gen.PPL values of 33.96 on OWT and 81.17 on LM1B, outperforming continuous-MSE and cross-entropy continuation controls and attaining the lowest Gen.PPL at every tested sampling budget on both datasets.