Latest AI updates

Metamodels Forecast LLM Answer Activations Before First Token

Researchers propose context-answer metamodels that map context activations to forecasts of mean-pooled answer activations, capturing variability across stochastic rollouts. The metamodels achieve high fidelity, scale with training data up to 500,000 prompts, and generalize zero-shot to out-of-domain datasets. Forecasts retain behavior-predictive signal and transfer across models, suggesting scalable pre-generation behavior prediction.