Live intelligence

Latest AI updates

1 updates
Research

RESYNC: Residual Latent-State Synchronization for Deadline-Bound Streaming Inference

RESYNC addresses high-rate streaming inference where a large model may miss per-observation deadlines. The large model runs asynchronously while a fast path updates its stale latent state using newer observations and reuses the unchanged prediction head, learning from frozen-model representations without task labels. Across one real and two synthetic streams and five architecture families, RESYNC outperformed compact or distilled alternatives in all twelve dataset-architecture pairs at anchor ages of 4–12 observations, and in eleven pairs at older ages up to 32. In 1 ms arrival replay, it served at least 99.4% of arrivals within the interval with 0.35–0.62 ms median latency on one CPU thread.

RESYNCstreaming inferencelatency optimization