Back to AI News
arxiv

Controlling Temporal Pseudo-Label Marginals for Stable Online Test-Time Adaptation

The paper introduces SCALAR, a plug-and-play module for continual test-time adaptation that stabilizes online model updates by controlling pseudo-label class distributions over a short temporal window. It uses a buffer of recent predictions and the Sinkhorn algorithm to balance soft targets toward a specified distribution, addressing drift and collapse on skewed, non-stationary data. The method adds only about 10 MB peak GPU memory on ImageNet-C and works with existing entropy- or diversity-based stabilization approaches.

AI Brief

Offers a concrete, lightweight stabilization technique for a timely and challenging problem in online test-time adaptation.

arXiv:2611.12242v1 Announce Type: cross Abstract: Continual Test-Time Adaptation (CTTA) methods update models online using a stream of unlabeled data to overcome distribution shifts encountered at test time. However, CTTA is often unstable, as it can reinforce biases in early predictions and increasingly predict only a few classes, resulting in drift or even collapse. In this paper, we identify narrowing pseudo-label class usage as an indicator of CTTA instability and quantify this behavior. Most existing methods have attempted to address this issue by either selecting reliable adaptation samples or re-weighting the loss to encourage diverse pseudo-labels, but they do not explicitly enforce a class marginal over a temporal window. Instead, we propose Sinkhorn Class-Marginal Regularization (SCALAR), a simple plug-and-play module that controls the pseudo-label distribution over a short temporal window. SCALAR works by maintaining recent predictions in a buffer, balancing their class distribution via the Sinkhorn algorithm to match a specified target distribution, and using the balanced soft targets for adaptation. Without changing the model architecture, SCALAR maintains the per-step runtime and increases the peak allocated GPU memory by only 10 MB on ImageNet-C. Our method is particularly effective on skewed, non-stationary data with severe distribution shifts, where SCALAR significantly alleviates drift and collapse when combined with existing stabilization methods based on entropy or diversity.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!