arXiv:2611.12242v1 Announce Type: cross
Abstract: Continual Test-Time Adaptation (CTTA) methods update models online using a stream of unlabeled data to overcome distribution shifts encountered at test time. However, CTTA is often unstable, as it can reinforce biases in early predictions and increasingly predict only a few classes, resulting in drift or even collapse. In this paper, we identify narrowing pseudo-label class usage as an indicator of CTTA instability and quantify this behavior. Most existing methods have attempted to address this issue by either selecting reliable adaptation samples or re-weighting the loss to encourage diverse pseudo-labels, but they do not explicitly enforce a class marginal over a temporal window. Instead, we propose Sinkhorn Class-Marginal Regularization (SCALAR), a simple plug-and-play module that controls the pseudo-label distribution over a short temporal window. SCALAR works by maintaining recent predictions in a buffer, balancing their class distribution via the Sinkhorn algorithm to match a specified target distribution, and using the balanced soft targets for adaptation. Without changing the model architecture, SCALAR maintains the per-step runtime and increases the peak allocated GPU memory by only 10 MB on ImageNet-C. Our method is particularly effective on skewed, non-stationary data with severe distribution shifts, where SCALAR significantly alleviates drift and collapse when combined with existing stabilization methods based on entropy or diversity.
arxiv
Controlling Temporal Pseudo-Label Marginals for Stable Online Test-Time Adaptation
The paper introduces SCALAR, a plug-and-play module for continual test-time adaptation that stabilizes online model updates by controlling pseudo-label class distributions over a short temporal window. It uses a buffer of recent predictions and the Sinkhorn algorithm to balance soft targets toward a specified distribution, addressing drift and collapse on skewed, non-stationary data. The method adds only about 10 MB peak GPU memory on ImageNet-C and works with existing entropy- or diversity-based stabilization approaches.
Offers a concrete, lightweight stabilization technique for a timely and challenging problem in online test-time adaptation.
Comments & Questions (0)
No comments yet
Be the first to comment!