Back to AI News
arxiv

e-means clustering claims faster convergence than Lloyd's k-means

Researchers introduce e-means, a mini-batch k-means procedure using an exponential moving average optimization scheme and scaling laws for decay rate and batch size. The paper reports that e-means reaches Lloyd k-means' 25-epoch loss in 1.4 to 2.6 epochs across embedding datasets, yielding claimed 10-45x GPU wall-clock speedups. It also reports partitioning 10 billion DINO embeddings into 16 million clusters in 2.5 epochs. The method is positioned for clustering, retrieval, quantization and data curation workloads.

AI Brief

It presents a concrete, faster clustering method relevant to teams processing large embedding datasets for retrieval or data curation.

arXiv:2611.09501v1 Announce Type: cross Abstract: Lloyd's k-means algorithm is a long-running standard for clustering, with applications in data analysis, retrieval, quantization and data curation. In this paper, we introduce an exponential moving average optimization procedure for k-means with mini-batch updates, powered by modern deep learning machinery and new insights. We also establish scaling laws on how decay rate and batch size impact final performance. This gives us a formulaic rule to fix them for multiple problem sizes and embedding datasets, without the need for any hyperparameter sweep. The resulting clustering algorithm, coined e-means, is much faster than Lloyd k-means and converges to lower losses, frequently even beating k-means++. On a wide array of embeddings datasets, it beats the loss of Lloyd k-means at 25 epochs in 1.4 to 2.6 epochs, which translates to 10-45x GPU wall-clock speedups compared to state-of-the-art k-means implementations. As an example, e-means partitions DINO10B, a dataset of 10 billion DINO embeddings, into 16 million clusters in only 2.5 epochs, reaching a loss below Lloyd's at 25 epochs.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!