Back to AI News
arxiv

Vision Foundation Model Outperforms Specialized EEG Encoders When Signals Are Rendered as Images

Researchers introduce SeeEEG, which renders EEG signals as waveform or scalogram images so a frozen vision transformer can serve as an EEG encoder. Only the display parameters are learned, amounting to at most 1,113 trainable parameters per view. With a frozen DINOv3 ViT-B, SeeEEG outperforms seven EEG foundation models across eight clinical, sleep, affective, and brain-computer interface datasets. The learned displays also transfer to V-JEPA 2.1, while replacing DINOv3 with a randomly initialized ViT-B lowers performance by 15.2 points on average.

AI Brief

It presents a concrete and surprising alternative to specialized EEG foundation models, with evidence that visual pretraining and learned signal rendering can improve frozen-feature transfer across multiple clinical and BCI tasks.

arXiv:2611.17031v1 Announce Type: new Abstract: Electroencephalography (EEG) foundation models are pretrained on thousands of hours of recordings, yet their frozen features transfer poorly to downstream tasks, and this transfer does not consistently improve with larger models or more pretraining data. We show that a vision foundation model that has never seen EEG is a stronger frozen EEG encoder than these specialized models, once the signals are rendered as images that a vision transformer can read. Our method, SeeEEG, keeps the vision backbone frozen and learns only how EEG is displayed to it. Each channel occupies a dedicated row of patches and is drawn either as a waveform at a fixed gain or as a scalogram whose frequency range spans the patch height, so that task-relevant patterns fall within individual tokens. The montage, filtering, and contrast parameters of this display are initialized from a standard clinical montage and optimized with the task loss through the frozen backbone, which amounts to at most 1,113 trainable parameters per view. We compare SeeEEG with seven EEG foundation models on eight clinical, sleep, affective, and brain-computer interface datasets, with every encoder frozen and paired with identical readout heads. With a frozen DINOv3 ViT-B, SeeEEG outperforms every EEG foundation model in balanced accuracy on all eight datasets and reaches a mean rank of 1.3 in Cohen kappa or AUROC, compared with 2.9 for the strongest baseline, REVE. The learned displays transfer without further optimization to the video encoder V-JEPA 2.1, which outperforms every EEG foundation model on both metrics on six of the eight datasets. Replacing DINOv3 with a randomly initialized ViT-B lowers the score by 15.2 points on average, so the ability to read rendered EEG comes from visual pretraining.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!