Back to AI News
arxiv

Gradient Inversion Exposes Privacy Leakage in Federated LoRA-Tuned Vision Transformers

This paper studies private-data leakage in federated learning when Vision Transformers are fine-tuned with LoRA, where low-rank updates weaken the gradient signal available to the server. It proposes a spatially anchored generative inversion framework that introduces server-prescribed trainable positional embeddings during client fine-tuning, using the resulting positional gradients as layer-wise spatial anchors. The method reports PSNR gains of 10.14 dB, SSIM gain of 0.45 and LPIPS gain of 0.098 on CIFAR-10, plus PSNR gains of 7.83 dB and 11.74 dB on CelebA and CARS in the FedSGD setting. The authors conclude that LoRA-based parameter-efficient fine-tuning does not by itself prevent data leakage in federated learning.

AI Brief

It gives privacy and federated-learning practitioners concrete evidence that LoRA does not automatically protect client data and describes an attack that improves reconstruction quality.

arXiv:2611.09652v1 Announce Type: cross Abstract: Parameter-efficient fine-tuning (PEFT) has been increasingly adopted to fine-tune pretrained models with low computational cost and is therefore also preferred in the federated learning paradigm to reduce computational and communication overhead. In this work, we investigate the sparsely explored problem of the risk of private data leakage in LoRA-adapted Vision Transformers (ViTs), where low-rank updates weaken the gradient signal available to a server with access to individual client updates. We show that quality of reconstruction is limited by two coupled ambiguities: reduced information in LoRA gradients and missing spatial cues from frozen positional embeddings. To address this, we propose a spatially anchored generative inversion framework that incorporates server-prescribed trainable positional embeddings into ViT blocks during fine-tuning at clients. The resulting positional gradients act as layer-wise spatial anchors, providing additional positional cues, while the generative prior keeps reconstructions on a realistic image manifold, reducing duplicate-data and patch-permutation artifacts, {which produce similar gradients as those produced when the target client images are used}. Adding our proposed spatial anchoring to GIAS improves PSNR by 10.14 dB, SSIM by 0.45, and LPIPS by 0.098 on CIFAR-10, as well as PSNR gains of 7.83 and 11.74 dB on high-resolution CelebA and CARS datasets, respectively, in the FedSGD setting. Such gains in reconstructed image quality metrics are also maintained at higher batch sizes and in FedAvg settings. These results indicate that LoRA-based PEFT does not, by itself, prevent data leakage in FL, and carefully incorporated positional embeddings can help in exposing private information.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!