From LoRA to Language: Recovering Skill Documents from Adapters
The paper investigates whether textual agent skill documents can be reconstructed from LoRA adapters and their frozen base models. A small writer trained on top of the adapted model recovers 39-44% of key instructions from held-out skills, versus 1.3-1.5% when using another skill's adapter, and restores 85-91% of the original task gain on 41 external skills. For adapters trained on behavior alone, an agent compares base and adapted model outputs to write a skill, recovering 58-71% of a LoRA's gain on ALFWorld and transferring to Llama-3.1-8B. The strongest results occur when adapter training stored the document itself.