ZeroVLN: Learning to Navigate without Supervised Fine-Tuning of the VLM Brain
ZeroVLN is a slow-fast vision-language navigation framework that avoids navigation-domain supervised fine-tuning of the VLM brain. Instead of predicting geometric goals, the VLM uses a semantic tool-calling interface to send movement decisions in natural language to an action expert. The action expert is first adapted to execute these requests, after which closed-loop reinforcement learning jointly improves movement decisions and continuous execution. The authors report that ZeroVLN outperforms representative SFT-based hierarchical VLN methods on VLN-CE benchmarks while only minimally changing the VLM brain and largely preserving its general capabilities.