Research
SALA Aligns Latent Actions Across Human and Robot Video for Embodied Foundation Models
SALA introduces semantically aligned latent actions to improve cross-embodiment learning for embodied foundation models. Without paired data, it uses language descriptions of unpaired human and robot clips to estimate semantic similarities and supervise correspondence between human and robot latent actions. Soft neighborhood structures guide human latent actions toward semantically related robot latents, while robot action reconstruction grounds the shared latent space in control. The method claims dual-axis scaling: more general human video for pre-training reduces robot action prediction error, and more task-specific human video for post-training improves real-world robot task success rates.
embodied foundation modelslatent action modelscross-embodiment learning