RayViT: Ray-Conditioned Visual Representations for Viewpoint-Robust Imitation Learning

arXiv:2607.29622v1 Announce Type: new Abstract: Visual imitation learning enables robots to acquire visuomotor skills directly from images, yet RGB observations lack explicit geometric cues, making learned policies brittle to camera perturbations. To address this, we propose \textbf{Ray-conditioned Vision Transformer Encoder (RayViT)}, a lightweight architecture that injects camera geometry into pretrained ViT backbones. RayViT represents camera geometry as a Pl\"ucker ray map, patchifies it int...