PROOF
Perceptive Robustness via Out-of-Distribution Fallback for Legged Locomotion
Coming soonComing soon
TL; DR We boost the robustness of vision-based perceptive locomotion via automatic OOD-triggered proprioceptive behaviour fallback, avoiding policy failures under severe visual degradation, while retaining agile perceptive parkour performance.
Abstract
Vision-based perceptive locomotion has enabled legged robots to achieve agile and terrain-aware locomotion across complex environments. However, the reliability of such systems heavily depends on visual observations, which are inherently susceptible to occlusion, reflective surfaces, and illumination variations, often resulting in catastrophic locomotion failures under degraded perception conditions. In this paper, we present PROOF, a perceptive locomotion framework that enables robust locomotion under severe visual disturbances. Our method adopts a two-stage teacher–student learning paradigm that explicitly models perception reliability: the teacher learns both perception-driven and proprioceptive fallback behaviors, while the student predicts exteroceptive features with uncertainty and suppresses them under out-of-distribution inputs. We evaluate our method on a Unitree Go2 quadruped equipped with a RealSense D435 depth camera in real-world settings. Under normal visual conditions, our method enables the robot to successfully perform high-dynamic parkour maneuvers, including traversing large gaps, elevated platforms, and narrow beam terrains. Under severe visual degradation, such as heavy occlusion and strong illumination changes, the robot continues to maintain stable and robust locomotion performance.
Motivation
Conventional end-to-end perceptive locomotion policies rely heavily on the quality of perception information. Once perception shifts out-of-distribution, degraded or misleading inputs can trigger unsafe behaviors and abrupt policy failures. This sensitivity severely limits their usability in open-world deployment, as shown in the videos below.
End-to-End Perceptive Policy
Fails under visual disturbances
PROOF Policy
Robust to visual disturbances while retaining agile parkour skills
Method
(1) Teacher policy. We first train an elevation-map teacher policy with PPO in simulation, where elevation maps provide accurate and low-cost exteroceptive supervision. During teacher training, the exteroceptive latent is randomly masked on non-parkour terrains, forcing the same policy to acquire both agile perceptive locomotion and proprioceptive fallback behavior.
(2) Student policy. We distill the teacher into a deployable depth-vision student with online DAgger-style supervision. A CNN-GRU estimator predicts the teacher's exteroceptive latent features and learns a perceptive uncertainty through a negative log-likelihood (NLL) loss objective. A learned gate then suppresses unreliable visual features, enabling proprioceptive fallback under OOD depth observations.
Simulation & Real-World Results
Simulation Results
- Curve on the lower left shows the predicted log-variance that reflects terrain uncertainty.
- Curve in the lower middle shows the learned gate.
- Panel on the lower right shows the simulated depth image.
- For parkour terrains (gap, high platform), no OOD disturbance is injected. For non-parkour terrains (stairs, slopes), we randomly inject vision degradation stronger than that used during training to test how the log-variance and gate respond under more severe visual corruption.
- For gap and high platform, the learned gate stays open while the predicted log-variance rises as the robot crosses a gap or climbs a high platform, then decreases during regular flat plane.
- For stair and slope, when strong visual corruption is manually injected, the learned gate switches actively.
We further evaluate robustness under randomized strong visual corruption on non-parkour terrains. Each table entry reports Rvx / S, where Rvx is the mean x-direction velocity tracking reward and S is the traversal success rate over 100 test episodes with 100 parallel environments.
| Method | Rough Flat | Uphill | Downhill | Upstairs | Downstairs |
|---|---|---|---|---|---|
| Baseline 1: w/o teacher mask | 0.28 / 0 | 0.11 / 0 | 0.35 / 0 | 0.15 / 0 | 0.22 / 0 |
| Baseline 2: MSE latent distillation | 0.83 / 96 | 0.76 / 79 | 0.75 / 84 | 0.69 / 73 | 0.71 / 91 |
| PROOF | 0.86 / 100 | 0.79 / 99 | 0.78 / 100 | 0.72 / 95 | 0.93 / 100 |
- Baseline 1 removes exteroceptive masking during teacher training and therefore lacks a reliable proprioceptive fallback target under OOD depth observations.
- Baseline 2 inherits the masked teacher's fallback behavior but removes log-variance and NLL training, using MSE latent distillation instead.
Real-World Videos
Failure Cases
- Although our method generalizes to a large extent in the real world, the learned gate can still fail to switch correctly in some scenarios.
- During continues extreme parkour operation, we observe severe motor heating on the Unitree Go2, which changes motor characteristics and enlarges the sim-to-real gap.
Acknowledgements
We thank our co-authors for their valuable contributions. Special thanks go to Botian Xu for his fruitful and constructive discussions, which substantially benefited this work. We also acknowledge Hao Cheng for the camera mount design.
BibTeX
@inproceedings{wang2026proof,
title={{PROOF}: Perceptive Robustness via Out-of-Distribution Fallback for Legged Locomotion},
author={Wang, Siyuan and Zhu, Dongjie and Lu, Jinxuan and Xu, Wenjun and Wang, Jiankun},
booktitle={Conference on Robot Learning},
year={2026}
}