SimAug: Learning Robust Representations from Simulation for Trajectory Prediction
Junwei Liang1, Lu Jiang2, Alexander Hauptmann1
1Carnegie Mellon University, 2Google Research
In Proceedings of the European Conference on Computer Vision (ECCV), 2020.
This paper studies the problem of predicting future trajectories of people in unseen cameras of novel scenarios and views. We approach this problem through the real-data-free setting in which the model is trained only on 3D simulation data and applied out-of-the-box to a wide variety of real cameras. We propose a novel approach to learn robust representation through augmenting the simulation training data such that the representation can better generalize to unseen real-world test data. The key idea is to mix the feature of the hardest camera view with the adversarial feature of the original view. We refer to our method as SimAug. We show that SimAug achieves promising results on three real-world benchmarks using zero real training data, and state-of-the-art performance in the Stanford Drone and the VIRAT/ActEV dataset when using in-domain training data.
ECCV'20 Presentation
Multi-view Simulation Data

We propose to learn robust representations only from 3D simulation data that could generalize toreal-world videos captured by unseen cameras.

Release Log