V2X-Sim

The First Public Large-scale Collaborative Perception Dataset

V2X-Sim, short for vehicle-to-everything simulation, is the first synthetic collaborative perception dataset in autonomous driving developed by AI4CE Lab at NYU and MediaBrain Group at SJTU to facilitate collaborative perception between multiple vehicles and roadside infrastructure. We collect data from both roadside and vehicles when they are presented near the same intersection. With information from both the roadside infrastructure and vehicles, the dataset aims to encourage research on collaborative perception tasks.

Although not collected from the real world, we use highly realistic traffic simulation software to ensure the representativeness of our dataset compared to real-world driving scenarios. To be more exact, the traffic flow of our recording files is managed by CARLA-SUMO co-simulation, and we currently use three town maps from CARLA to increase the diversity of our dataset.

I functioned as an undergrad research assistant in this project and worked closely with Yiming Li, who was the project lead and a PhD student at NYU Tandon.

Check out the website that I developed here for more information and a video demo.

On the left, an example of collaborative perception. Middle, front cameras of four vehicles in a scene. Right, cameras of infrastructure in a scene.
Sensor set up and configuration for both vehicles and infrastructure.

More information on sensors and data format can be found at the official website of our dataset.