Generative Simulation for Vehicle Motion Planning

Paper | Supplementary | Video

SLEDGE: Synthesizing Simulation Environments for Driving Agents with Generative Models
Kashyap Chitta, Daniel Dauner, and Andreas Geiger
University of Tübingen, Tübingen AI Center

This repo contains SLEDGE, the first generative simulator for vehicle motion planning trained on real-world driving logs. We publicly provide our code for simulation, evaluation, and training (including pre-trained checkpoints).

video.mp4

News

• 27 Mar, 2024: We released our paper on arXiv. Code and models are coming soon, please stay tuned!

Overview

• SLEDGE is a generative simulator, able to synthesize agent bounding boxes and lane graphs, which serve as an initial state for traffic simulation.

• The unique properties of the entities to be generated for SLEDGE, such as their connectivity and variable count per scene, render the naive application of most modern generative models to this task non-trivial.

• Therefore, we introduce a novel raster-to-vector autoencoder (RVAE). It encodes agents and the lane graph into distinct channels in a rasterized latent map. This facilitates both lane-conditioned agent generation and combined generation of lanes and agents with a Diffusion Transformer.

• Using generated entities in SLEDGE enables greater control over the simulation, e.g. long routes, upsampling turns, or increasing traffic density. SLEDGE presents new challenges for planning algorithms, evidenced by failure rates of over 40% for PDM, when tested on hard routes and dense generated traffic.

• Compared to nuPlan, SLEDGE requires 500× less storage to set up (<4GB), making it a more accessible option and helping with democratizing future research in this field.

TODO

• Code release
• Supplementary material & video
• Initial repository & preprint release

Contact

If you have any questions or suggestions, please feel free to open an issue or contact us (daniel.dauner@uni-tuebingen.de).

Citation

If you find SLEDGE useful, please consider giving us a star 🌟 and citing our paper with the following BibTeX entry.

@InProceedings{Chitta2024ARXIV,
  title={SLEDGE: Synthesizing Simulation Environments for Driving Agents with Generative Models},
  author={Chitta, Kashyap and Dauner, Daniel and Geiger, Andreas},
  journal={arXiv},
  volume={2403.17933},
  year={2024}
}

Other resources

• NAVSIM | tuPlan garage | CARLA garage | Survey on E2EAD
• PlanT | KING | TransFuser | NEAT

(back to top)