Resimulation

Table of Contents

Introduction top#

Resimulation is a tool for the SVL Simulator with the with the purpose of recreating and generating driving scenarios based on recorded sensor data of simulated and real-world driving sequences.

Among the use-cases of resimulation are:

  • Generating easily repeatable scenarios for testing
  • Recreating interesting cases encountered in real or simulated drives for further study
  • Modifying environmental conditions, including lighting, weather, and road wetness on previously observed driving sequences

The current release provides the ability to resimulate previously recorded simulations (provided that the ego vehicle uses ROS bridge) and to use Apollo to control the ego vehicle during the resimulation. This document will describe how to use this feature.

Resimulation from simulation enables users to record and replay traffic agents' behaviors during a simulation. This allows users to discover, during randomized testing, edge cases and interesting failure cases of their AD system. They can reproduce these exact cases to analyze what went wrong to then improve and retest their system. In addition, by varying environmental conditions, they can create permutations to enable better test coverage.

Package overview top#

Resimulation is delivered as a zip archive with the following directory layout:

lgsvl-resimulation-<release-tag>
├── data
├── docker
│ ├── docker-compose.yml
│ └── lgsvl_resimulate-<release-tag>.tar
├── docs
│ ├── images
│ └── README.md
└── scripts
  ├── functions
  ├── record.sh
  └── resimulate.sh
  • The data directory is the default location for storing rosbag recordings or pickle files.
  • To use a different directory for saving rosbags set the DATA_VOLUME environment variable to the new path:
export DATA_VOLUME=/path/to/some/folder/to/save/rosbags
  • The docker directory contains two docker images saved as tarballs.
  • lgsvl_resimulate-<release-tag>.tar contains the resimulation tool.
  • lgsvl_ros_recorder-<release-tag>.tar is a ROS container which also includes rosbridge_suite and lgsvl_msgs. This image is used for recording rosbags from simulations.
  • The scripts directory holds scripts for loading and launching the docker images.
  • record.sh starts the recording docker container which launches a rosbridge and records incoming messages from the simulator.
  • resimulate.sh starts the resimulation container which will parse recorded rosbags, recreate them as scenarios, and play them in the simulator through the simulator's Python API.

Resimulation from simulation workflow top#

  1. Setup simulation in web UI
  2. Run simulation
  3. Record rosbag of gps and ground truth sensors published by the simulator
  4. Run the resimulation script

Requirements top#

Resimulation has the following dependencies:

  1. SVL Simulator -- the release tag of the simulator should match the release tag of resimulation to ensure compatibility.
  2. Docker
  3. Docker compose

Downloading and using the SVL Simulator top#

The Simulator release can be downloaded as a prebuilt binary from the premium release download page. Make sure to download the simulator binary from the same release as the resimulation release under use.

Download and extract the zip archive at the desired location. In the unzipped directory, run the executable file named simulator to launch the SVL Simulator.The main window of the simulator will open up as seen below. Click on the Open Browser button to launch the simulator Web UI in a web browser. The Web UI controls all aspects of the simulator.

You will need to create an account to login to the Web UI. Once logged in you will be able to configure vehicles sensor arrangements, select maps, and create simulations to run in the simulator.

Installing Docker top#

To install docker, follow this guide on the docker website.

Also complete the first section (titled "Manage Docker as a non-root user") of the post-installation steps for Linux.

Installing Docker Compose top#

To install Docker Compose follow the steps in this guide.

Recording a simulation top#

Setting up the test case top#

Running a simulation requires AssetBundles for the map and vehicle as well as configuration parameters for sensors.

See simulator documentation for how to add a map and how to add a vehicle.

Sensor configuration top#

Currently, recording a simulation requires a using a vehicle in the simulator that uses the ROS bridge type. While other bridge types are not supported for recording, they can be used during playback of a resimulation. At a minimum the following three sensors are required for a recording to be resimulated:

  • The 3D Ground Truth sensor which publishes 3D bounding box data for vehicles and pedestrians within a configurable radius of the sensor
  • The Signal sensor which provides ground truth information on the state of traffic lights facing the ego vehicle within a configurable radius of the sensor
  • The GPS Odometry sensor which publishes localization information in the form of accurate GPS coordinates and orientation.
  • (Optional) The Manual Control sensor enables keyboard control of the ego vehicle. Without a control sensor the ego vehicle will remain stationary.

Sensors and their configuration parameters can be entered into the vehicle configuration window in the Web UI, formatted as JSON.

A simple JSON config including the required sensors and car control would be as follows:

[
  {
    "type": "PerceptionSensor3D",
    "name": "3D Ground Truth",
    "params": {
      "Frequency": 10,
      "Topic": "/simulator/ground_truth/3d_detections",
      "Frame": "gps",
      "MaxDistance": 300
    },
    "transform": {
      "x": 0,
      "y": 0,
      "z": 0,
      "pitch": 0,
      "yaw": 0,
      "roll": 0
    }
  },
  {
    "type": "SignalSensor",
    "name": "Signal",
    "params": {
      "Frequency": 10,
      "Topic": "/simulator/ground_truth/signals",
      "Frame": "gps",
      "MaxDistance": 300
    },
    "transform": {
      "x": 0,
      "y": 0,
      "z": 0,
      "pitch": 0,
      "yaw": 0,
      "roll": 0
    }
  },
  {
    "type": "GpsOdometrySensor",
    "name": "GPS Odometry",
    "params": {
      "Frequency": 50,
      "Topic": "/simulator/sensor/gps",
      "Frame": "gps",
      "IgnoreMapOrigin": false
    },
    "transform": {
      "x": 0,
      "y": 0,
      "z": 0,
      "pitch": 0,
      "yaw": 0,
      "roll": 0
    }
  },
  {
    "type": "KayboardControlSensor",
    "name": "Manual Car Control"
  }
]

The Bridge Type should be set to ROS.

Note the topic names for each sensor; these topics will be recorded and used for resimulation. The frequency of the topics is also important as it will affect the accuracy of resimulation. It is advisable to have a high frequency for GPS Odometry (>= 50). Increasing the publishing frequency for Ground Truth 3D past 10 hz is not advisable as it will not improve accuracy much, but will significantly increase the parsing time for the rosbag.

The transform field determines the location and orientation of the sensor on the ego vehicle. The GPS Odometry sensor, Ground Truth 3D sensor, and Signal sensor must all have the same transform values.

Creating simulations in the Web UI top#

The simulator Web UI provides functionality to create simulations to be run in the simulator. We suggest creating one simulation for recording rosbags and another for running resimulations from the recordings.

Creating the recording simulation top#

  1. On the Simulations page in the Web UI click on Add New in the upper right corner.
  2. Enter a name for the simulation, select the appropriate cluster, and enter a description (optional) and click Next.

  3. On the Test case tab:

    • Set the Runtime Template to Random Traffic.
    • Select the Map from the drop down menu.
    • Select the vehicle created earlier under Vehicle.
    • Set Date and Weather to desired values (or leave at default).
    • Enable Random Traffic and Random Pedestrians and click Next.

    4. On the Autopilot tab:

    • Select any of the AD stacks that use ROS (Autoware AI v1.12 or Autoware AI v1.14).
    • Under Bridge IP enter bridge IP and Port number (localhost:9090 if bridge is running on the same machine)

  4. On the Publish tab, click Make Private or Make Public to publish the simulation (simulations must be published before they can be run).

Creating a simulation to run resimulations top#

Follow these steps to create a second simulation which will be used for running the resimulation from a recording.

  1. On the Simulations page in the Web UI click on Add New in the upper right corner.
  2. On the General tab, enter a name for the simulation, choose a cluster, enter a description and tags (optional) and click Next.
  3. On the Test case tab select API Only as the runtime template and click Next.
  4. There will be no need to specify an Autopilot, click Next and publish on the last tab.

See simulator documentation on the Simulations page for more information on setting up simulations.

Recording the rosbag top#

To record a rosbag we first need to start a test case from the Web UI and then use the script provided in the resimulation package to record it. Currently, recording resimulation only supports recording ROS 1 bags (not ROS 2 or CyberRT records).

  1. In the Web UI, go to the Simulations page and run the resimulation recording simulation by clicking on the Run Simulation button.

  2. Navigate to the unzipped directory of the resimulation package in a terminal window and run the record script. The name of the recording cab be provided as a command-line argument:

    cd scripts
./record.sh sample.bag

  3. Start driving around in the simulator using the arrow keys on the keyboard.

  4. End the recording by pressing Ctrl+C in the terminal window. The recorded rosbag should appear in the data directory.
  5. Stop the simulation from the Web UI by pressing the Stop Simulation button.

Resimulating the recording top#

The resimulation tool parses recordings, processes them, and recreates them as resimulations. The parsed recordings can also be saved as scenario files with a .p extension which can be later used as input to the resimulation tool to avoid re-parsing recordings each time a resimulation is to be run.

To run a resimulation from a rosbag follow these steps:

  • From the simulator Web UI start the API-only simulation which was created for resimulation.

  • Navigate to the unzipped directory of the resimulation package and run the resimulate.sh script:

    cd scripts
./resimulate.sh -m "map name" sample.bag

  • Use the flag -m to specify the map the rosbag was recorded in.

  • For more command line arguments, including spawning and specifying an ego vehicle and specifying weather and road conditions, run ./resimulate.sh --help.
  • Once the resimulation is ready to be run you will be prompted to press the Enter key to start. Pressing Enter will start the playback in the simulator window.

Once a specific recording has been resimulated, the parsed scenario file will be stored in a hidden folder within the data directory. If a cached parse file is available for a recording, the parsing step will be skipped and the parsed scenario file will be used as input for resimulation. If you wish to parse the rosbag recording again, use the --force-parse command-line argument. The cached scenario file can also be retrieved and stored in-place of the rosbag.

By default the ego vehicle is not loaded. To load the ego vehicle the command-line argument --ego will spawn a Lincoln2017MKZ (apollo 5.0) vehicle (if the vehicle is added in My Library) in the appropriate location. To specify the vehicle model, enter its name following the -v flag. For example: ./resimulate.sh -m "map name" --ego -v "Hyundai2018Nexo" sample.bag.

Here are some examples of what a resimulation would look like with different road wetness conditions:

​ Resimulating the original simulation in daylight environment

​ Resimulating the same scenario, with road wetness

​ Resimulating the scenario, with wet road and in night time

Resimulation with Apollo in control of ego vehicle top#

The resimulation package includes functionality to run a resimulation while the ego vehicle is controlled by Apollo. This requires Apollo and the resimulation to be synchronized to ensure that all agents are at the correct state when Apollo begins to drive. For this to happen, an additional sensor plugin has been developed for the simulator which waits for the first control command from Apollo to be received by the simulator before starting the resimulation. Follow these steps to run Apollo with a resimulation:

  • Start the SVL Simulator

  • In the web UI, add a sensor configuration for the ApolloControlSensor to the Apollo vehicle by choosing the vehicle in My Library, choosing Set Up Sensors and adding the following entry to the JSON. Note that if you are not the owner of the vehilce you will need to Clone the vehicle to edit the sensor configuration by clicking on the Clone button. Clones vehicles will need to have a different name than the original vehicle's name so make sure to take note of this name and use it later for resimulation.

    {
  "type" : "ApolloControlSensor",
  "name" : "Apollo Control Sensor",
    "params": {
      "Topic": "/apollo/control"
    }
}

  • From the Test Cases tab, start the API-only test case

  • Start Apollo 5.0

    • Open a terminal, change directory to the apollo-5.0 project (Use the lgsvl fork to ensure compatibility)
    • Start the Apollo container: ./docker/dev_start.sh
  • Enter the Apollo container: ./docker/dev_into.sh
    • If Apollo has not been built with the GPU option, build it using: ./apollo.sh build_gpu
    • Start dreamview and cyber bridge: bootstrap && bridge.sh &
    • Open browser, enter address http://localhost:8888 to open dreamview and select the desired Map and Vehicle in the top right corner.

Note: Apollo modules will be started and stopped by the resimulation script, make sure all modules are off before running resimulation.

  • Start Resimulation

    • Navigate to the unzipped directory of the resimulation package and run the resimulate.sh script:

      cd scripts
./resimulate.sh -m "map name" --apollo --ego -v "your apollo 5.0 vehicle" sample.bag

    • Once the Simulator receives the control message from Apollo 5.0, resimulation will spawn agents and become ready to be run. You will be prompted to press the Enter key to start. Pressing Enter key will start the playback in the simulator window.

Note: resimulation will automatically set the destination in Apollo 5.0 as the nearest lane point of the recorded last position of ego vehicle. If you are recording your own data, make sure the ego vehicle's last pose is at a proper location.

Here is an example of what a resimulation will look like when running with Apollo:

Resimulation command-line arguments top#

Resimulation supports the following command-line arguments:

Command-line argument Description Default
--simulator-api-host, -H IP address of machine running the simulator SIMULATOR_HOST or 127.0.0.1
--simulator-api-port, -P Simulator API port number 8181
--scene, -m Name of scene to load in simulator BorregasAve
--ego Spawn ego vehicle
--vehicle, -v Specifies name of the ego vehicle model Lincoln2017MKZ (Apollo 5.0)
--time-of-day, -t Time of the day as float [0..24]
--fix-time-of-day, -x Fixed time of day
--wetness, -w Road wetness as a float [0..1]
--rain, -r Rain intensity as a float [0..1]
--fog, -f Fog intensity as a float [0..1]
--save-waypoints, -p Path to save waypoints file
--nowait, -W Start simulation without user confirmation
--log-level, -L Specify logging level INFO
--config, -c Path to configuration file
--gen-config Generate configuration file
--force-parse Do not use cached pickle files False
--start, -s Start SEC seconds into the bag file
--duration, -d Play only the next SEC seconds from the bag file
--apollo Drive ego vehicle using Apollo during a resimulation False