Running Apollo 6.0 with SVL Simulator

These instructions have been tested with the r6.0.0 branch from the Apollo repository (latest commit at the time of writing).

Apollo 6.0 introduced major changes to the perception stack and as a result camera perception is not yet working, and LiDAR perception is unstable. Because of this, this guide will focus on using the modular testing feature of the SVL Simulator to simulate perception instead of using Apollo's perception module.

For those who used our fork of Apollo 5.0 before: please note the new step to select the correct setup mode in Dreamview.

Table of Contents

Getting Started top#

This guide outlines the steps required to setup Apollo for use with the SVL Simulator. If you have not already set up the simulator, please do so first by following the instructions in Installing the SVL Simulator.

Prerequisites top#

  • Ubuntu 18.04 or later
  • NVIDIA graphics card (required for perception and prediction)
    • NVIDIA proprietary driver (>=410.48) must be installed

Setup top#

Docker top#

Apollo is designed to run out of docker containers. The image will mount this repository as a volume so the image will not need to be rebuilt each time a modification is made.

Installing Docker CE top#

To install Docker CE please refer to the official documentation.

NOTE Apollo does not work if the docker is started with sudo. We suggest following through with the post installation steps.

Installing NVIDIA Docker top#

Before installing nvidia-docker make sure that you have an appropriate NVIDIA driver installed. To test if nvidia drivers are properly installed enter nvidia-smi in a terminal. If the drivers are installed properly an output similar to the following should appear.

    +-----------------------------------------------------------------------------+
    | NVIDIA-SMI 440.64       Driver Version: 440.64       CUDA Version: 10.2     |
    |-------------------------------+----------------------+----------------------+
    | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
    | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
    |===============================+======================+======================|
    |   0  GeForce GTX 1080    Off  | 00000000:01:00.0  On |                  N/A |
    | 27%   29C    P8     7W / 180W |    579MiB /  8117MiB |      1%      Default |
    +-------------------------------+----------------------+----------------------+

    +-----------------------------------------------------------------------------+
    | Processes:                                                       GPU Memory |
    |  GPU       PID   Type   Process name                             Usage      |
    |=============================================================================|
    |    0      1745      G   /usr/lib/xorg/Xorg                            40MiB |
    |    0      1862      G   /usr/bin/gnome-shell                          49MiB |
    |    0      4409      G   /usr/lib/xorg/Xorg                           223MiB |
    |    0      4545      G   /usr/bin/gnome-shell                         140MiB |
    |    0      4962      G   ...uest-channel-token=10798087356903621100    27MiB |
    |    0      9570      G   /proc/self/exe                                50MiB |
    |    0     17619      G   ...uest-channel-token=14399957398263092148    40MiB |
    +-----------------------------------------------------------------------------+

The installation steps for nvidia-docker are available at the official repo.

Cloning the Repository top#

Clone the Apollo using the following command:

git clone https://github.com/ApolloAuto/apollo

Checkout the r6.0.0 branch:

cd apollo
git checkout r6.0.0

Do not checkout the v6.0.0 tag as it does not include the latest changes.

Building Apollo and the bridge top#

Now everything should be in place to build Apollo. Apollo must be built from the container. To launch the container navigate to the directory where the repository was cloned and enter:

./docker/scripts/dev_start.sh

This should launch the container and mount a few volumes. It could take a few minutes to pull the latest volumes on the first run.

To get into the container:

./docker/scripts/dev_into.sh

Build Apollo (optimized, not debug, with GPU support):

./apollo.sh build_opt_gpu

NOTE The Apollo build may fail on machines with less than 1GB of RAM per CPU core due to aggressive parallelization in the build, as discussed in Apollo issue 7719.

If the build fails, you can try re-start it until it succeeds, but if it continues to fail (especially when running the linker) then you'll need to address the low memory situation by either adding more memory to your build machine or enabling or increasing available swap space. If your Apollo build is crashing on a 16GB machine with little or no swap, try setting it to 16GB.

Adding a Vehicle to Apollo top#

  • In order to run Apollo with the simulator, the calibration files for the vehicle you use in the simulator must be available in Apollo for perception module to work correctly.
  • Currently, only calibration files for Lincoln 2017 MKZ are included in the latest Apollo. More calibration files may be added later.
  • To add a new vehicle to Apollo:
    • Create a new folder with the new vehicle's name under APOLLO_ROOT/modules/calibration/data.
    • Copy all calibration files of Lincoln 2017 MKZ to the folder.
    • Update vehicle parameters in vehicle_param.pb.txt if your new vehicle is different from Lincoln 2017 MKZ.
    • Update LiDAR calibration files in velodyne_params folder. If you have different configuration for LiDAR with respect to novatel, you need to update LiDAR's height and extrinsics.
    • Update camera calibration files in camera_params folder. You need to update camera intrinsics and extrinsics for your own camera and sensor configurations.
    • You may also need to update other sensor calibration files if you are using them on your vehicle.
  • After you add the new vehicle, you need to restart Dreamview to refresh the vehicle list.
    • Inside Apollo docker:
      • bootstrap_lgsvl.sh stop && bootstrap_lgsvl.sh

Adding an HD Map to Apollo top#

  • An HD map in Apollo format corresponding to the environment you use in the simulator must be available in Apollo for modules like routing, planning, etc. to work.

  • Currently, only HD map files for Borregas Ave are included in the the latest Apollo at this time. More HD maps may be added later.

  • You can also download additional maps from here, and manually add them to /apollo/modules/map/data/.

  • Or you can export your own map file following Export Map Annotations, then generate routing_map and sim_map following these instructions.

  • In our Apollo 5.0 fork, we have a script generate_map.sh (link) to help users generate map files. You can copy that script to APOLLO_ROOT/scripts/ folder to use it. For details, please follow here.

  • After you add the new map, you need to restart Dreamview to refresh the map list.

    • Inside Apollo docker:
      • bootstrap_lgsvl.sh stop && bootstrap_lgsvl.sh

Alternatively, you can analyze top while building, and decide how many jobs to enable to avoid running out of memory; then set a specific value in JOB_ARG, e.g. --jobs=6.

Launching Apollo alongside the Simulator top#

Here we only describe only a simple case of driving from point A to point B using Apollo and the simulator.

To launch Apollo, first launch and enter a container as described in the previous steps.

  • To start Apollo:

    Note: you may receive errors about Dreamview not being built if you do not run the script from the /apollo directory.

./scripts/bootstrap_lgsvl.sh
  • Launch bridge (inside docker container):
./scripts/bridge.sh
  • Run the SVL Simulator outside of docker. See instructions for Running the Simulator
    • Create a Simulation the BorregasAve map and Lincoln2017MKZ (Apollo 5.0) vehicle
    • Enter localhost:9090 as the Bridge Connection String
    • (Optional) Enable Traffic and Pedestrians
    • (Optional) Set the Time of Day and weather settings
    • Submit the Simulation
    • Select the created Simulation and click "Play"

  • Open Apollo Dreamview in a browser by navigating to: localhost:8888
    • NEW for Apollo Master: Select the Mkz Lgsvl setup mode, from the menu to the left of the vehicle menu.
    • Select the Lincoln2017MKZ LGSVL vehicle and Borregas Ave map in the top right corner.
    • Open the Module Controller tap (on the left bar).
    • Enable Localization, Transform, Perception, Planning, Prediction, Routing, and Control.
    • Navigate to the Route Editing tab.
    • Select a destination by clicking on a lane-line and clicking Submit Route.
    • Watch the vehicle navigate to the destination.

  • To stop the docker container run the dev_start.sh stop script in apollo/docker/scripts in a new terminal (not in the docker container).

  • If you are using ufw, it is easiest to completely disable the firewall to allow connections sudo ufw disable

    • If that is not possible, add the following rules:
      • These are required even if running the simulator and Apollo on the same machine
      • sudo ufw allow 8888
      • sudo ufw allow 9090

Adding a Vehicle top#

Only calibration files for Lincoln 2017 MKZ is included in the latest Apollo at this time. More calibration files map be added later.

Adding an HD Map top#

Only HD map files for Borregas Ave is included in the the latest Apollo at this time. More HD maps may be added later.

You can also download additional maps from here, and manually add them to /apollo/modules/map/data/.

Or you can export your own map from our map annotation tool, name it base_map.bin and then generate routing_map and sim_map following these instructions.