E2ES

End-to-End UAV Simulation for Visual SLAM and Navigation

Introduction

This Kit provides an end to end simulation solution for MAV visual SLAM (or path planning) researches.
In this work, several features are added to the default PX4 Gazebo Simulator:
-A realsense D435 model (based on realsense_gazebo_plugin)
-Modified IRIS MAV model
-Several structured/unstructured simulation world

The work has been verified on Ubuntu 18.04/20.04 + ROS melodic/noetic environment
Also, if you want to save time, we strongly suggest you to use docker image! The link is right here.

Demo Video

manual control	Click and fly navigation

Usage

If you still insist not to use docker, here is the installation instructions:

Install pre-requisites (ubuntu 18.04):

sudo apt-get install ros-melodic-mavros ros-melodic-mavros-extras ros-melodic-mavros-msgs libncurses5-dev python3-pip libgstreamer1.0-dev python-jinja2 python-pip python-testresources libignition-math2 libgazebo9-dev
pip3 install --user empy toml numpy packaging jinja2
pip install numpy toml empy packaging
sudo apt upgrade libignition-math2
wget https://raw.githubusercontent.com/mavlink/mavros/master/mavros/scripts/install_geographiclib_datasets.sh
sudo bash ./install_geographiclib_datasets.sh

Install pre-requisites (ubuntu 20.04):

sudo apt-get install ros-noetic-mavros ros-noetic-mavros-extras ros-noetic-mavros-msgs libncurses5-dev python3-pip libgstreamer1.0-dev python-jinja2 python3-pip python3-testresources libignition-math4 libgazebo11-dev
pip3 install --user empy toml numpy packaging jinja2
pip3 install numpy toml empy packaging
sudo apt upgrade libignition-math4
wget https://raw.githubusercontent.com/mavlink/mavros/master/mavros/scripts/install_geographiclib_datasets.sh
sudo bash ./install_geographiclib_datasets.sh
# neglect all gazebo9-related warning

Download and install the PX4 (1.11.0)

git clone https://github.com/PX4/PX4-Autopilot.git
cd PX4-Autopilot/
git checkout 71db090
git submodule sync --recursive
git submodule update --init --recursive
bash ./Tools/setup/ubuntu.sh
sudo apt upgrade libignition-math2 #(libignition-math4 for noetic)
make px4_sitl_default gazebo

Clone this repository to catkin src folder say: ~/catkin_ws/src

cd ~/catkin_ws/src
git clone https://github.com/HKPolyU-UAV/E2ES.git

Install 3rd Part Library

cd ~/catkin_ws/src/e2es/3rdPartLib
sudo ./install3rdPartLib.sh
sudo ln -s /usr/include/eigen3/Eigen /usr/include/Eigen # your Sophus was installed with ros-noetic-sophus (not source)

Compile

cd ~/catkin_ws/
catkin_make

By default ROS_WORK_SPACE_PATH is ~/catkin_ws and PX4_Firmware_PATH is ~/PX4-Autopilot
If Not, Edit the path in E2ES/sim.sh script(Line 1 to 4)

export GAZEBO_RESOURCE_PATH=$GAZEBO_RESOURCE_PATH:~/catkin_ws/src/e2es/gazebo
export GAZEBO_MODEL_PATH=$GAZEBO_MODEL_PATH:~/catkin_ws/src/e2es/gazebo/models
export GAZEBO_PLUGIN_PATH=$GAZEBO_PLUGIN_PATH:~/catkin_ws/devel/lib
cd ~/PX4-Autopilot

to

export GAZEBO_RESOURCE_PATH=$GAZEBO_RESOURCE_PATH:ROS_WORK_SPACE_PATH/src/e2es/gazebo
export GAZEBO_MODEL_PATH=$GAZEBO_MODEL_PATH:ROS_WORK_SPACE_PATH/src/e2es/gazebo/models
export GAZEBO_PLUGIN_PATH=$GAZEBO_PLUGIN_PATH:ROS_WORK_SPACE_PATH/devel/lib
cd PX4_Firmware_PATH

Run the simulator

roscd e2es
./sim.sh

Using keyboard to control the MAV in simulator

roslaunch e2es keyboard_ctr.launch

Running End-To-End simulation

1. SLAM

A.

You can use the FLVIS + MLMapping kit combination to run SLAM. Please refer to the respective repo, FLVIS & MLMap to build them. Also make sure that you safisfy their prerequisites.

The launch file for E2ES are here_FLVIS & here_MAP. Hence, do

roslaunch flvis e2es.launch # for VIO
roslaunch mlmapping mlmapping_e2es.launch # for Mapping

B.

The topics there you might be interested are as follows:

Topic	Topic Type	Topic Name	Hz
Pose (GT)	geometry_msgs/PoseStamped	/gt_iris_base_link_imu	50
Pose (VIO)	geometry_msgs/PoseStamped	/mavros/vision_pose/pose	200
Odom (VIO)	nav_msgs/Odometry	/imu_odom	200
IMU (PX4)	sensor_msgs/IMU	/mavros/imu/data	50
IMU (Gazebo)	sensor_msgs/IMU	/iris/imu/data	200
RAW PCL	sensor_msgs/Image	/camera/depth_aligned_to_color_and_infra1/image_raw	30
MAP	sensor_msgs/PointCloud2	/global_map	20

You can modify the launch file, and

2. Autonomous Navigation (deprecated)

You can use the FLVIS-glmapping-FUXI(localization-mapping-planning kits) navigation system.
The first step is to install FLVIS, glmapping and FUXI accordinly.
Then start the simulator localization mapping and planning kit in sequence.

roscd e2es
./sim.sh

# flvis
roslaunch flvis e2es.launch # under flvis package

# fuxi
## please refer to fuxi repo

Acknowledgement

This work are based on PX4 Projcet and realsense_gazebo_plugin

Maintainer

Patrick Lo, AIRo-Lab, RCUAS, PolyU
Shengyang Chen, Dept.ME, PolyU