Welcome to AirSimExt (aka AirSimExtensions)#

AirSimExt is a simulator for drones, cars and more, built on Unreal Engine 5.2. It is open source, cross platform, and supports software-in-the-loop simulation with popular flight controllers such as PX4 and ArduPilot and hardware-in-loop with PX4 for physically and visually realistic simulations. It is developed as an Unreal plugin and can simply be dropped into any Unreal environment. We also have an experimental release for a Unity plugin.

Our goal is to develop AirSimExt as a platform for AI research to experiment with deep learning, computer vision and reinforcement learning algorithms for autonomous vehicles as it relates to the DARPA ANSR program. For this purpose, AirSimExt also exposes APIs to retrieve data and control vehicles in a platform-independent way.

Demos:

Using AirSim#

Getting Started#

AirSimExt is a fork of AirSim OSS, developed in conjunction with STR for DARPA ANSR.

The proper way to run AirSimExt is to build/run the latest docker container. Instructions can be found here.

There are two ways to use AirSimExt: As a product or for development purposes. To use AirSimExt as a product, download and install the AirSimExt binary for the AirSim release page. To develop on AirSim, follow the "Build It" instructions.

Linux
Download binaries Build it

For more details on working with the biaries, see the precompiled binaries document. For a broader overview of AirSim instructions and uses, view our detailed documentation.

Using AirSim Simulator#

Manual Drive#

If you have remote control (RC) as shown below, you can manually control the drone in the simulator. For cars, you will use keyboard arrow keys to drive manually.

RC Documentation

Gathering Training Data#

There are two different ways to generate training data from for deep learning: 1. The fastest and easiest way is to simply press the 'record' button in the lower right corner -- this will record poses and images for each frame. The data logging code is pretty simple and can be easily modified (see below). 1. The more comprehensive way to generate training data is by accessing the APIs, which grant the user full control of the how, what, where and when data is logged.

Computer Vision Mode#

Computer Vision Mode allows the user to use AirSim without vehicles or physics. The keyboard is used to navigate the scene. APIs enable the user to adjust available cameras into arbitrary poses, and collect images such as depth, disparity, surface normals or object segmentation.

More details

Weather Effects#

You can control the weather in one of two ways: 1. Press F10 to see available weather effects. 1. Weather control using APIs

For more available options, press F1.

Developing with AirSimExt#

Programmatic control#

AirSimExtensions exposes APIs so you can interact with the vehicle in the simulation programmatically. You can use these APIs to retrieve images, get state, control the vehicle and so on. The APIs are exposed through the RPC, and are accessible via a variety of languages, including C++, Python, C# and Java.

These APIs are also available as part of a separate, independent cross-platform library, so you can deploy them on a companion computer on your vehicle. This way you can write and test your code in the simulator, and later execute it on the real vehicles. Transfer learning and related research is one of our focus areas.

Note that you can use SimMode setting to specify the default vehicle or the new ComputerVision mode so you don't get prompted each time you start AirSimExtensions.

More details

Using ROS on AirSimExtensions#

ROS Documentation

Tutorials#

• Video - Setting up AirSim with Pixhawk Tutorial by Chris Lovett
• Video - Using AirSim with Pixhawk Tutorial by Chris Lovett
• Video - Using off-the-self environments with AirSim by Jim Piavis
• Webinar - Harnessing high-fidelity simulation for autonomous systems by Sai Vemprala
• Reinforcement Learning with AirSim by Ashish Kapoor
• The Autonomous Driving Cookbook by Microsoft Deep Learning and Robotics Garage Chapter
• Using TensorFlow for simple collision avoidance by Simon Levy and WLU team

Participate#

Paper#

More technical details are available in AirSim paper (FSR 2017 Conference). Please cite this as:

@inproceedings{airsim2017fsr,
  author = {Shital Shah and Debadeepta Dey and Chris Lovett and Ashish Kapoor},
  title = {AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles},
  year = {2017},
  booktitle = {Field and Service Robotics},
  eprint = {arXiv:1705.05065},
  url = {https://arxiv.org/abs/1705.05065}
}

Contribute#

Please take a look at open issues if you are looking for areas to contribute to.

• More on AirSim design
• More on code structure
• Contribution Guidelines
• This repository is utilizing Azure Boards. See the status below:
  

Who is Using AirSim?#

We are maintaining a list of a few projects, people and groups that we are aware of who are using AirSim. If you would like to be featured in this list please make a request here.

Contact#

Join our GitHub Discussions group to stay up to date or ask any questions.

We also have an AirSim group on Facebook.

What's New#

• Cinematographic Camera
• ROS2 wrapper
• API to list all assets
• movetoGPS API
• Optical flow camera
• simSetKinematics API
• Dynamically set object textures from existing UE material or texture PNG
• Ability to spawn/destroy lights and control light parameters
• Support for multiple drones in Unity
• Control manual camera speed through the keyboard

For complete list of changes, view our Changelog

FAQ#

If you run into problems, check the FAQ and feel free to post issues in the AirSim repository.

Code of Conduct#

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

License#

This project is released under the MIT License. Please review the License file for more details.