AirSim provides a feature that constructs ground truth voxel grids of the world directly from Unreal Engine. A voxel grid is a representation of the occupancy of a given world/map, by discretizing into cells of a certain size; and recording a voxel if that particular location is occupied.

The logic for constructing the voxel grid is in WorldSimApi.cpp->createVoxelGrid(). For now, the assumption is that the voxel grid is a cube - and the API call from Python is of the structure:

simCreateVoxelGrid(self, position, x, y, z, res, of)

position (Vector3r): Position around which voxel grid is centered in meters
ncells_x, ncells_y, ncells_z (int): Number of cells per dimension
res (float): Cell width in meters
of (str): Name of output file to save voxel grid

Within createVoxelGrid(), the main Unreal Engine function that returns occupancy is OverlapMultiByChannel.

OverlapBlockingTestByChannel(position, rotation, ECollisionChannel, FCollisionShape, params);

This function is called on the positions of all the 'cells' we wish to discretize the map into, and the returned occupancy result is collected into an array voxel_grid_. The indexing of the cell occupancy values follows the convention of the binvox format.

for (float i = 0; i < ncells_x; i++) {
    for (float k = 0; k < ncells_z; k++) {
        for (float j = 0; j < ncells_y; j++) {
            TArray<FOverlapResult> overlapping_objects;
            int idx = i + ncells_x * (k + ncells_z * j);
            FVector vposition = FVector((i - ncells_x / 2) * scale_cm, (j - ncells_y / 2) * scale_cm, (k - ncells_z / 2) * scale_cm) + position_in_UE_frame;
            bool overlap_result = simmode_->GetWorld()->OverlapMultiByChannel(overlapping_objects, vposition, FQuat::Identity, ECollisionChannel::ECC_Pawn, FCollisionShape::MakeBox(FVector(scale_cm / 2)), params);
            if (overlap_result) {
                if (use_segmentation) {
                    int segmentationID = overlapping_objects[0].GetComponent()->CustomDepthStencilValue;
                    voxel_grid_[idx] = segmentationID;
                }
                else {
                    voxel_grid_[idx] = 1;
                }
            }
            else {
                voxel_grid_[idx] = 0;
            }
        }
    }
}

The occupancy of the map is calculated iteratively over all discretized cells, which can make it an intensive operation depending on the resolution of the cells, and the total size of the area being measured. If the user's map of interest does not change much, it is possible to run the voxel grid operation once on this map, and save the voxel grid and reuse it. For performance, or with dynamic environments, we recommend running the voxel grid generation for a small area around the robot; and subsequently use it for local planning purposes.

The voxel grids are stored in the binvox format which can then be converted by the user into an octomap .bt or any other relevant, desired format. Subsequently, these voxel grids/octomaps can be used within mapping/planning. One nifty little utility to visualize a created binvox files is viewvox. Similarly, binvox2bt can convert the binvox to an octomap file.

Example voxel grid in Blocks:#

Blocks voxel grid converted to Octomap format (visualized in rviz):#

As an example, a voxel grid can be constructed as follows, once the Blocks environment is up and running:

import airsim
c = airsim.VehicleClient()
center = airsim.Vector3r(0, 0, 0)
output_path = os.path.join(os.getcwd(), "map.binvox")
c.simCreateVoxelGrid(center, 100, 100, 100, 0.5, output_path)

And visualized through viewvox map.binvox.

Segmented voxel grid#

There is a variation of the voxel grid that distinguishes the object occupied by the cube by saving its ID. The object ID is the same as the segmentation ID used by the camera. See the documentation on segmentation to know how to obtain the IDs of each object, and see this documentation on how to get the name of the meshes(image_apis#how-to-find-mesh-names).