Spatial awareness
The Spatial Awareness system provides real-world environmental awareness in mixed reality applications. When introduced on Microsoft HoloLens, Spatial Awareness provided a collection of meshes, representing the geometry of the environment, which allowed for compelling interactions between holograms and the real-world.
Note
At this time, the Mixed Reality Toolkit does not ship with Spatial Understanding algorithms as originally packaged in the HoloToolkit. Spatial Understanding generally involves transforming Spatial Mesh data to create simplified and/or grouped Mesh data such as planes, walls, floors, ceilings, etc.
Getting started
Adding support for Spatial Awareness requires two key components of the Mixed Reality Toolkit: the Spatial Awareness system and a supported platform provider.
- Enable the Spatial Awareness system
- Register and configure one or more spatial observers to provide mesh data
- Build and deploy to a platform that supports Spatial Awareness
Enable the spatial awareness system
The Spatial Awareness system is managed by the MixedRealityToolkit object (or another service registrar component). Follow the steps below to enable or disable the Spatial Awareness system in the MixedRealityToolkit profile.
The Mixed Reality Toolkit ships with a few default pre-configured profiles. Some of these have the Spatial Awareness system enabled OR disabled by default. The intent of this pre-configuration, particularly for when disabled, is to avoid the visual overhead of calculating and rendering the meshes.
| Profile | System Enabled by Default |
|---|---|
| DefaultHoloLens1ConfigurationProfile | False |
| DefaultHoloLens2ConfigurationProfile | False |
| DefaultMixedRealityToolkitConfigurationProfile | True |
Select the MixedRealityToolkit object in the scene hierarchy to open in the Inspector Panel.
Navigate to the Spatial Awareness System section and check Enable Spatial Awareness System
Select the desired Spatial Awareness system implementation type. The
MixedRealitySpatialAwarenessSystemis the default provided.
Register observers
Services in the Mixed Reality Toolkit can have Data Provider services that supplement the main service with platform specific data and implementation controls. An example of this is the Mixed Reality Input System which has multiple data providers to get controller and other related input information from various platform-specific APIs.
The Spatial Awareness system is similar in that data providers supply the system with mesh data about the real-world. The Spatial Awareness profile must have at least one Spatial Observer registered. Spatial Observers are generally platform specific components that act as the provider for surfacing various types of mesh data from a platform specific endpoint (i.e HoloLens).
Open or expand the Spatial Awareness System profile
Click the "Add Spatial Observer" button
Select the desired Spatial Observer implementation type
Modify configuration properties on the observer as necessary
Note
Users of the DefaultMixedRealityToolkitConfigurationProfile will have the Spatial Awareness system pre-configured for the Windows Mixed Reality platform which uses
the WindowsMixedRealitySpatialMeshObserver class.
Build and deploy
Once the Spatial Awareness system is configured with the desired observer(s), the project can be built and deployed to the target platform.
Important
If targeting the Windows Mixed Reality platform (ex: HoloLens), it is important to ensure the Spatial Perception capability is enabled in order to use the Spatial Awareness system on device.
Warning
Some platforms, including Microsoft HoloLens, provide support for remote execution from within Unity. This feature enables rapid development and testing without requiring the build and deploy step. Be sure to do final acceptance testing using a built and deployed version of the application, running on the target hardware and platform.
Next steps
After following the procedures above to enable the Spatial Awareness system, the system can be configured and controlled in more detail.
Information for configuring observers in inspector:
Information for controlling and extending observers via code: