mrtk_developmentreleases/2.0.0releases/2.1.0releases/2.2.0releases/2.3.0releases/2.4.0releases/2.5.0releases/2.5.1releases/2.5.2releases/2.5.3
  • Features and Architecture
  • API Documentation

We've moved!

Starting from MRTK 2.6, we are publishing both conceptual docs and API references on docs.microsoft.com. For conceptual docs, please visit our new landing page. For API references, please visit the MRTK-Unity section of the dot net API explorer. Existing content will remain here but will not be updated further.

  • Features and Architecture
  • Feature Overviews
  • Spatial Awareness System
  • Spatial Observers
  • Controlling Observers via Code
Search Results for

    Show / Hide Table of Contents
    • Welcome to MRTK
      • Installation Guide
      • Configuration
        • Using the Unity Package Manager
        • MRTK configuration dialog
        • Getting started with MRTK and XR SDK
      • Updates and Deployment
        • Updating from earlier versions
        • Upgrading from HTK
        • Building and Deploying MRTK
      • Packages and Release Notes
        • Release Notes
        • MRTK Packages
      • Performance and Best Practices
        • Performance
        • Hologram Stabilization
        • Using MRTK in large projects
    • Architecture
      • Overview
      • Framework and Runtime
      • Input System
        • Terminology
        • Core System
        • Controllers, Pointers, and Focus
      • Systems, Extension Services and Data Providers
    • Feature Overviews
      • Boundary System
        • Boundary System Overview
        • Configuring the Boundary Visualization
      • Camera System
        • Camera System Overview
        • Camera Settings Providers
          • Windows Mixed Reality Camera Settings
          • Unity AR Camera Settings [Experimental]
          • Creating a camera settings provider
      • Cross Platform Support
        • Configure MRTK for iOS and Android
        • Configure MRTK for Leap Motion Hand Tracking
        • Configure MRTK for Oculus Quest
      • Detecting Platform Capabilities
      • Diagnostics System
        • Diagnostics System Overview
        • Configuring the Diagnostics System
        • Using the Visual Profiler
      • Extension Services
        • Extension Service Creation Wizard
        • Scene Transition Service Overview
        • Hand Physics Service Overview
      • Input System
        • Input Overview
        • Input Actions
        • Input Events
        • Input Providers
          • Input Providers Overview
          • Creating an input data provider
        • Controllers
        • Eyes
          • Overview
          • Getting Started
          • Access Data via Code
          • Validate Tracking Calibration
        • Gaze
        • Gestures
        • Hands
        • How to Add Near Interaction
        • In-Editor Input Simulation
        • Pointers
        • Voice Input
          • Dictation
          • Speech (Command and Control)
      • Multi Scene System
        • Multi Scene System Overview
        • Scene Types
        • Content Scene Loading
        • Monitoring Content Loading
        • Lighting Scene Operations
      • Packaging
        • MRTK Packages
        • MRTK Modularization
      • Profiles
        • Profiles Overview
        • Configuration Guide
      • Rendering
        • MRTK Standard Shader
        • Material Instance Overview
        • Hover Light Overview
        • Proximity Light Overview
        • Clipping Primitive Overview
      • Services
        • What makes a mixed reality feature
        • What are the MixedRealityServiceRegistry and IMixedRealityServiceRegistrar
        • Extension services
      • Spatial Awareness System
        • Spatial Awareness Overview
        • Spatial Observers
          • Configuring Observers for Device
          • Configuring Observers for Editor
          • Controlling Observers via Code
          • Creating a custom Observer
      • Teleport System Overview
      • Tools
        • Dependency Window
        • Extension Service Creation Wizard
        • Holographic Remoting
        • Input Animation Recording
          • Input Animation File Format Specification
        • Migration Window
        • Optimize Window
        • Runtime tools
          • Controller Mapping tool
          • InputFeatureUsage tool
      • UX Building Blocks
        • Toolbox Window
        • Button
        • Bounds Control
        • Object Manipulator
        • Constraint Manager
        • Slate
        • System Keyboard
        • Interactable
        • Solvers
          • Tap to Place
        • Object Collection
        • Scrolling Object Collection
        • Tooltips
        • Slider
        • Hand Menu
        • Near Menu
        • App Bar
        • Fingertip Visualization
        • Progress Indicator
        • Dialog [Experimental]
        • Hand Coach [Experimental]
        • Pulse Shader [Experimental]
        • Dock Control [Experimental]
        • HoloLens Keyboard Helpers [Experimental]
        • Rigged Hand Visualizer [Experimental]
        • Elastic System [Experimental]
        • Bounding Box [Obsolete]
        • Manipulation Handler [Obsolete]
      • Example Scenes
        • Examples Hub
        • Hand Interaction Example
        • Eye Tracking Interaction Example
    • Contributing
      • Contributing Overview
      • Coding Guidelines
      • Writing and Running Tests
      • Writing Documentation
      • Pull Requests
      • Experimental Features
      • Breaking Changes
      • How to use DocFX
    • Planning
      • Roadmap
    • Notice
    • Authors

    Configuring mesh observers via code

    This article will discuss some of the key mechanisms and APIs to programmatically configure the Spatial Awareness system and related Mesh Observer data providers.

    Accessing mesh observers

    Mesh Observer classes that implement the IMixedRealitySpatialAwarenessMeshObserver interface provide platform-specific mesh data to the Spatial Awareness system. Multiple Observers can be configured in the Spatial Awareness profile.

    Accessing the data providers of the Spatial Awareness system is mostly the same as for any other Mixed Reality Toolkit service. The Spatial Awareness service must be casted to the IMixedRealityDataProviderAccess interface to access via the GetDataProvider<T> APIs, which can then be utilized to access the Mesh Observer objects directly at runtime.

    // Use CoreServices to quickly get access to the IMixedRealitySpatialAwarenessSystem
    var spatialAwarenessService = CoreServices.SpatialAwarenessSystem;
    
    // Cast to the IMixedRealityDataProviderAccess to get access to the data providers
    var dataProviderAccess = spatialAwarenessService as IMixedRealityDataProviderAccess;
    
    var meshObserver = dataProviderAccess.GetDataProvider<IMixedRealitySpatialAwarenessMeshObserver>();
    

    The CoreServices.GetSpatialAwarenessSystemDataProvider<T>() helper simplifies this access pattern as demonstrated below.

    // Get the first Mesh Observer available, generally we have only one registered
    var meshObserver = CoreServices.GetSpatialAwarenessSystemDataProvider<IMixedRealitySpatialAwarenessMeshObserver>();
    
    // Get the SpatialObjectMeshObserver specifically
    var meshObserverName = "Spatial Object Mesh Observer";
    var spatialObjectMeshObserver = dataProviderAccess.GetDataProvider<IMixedRealitySpatialAwarenessMeshObserver>(meshObserverName);
    

    Starting and stopping mesh observation

    One of the most common tasks when dealing with the Spatial Awareness system is turning the feature off/on dynamically at runtime. This is done per Observer via the IMixedRealitySpatialAwarenessObserver.Resume and IMixedRealitySpatialAwarenessObserver.Suspend APIs.

    // Get the first Mesh Observer available, generally we have only one registered
    var observer = CoreServices.GetSpatialAwarenessSystemDataProvider<IMixedRealitySpatialAwarenessMeshObserver>();
    
    // Suspends observation of spatial mesh data
    observer.Suspend();
    
    // Resumes observation of spatial mesh data
    observer.Resume();
    

    This code functionality can also be simplified via access through the Spatial Awareness system directly.

    var meshObserverName = "Spatial Object Mesh Observer";
    CoreServices.SpatialAwarenessSystem.ResumeObserver<IMixedRealitySpatialAwarenessMeshObserver>(meshObserverName);
    

    Starting and stopping all mesh observation

    It is generally convenient to start/stop all mesh observation in the application. This can be achieved through the helpful Spatial Awareness system APIs, ResumeObservers() and SuspendObservers().

    // Resume Mesh Observation from all Observers
    CoreServices.SpatialAwarenessSystem.ResumeObservers();
    
    // Suspend Mesh Observation from all Observers
    CoreServices.SpatialAwarenessSystem.SuspendObservers();
    

    Enumerating and accessing the meshes

    Accessing the meshes can be done per Observer and then enumerating through the meshes known to that Mesh Observer via the IMixedRealitySpatialAwarenessMeshObserver API.

    If running in editor, one can use the AssetDatabase.CreateAsset() to save the Mesh object to an asset file.

    If running on device, there are many community and store plugins available to serialize the MeshFilter data into a model file type(OBJ Example).

    // Get the first Mesh Observer available, generally we have only one registered
    var observer = CoreServices.GetSpatialAwarenessSystemDataProvider<IMixedRealitySpatialAwarenessMeshObserver>();
    
    // Loop through all known Meshes
    foreach (SpatialAwarenessMeshObject meshObject in observer.Meshes.Values)
    {
        Mesh mesh = meshObject.Filter.mesh;
        // Do something with the Mesh object
    }
    

    Showing and hiding the spatial mesh

    It's possible to programmatically hide/show meshes using the sample code below:

    // Get the first Mesh Observer available, generally we have only one registered
    var observer = CoreServices.GetSpatialAwarenessSystemDataProvider<IMixedRealitySpatialAwarenessMeshObserver>();
    
    // Set to not visible
    observer.DisplayOption = SpatialAwarenessMeshDisplayOptions.None;
    
    // Set to visible and the Occlusion material
    observer.DisplayOption = SpatialAwarenessMeshDisplayOptions.Occlusion;
    

    Registering for mesh observation events

    Components can implement the IMixedRealitySpatialAwarenessObservationHandler<SpatialAwarenessMeshObject> and then register with the Spatial Awareness system to receive Mesh Observation events.

    The DemoSpatialMeshHandler (Assets/MRTK/Examples/Demos/SpatialAwareness/Scripts) script is a useful example and starting point for listening to Mesh Observer events.

    This is a simplified example of DemoSpatialMeshHandler script and Mesh Observation event listening.

    // Simplify type
    using SpatialAwarenessHandler = IMixedRealitySpatialAwarenessObservationHandler<SpatialAwarenessMeshObject>;
    
    public class MyMeshObservationExample : MonoBehaviour, SpatialAwarenessHandler
    {
        private void OnEnable()
        {
            // Register component to listen for Mesh Observation events, typically done in OnEnable()
            CoreServices.SpatialAwarenessSystem.RegisterHandler<SpatialAwarenessHandler>(this);
        }
    
        private void OnDisable()
        {
            // Unregister component from Mesh Observation events, typically done in OnDisable()
            CoreServices.SpatialAwarenessSystem.UnregisterHandler<SpatialAwarenessHandler>(this);
        }
    
        public virtual void OnObservationAdded(MixedRealitySpatialAwarenessEventData<SpatialAwarenessMeshObject> eventData)
        {
            // Do stuff
        }
    
        public virtual void OnObservationUpdated(MixedRealitySpatialAwarenessEventData<SpatialAwarenessMeshObject> eventData)
        {
            // Do stuff
        }
    
        public virtual void OnObservationRemoved(MixedRealitySpatialAwarenessEventData<SpatialAwarenessMeshObject> eventData)
        {
            // Do stuff
        }
    }
    

    See also

    • Spatial Awareness Getting Started
    • Configuring the Spatial Awareness Mesh Observer
    • Spatial Awareness API documentation
    • Improve this Doc
    In This Article
    • Accessing mesh observers
    • Starting and stopping mesh observation
      • Starting and stopping all mesh observation
    • Enumerating and accessing the meshes
    • Showing and hiding the spatial mesh
    • Registering for mesh observation events
    • See also
    Back to top Generated by DocFX