🧠 OpenXR Fundamentals
🎯 The Universal XR API
OpenXR is an open, royalty-free standard from the Khronos Group that provides a unified API for AR, VR, and MR devices. It abstracts away vendor-specific SDKs (Oculus, SteamVR, WMR, etc.) so developers write once and deploy everywhere. Supported by Meta, Valve, Microsoft, HTC, Sony, and more.
Why OpenXR Matters
Write once, run anywhere: No more per-vendor SDKs. Future-proof: New devices support OpenXR automatically. Feature-rich: Hand tracking, passthrough, eye tracking, and more via extensions. Industry standard: Adopted by all major XR platforms. Open source: Reference implementation and headers available on GitHub.
// OpenXR minimal initialization (C++)
#include
XrInstance instance;
XrInstanceCreateInfo createInfo{XR_TYPE_INSTANCE_CREATE_INFO};
createInfo.enabledExtensionCount = 0;
createInfo.enabledExtensionNames = nullptr;
createInfo.applicationInfo.apiVersion = XR_CURRENT_API_VERSION;
XrResult result = xrCreateInstance(&createInfo, &instance);
if (XR_FAILED(result)) {
// handle error
}
// Get system (headset)
XrSystemId systemId;
XrSystemGetInfo systemInfo{XR_TYPE_SYSTEM_GET_INFO};
systemInfo.formFactor = XR_FORM_FACTOR_HEAD_MOUNTED_DISPLAY;
xrGetSystem(instance, &systemInfo, &systemId); 🏗️ Architecture & Runtime
🎯 How OpenXR is structured
OpenXR consists of an API layer (what developers call), a runtime (vendor implementation), and drivers (device-specific). The runtime is provided by the platform (Oculus Runtime, SteamVR, Windows Mixed Reality).
OpenXR Architecture
Application: Your XR app calls OpenXR functions. Loader: Dispatches calls to the active runtime. Runtime: Vendor implementation (Oculus, SteamVR, etc.). Drivers: Talk to hardware (headset, controllers). Extensions: Optional features (hand tracking, passthrough, etc.).
// Query runtime properties
XrInstanceProperties instanceProps{XR_TYPE_INSTANCE_PROPERTIES};
xrGetInstanceProperties(instance, &instanceProps);
printf("Runtime: %s %d.%d.%d\n",
instanceProps.runtimeName,
XR_VERSION_MAJOR(instanceProps.runtimeVersion),
XR_VERSION_MINOR(instanceProps.runtimeVersion),
XR_VERSION_PATCH(instanceProps.runtimeVersion));
// Enumerate available extensions
uint32_t extensionCount;
xrEnumerateInstanceExtensionProperties(nullptr, 0, &extensionCount, nullptr);
std::vector extensions(extensionCount, {XR_TYPE_EXTENSION_PROPERTIES});
xrEnumerateInstanceExtensionProperties(nullptr, extensionCount, &extensionCount, extensions.data());
for (const auto& ext : extensions) {
printf("Extension: %s (v%d)\n", ext.extensionName, ext.extensionVersion);
} ⚙️ Instance & System
🎯 The entry points
XrInstance is the connection to the OpenXR runtime. XrSystem represents the XR device (headset). You must create an instance and get a system before doing anything else.
Instance & System Setup
Instance: Created with xrCreateInstance() — specifies app info, extensions, and API version. System: Retrieved with xrGetSystem() — identifies the headset. System properties: Query with xrGetSystemProperties() — display resolution, tracking capabilities, etc.
// Get system properties
XrSystemProperties systemProps{XR_TYPE_SYSTEM_PROPERTIES};
xrGetSystemProperties(instance, systemId, &systemProps);
printf("System: %s\n", systemProps.systemName);
printf("Vendor ID: %d\n", systemProps.vendorId);
printf("Max layers: %d\n", systemProps.graphicsProperties.maxLayerCount);
printf("Max swapchain width: %d\n", systemProps.graphicsProperties.maxSwapchainImageWidth);
printf("Max swapchain height: %d\n", systemProps.graphicsProperties.maxSwapchainImageHeight);
printf("Orientation tracking: %s\n", systemProps.trackingProperties.orientationTracking ? "yes" : "no");
printf("Position tracking: %s\n", systemProps.trackingProperties.positionTracking ? "yes" : "no");🎬 Session Lifecycle
🎯 The active XR experience
XrSession represents an active XR session. It manages the render loop, input, and lifecycle events (idle, ready, synchronized, visible, focused, stopping).
Session States
IDLE: Session created but not rendering. READY: Ready to render. SYNCHRONIZED: Synced with display. VISIBLE: Rendering to display. FOCUSED: Receiving input. STOPPING: Ending. Handle state changes via xrPollEvent().
// Create session
XrSession session;
XrSessionCreateInfo sessionInfo{XR_TYPE_SESSION_CREATE_INFO};
sessionInfo.next = &graphicsBinding; // e.g., XrGraphicsBindingOpenGL
sessionInfo.systemId = systemId;
xrCreateSession(instance, &sessionInfo, &session);
// Event loop
XrEventDataBuffer event;
while (running) {
event = {XR_TYPE_EVENT_DATA_BUFFER};
XrResult result = xrPollEvent(instance, &event);
if (result == XR_SUCCESS) {
switch (event.type) {
case XR_TYPE_EVENT_DATA_SESSION_STATE_CHANGED: {
auto* stateEvent = (XrEventDataSessionStateChanged*)&event;
handleSessionState(stateEvent->state);
break;
}
case XR_TYPE_EVENT_DATA_INSTANCE_LOSS_PENDING: {
running = false;
break;
}
}
}
// Render frame
renderFrame();
}📍 Spaces & Poses
🎯 Coordinate systems in OpenXR
XrSpace defines a coordinate system for tracking. Common spaces: VIEW (head), LOCAL (seated), STAGE (room-scale), LOCAL_FLOOR (floor-level). Poses are queried with xrLocateSpace().
Space Types
VIEW: Head pose, origin at head. LOCAL: Origin at seated position. STAGE: Origin at floor center of play area. LOCAL_FLOOR: Origin at floor level. ACTION: Attached to an input action. Poses are relative to a reference space.
// Create reference spaces
XrSpace viewSpace, localSpace, stageSpace;
XrReferenceSpaceCreateInfo viewInfo{XR_TYPE_REFERENCE_SPACE_CREATE_INFO};
viewInfo.referenceSpaceType = XR_REFERENCE_SPACE_TYPE_VIEW;
viewInfo.poseInReferenceSpace = identityPose;
xrCreateReferenceSpace(session, &viewInfo, &viewSpace);
XrReferenceSpaceCreateInfo localInfo{XR_TYPE_REFERENCE_SPACE_CREATE_INFO};
localInfo.referenceSpaceType = XR_REFERENCE_SPACE_TYPE_LOCAL;
localInfo.poseInReferenceSpace = identityPose;
xrCreateReferenceSpace(session, &localInfo, &localSpace);
XrReferenceSpaceCreateInfo stageInfo{XR_TYPE_REFERENCE_SPACE_CREATE_INFO};
stageInfo.referenceSpaceType = XR_REFERENCE_SPACE_TYPE_STAGE;
stageInfo.poseInReferenceSpace = identityPose;
xrCreateReferenceSpace(session, &stageInfo, &stageSpace);
// Locate head pose in stage space
XrSpaceLocation location{XR_TYPE_SPACE_LOCATION};
xrLocateSpace(viewSpace, stageSpace, predictedDisplayTime, &location);
if (location.locationFlags & XR_SPACE_LOCATION_POSITION_VALID_BIT) {
XrVector3f position = location.pose.position;
XrQuaternionf orientation = location.pose.orientation;
}🎮 Actions & Input
🎯 Unified input across devices
OpenXR uses an action-based input system. You define actions (e.g., "grab", "shoot") and bind them to device-specific inputs (trigger, grip, thumbstick) via interaction profiles.
Action System
Action Set: Group of related actions. Action: A semantic input (boolean, float, vector2, pose). Binding: Maps action to device input (e.g., Oculus Touch trigger). Interaction Profile: Device-specific bindings. Sync: xrSyncActions() updates action states each frame.
// Create action set
XrActionSet actionSet;
XrActionSetCreateInfo actionSetInfo{XR_TYPE_ACTION_SET_CREATE_INFO};
strcpy(actionSetInfo.actionSetName, "gameplay");
strcpy(actionSetInfo.localizedActionSetName, "Gameplay");
xrCreateActionSet(instance, &actionSetInfo, &actionSet);
// Create actions
XrAction grabAction, shootAction, moveAction;
XrActionCreateInfo grabInfo{XR_TYPE_ACTION_CREATE_INFO};
strcpy(grabInfo.actionName, "grab");
grabInfo.actionType = XR_ACTION_TYPE_BOOLEAN_INPUT;
grabInfo.countSubactionPaths = 2;
grabInfo.subactionPaths = handPaths; // left, right
xrCreateAction(actionSet, &grabInfo, &grabAction);
XrActionCreateInfo moveInfo{XR_TYPE_ACTION_CREATE_INFO};
strcpy(moveInfo.actionName, "move");
moveInfo.actionType = XR_ACTION_TYPE_VECTOR2F_INPUT;
xrCreateAction(actionSet, &moveInfo, &moveAction);
// Suggest bindings for Oculus Touch
XrActionSuggestedBinding bindings[] = {
{grabAction, xrGetPath("/user/hand/right/input/trigger/value")},
{moveAction, xrGetPath("/user/hand/left/input/thumbstick")},
};
XrInteractionProfileSuggestedBinding suggested{XR_TYPE_INTERACTION_PROFILE_SUGGESTED_BINDING};
suggested.interactionProfile = xrGetPath("/interaction_profiles/oculus/touch_controller");
suggested.suggestedBindings = bindings;
suggested.countSuggestedBindings = 2;
xrSuggestInteractionProfileBindings(instance, &suggested);
// Attach action set to session
XrSessionActionSetsAttachInfo attachInfo{XR_TYPE_SESSION_ACTION_SETS_ATTACH_INFO};
attachInfo.countActionSets = 1;
attachInfo.actionSets = &actionSet;
xrAttachSessionActionSets(session, &attachInfo);
// Sync actions each frame
XrActiveActionSet activeActionSet{actionSet, XR_NULL_PATH};
XrActionsSyncInfo syncInfo{XR_TYPE_ACTIONS_SYNC_INFO};
syncInfo.countActiveActionSets = 1;
syncInfo.activeActionSets = &activeActionSet;
xrSyncActions(session, &syncInfo);
// Read action state
XrActionStateGetInfo getInfo{XR_TYPE_ACTION_STATE_GET_INFO};
getInfo.action = grabAction;
getInfo.subactionPath = rightHandPath;
XrActionStateBoolean grabState{XR_TYPE_ACTION_STATE_BOOLEAN};
xrGetActionStateBoolean(session, &getInfo, &grabState);
bool isGrabbing = grabState.currentState;🖼️ Swapchains & Rendering
🎯 Getting pixels to the display
XrSwapchain manages the image buffers for rendering. You create a swapchain, acquire an image, render to it, release it, and the runtime composites it to the display.
Swapchain Workflow
1. Create swapchain → specify format, width, height, sample count. 2. Enumerate images → get the image buffers. 3. Acquire image → xrAcquireSwapchainImage(). 4. Wait → xrWaitSwapchainImage(). 5. Render → draw to the image. 6. Release → xrReleaseSwapchainImage(). 7. End frame → xrEndFrame().
// Create swapchain
XrSwapchain swapchain;
XrSwapchainCreateInfo swapchainInfo{XR_TYPE_SWAPCHAIN_CREATE_INFO};
swapchainInfo.arraySize = 1;
swapchainInfo.format = GL_RGBA8;
swapchainInfo.width = viewConfig.recommendedImageRectWidth;
swapchainInfo.height = viewConfig.recommendedImageRectHeight;
swapchainInfo.mipCount = 1;
swapchainInfo.faceCount = 1;
swapchainInfo.sampleCount = 1;
swapchainInfo.usageFlags = XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT;
xrCreateSwapchain(session, &swapchainInfo, &swapchain);
// Enumerate images
uint32_t imageCount;
xrEnumerateSwapchainImages(swapchain, 0, &imageCount, nullptr);
std::vector images(imageCount, {XR_TYPE_SWAPCHAIN_IMAGE_OPENGL_KHR});
xrEnumerateSwapchainImages(swapchain, imageCount, &imageCount, (XrSwapchainImageBaseHeader*)images.data());
// Render loop
XrSwapchainImageAcquireInfo acquireInfo{XR_TYPE_SWAPCHAIN_IMAGE_ACQUIRE_INFO};
uint32_t imageIndex;
xrAcquireSwapchainImage(swapchain, &acquireInfo, &imageIndex);
XrSwapchainImageWaitInfo waitInfo{XR_TYPE_SWAPCHAIN_IMAGE_WAIT_INFO};
waitInfo.timeout = XR_INFINITE_DURATION;
xrWaitSwapchainImage(swapchain, &waitInfo);
// Render to images[imageIndex].image
glBindFramebuffer(GL_FRAMEBUFFER, framebuffer);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, images[imageIndex].image, 0);
renderScene();
XrSwapchainImageReleaseInfo releaseInfo{XR_TYPE_SWAPCHAIN_IMAGE_RELEASE_INFO};
xrReleaseSwapchainImage(swapchain, &releaseInfo);
// End frame
XrCompositionLayerProjectionView projectionViews[2];
// ... fill projection views ...
XrCompositionLayerProjection projectionLayer{XR_TYPE_COMPOSITION_LAYER_PROJECTION};
projectionLayer.space = stageSpace;
projectionLayer.viewCount = 2;
projectionLayer.views = projectionViews;
const XrCompositionLayerBaseHeader* layers[] = {(XrCompositionLayerBaseHeader*)&projectionLayer};
XrFrameEndInfo endInfo{XR_TYPE_FRAME_END_INFO};
endInfo.displayTime = predictedDisplayTime;
endInfo.environmentBlendMode = XR_ENVIRONMENT_BLEND_MODE_OPAQUE;
endInfo.layerCount = 1;
endInfo.layers = layers;
xrEndFrame(session, &endInfo); 🧩 Composition Layers
🎯 Multiple layers for quality and performance
Composition layers let you composite multiple textures (3D scene, UI, video) at different resolutions. Types: Projection, Quad, Cylinder, Cube, Equirect.
Layer Types
Projection: Standard 3D scene (one per eye). Quad: Flat panel (UI, video). Cylinder: Curved surface. Cube: Skybox. Equirect: 360° panorama. Layers can be world-locked or head-locked.
// Quad layer for UI
XrCompositionLayerQuad quadLayer{XR_TYPE_COMPOSITION_LAYER_QUAD};
quadLayer.layerFlags = XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT;
quadLayer.space = stageSpace;
quadLayer.eyeVisibility = XR_EYE_VISIBILITY_BOTH;
quadLayer.subImage.swapchain = uiSwapchain;
quadLayer.subImage.imageRect = {0, 0, uiWidth, uiHeight};
quadLayer.pose = uiPose; // position and orientation
quadLayer.size = {1.0f, 0.5f}; // meters
// Projection layer for 3D scene
XrCompositionLayerProjection projectionLayer{XR_TYPE_COMPOSITION_LAYER_PROJECTION};
projectionLayer.space = stageSpace;
projectionLayer.viewCount = 2;
projectionLayer.views = projectionViews;
// Submit both layers
const XrCompositionLayerBaseHeader* layers[] = {
(XrCompositionLayerBaseHeader*)&projectionLayer,
(XrCompositionLayerBaseHeader*)&quadLayer
};
endInfo.layerCount = 2;
endInfo.layers = layers;
xrEndFrame(session, &endInfo);🔌 Extensions System
🎯 Optional features beyond core
OpenXR extensions add optional functionality: hand tracking, eye tracking, passthrough, spatial anchors, and more. Extensions are enabled at instance creation.
Key Extensions
XR_EXT_hand_tracking: Hand joint tracking. XR_EXT_eye_gaze_interaction: Eye tracking. XR_FB_passthrough: Meta passthrough. XR_MSFT_spatial_anchor: Spatial anchors. XR_KHR_composition_layer_depth: Depth layers. XR_EXT_hp_mixed_reality_controller: HP controllers. Check availability with xrEnumerateInstanceExtensionProperties().
// Enable extensions at instance creation
const char* enabledExtensions[] = {
XR_EXT_HAND_TRACKING_EXTENSION_NAME,
XR_FB_PASSTHROUGH_EXTENSION_NAME,
XR_EXT_EYE_GAZE_INTERACTION_EXTENSION_NAME
};
XrInstanceCreateInfo createInfo{XR_TYPE_INSTANCE_CREATE_INFO};
createInfo.enabledExtensionCount = 3;
createInfo.enabledExtensionNames = enabledExtensions;
xrCreateInstance(&createInfo, &instance);
// Check if extension is supported
bool isExtensionSupported(const char* extensionName) {
uint32_t count;
xrEnumerateInstanceExtensionProperties(nullptr, 0, &count, nullptr);
std::vector extensions(count, {XR_TYPE_EXTENSION_PROPERTIES});
xrEnumerateInstanceExtensionProperties(nullptr, count, &count, extensions.data());
for (const auto& ext : extensions) {
if (strcmp(ext.extensionName, extensionName) == 0) return true;
}
return false;
}
// Hand tracking extension
if (isExtensionSupported(XR_EXT_HAND_TRACKING_EXTENSION_NAME)) {
XrHandTrackerEXT handTracker;
XrHandTrackerCreateInfoEXT handInfo{XR_TYPE_HAND_TRACKER_CREATE_INFO_EXT};
handInfo.hand = XR_HAND_LEFT_EXT;
handInfo.handJointSet = XR_HAND_JOINT_SET_DEFAULT_EXT;
xrCreateHandTrackerEXT(session, &handInfo, &handTracker);
} ✋ Hand Tracking Extension
🎯 Natural hand input via OpenXR
The XR_EXT_hand_tracking extension provides 26 joint poses per hand, enabling controller-free interaction. Works on Quest, HoloLens 2, and other supported devices.
Hand Joints
26 joints: wrist, palm, and 5 joints per finger (metacarpal, proximal, distal, tip). Joint poses: queried with xrLocateHandJointsEXT(). Aim pose: pointing direction for each hand. Gesture: pinch strength, grab strength.
// Locate hand joints
XrHandJointLocationsEXT jointLocations{XR_TYPE_HAND_JOINT_LOCATIONS_EXT};
XrHandJointVelocitiesEXT jointVelocities{XR_TYPE_HAND_JOINT_VELOCITIES_EXT};
XrHandJointsLocateInfoEXT locateInfo{XR_TYPE_HAND_JOINTS_LOCATE_INFO_EXT};
locateInfo.baseSpace = stageSpace;
locateInfo.time = predictedDisplayTime;
XrHandJointLocationEXT jointLocationsArray[XR_HAND_JOINT_COUNT_EXT];
jointLocations.jointCount = XR_HAND_JOINT_COUNT_EXT;
jointLocations.jointLocations = jointLocationsArray;
xrLocateHandJointsEXT(handTracker, &locateInfo, &jointLocations);
// Access specific joints
XrHandJointLocationEXT indexTip = jointLocationsArray[XR_HAND_JOINT_INDEX_TIP_EXT];
XrHandJointLocationEXT thumbTip = jointLocationsArray[XR_HAND_JOINT_THUMB_TIP_EXT];
// Compute pinch distance
float dx = indexTip.pose.position.x - thumbTip.pose.position.x;
float dy = indexTip.pose.position.y - thumbTip.pose.position.y;
float dz = indexTip.pose.position.z - thumbTip.pose.position.z;
float pinchDistance = sqrtf(dx*dx + dy*dy + dz*dz);
bool isPinching = pinchDistance < 0.02f; // 2cm📷 Passthrough & AR
🎯 Blending real and virtual
Passthrough extensions (Meta, HoloLens) allow the real world to be seen through the headset cameras, enabling AR/MR experiences on VR hardware.
Passthrough Features
XR_FB_passthrough: Meta Quest color passthrough. XR_MSFT_holographic_window_attachment: HoloLens. Layer-based: Passthrough as a composition layer. Style: Color, grayscale, edge rendering. AR Foundation: Unity abstraction for passthrough.
// Meta passthrough (XR_FB_passthrough)
XrPassthroughFB passthrough;
XrPassthroughCreateInfoFB passthroughInfo{XR_TYPE_PASSTHROUGH_CREATE_INFO_FB};
passthroughInfo.flags = XR_PASSTHROUGH_IS_RUNNING_AT_CREATION_BIT_FB;
xrCreatePassthroughFB(session, &passthroughInfo, &passthrough);
// Create passthrough layer
XrPassthroughLayerFB passthroughLayer;
XrPassthroughLayerCreateInfoFB layerInfo{XR_TYPE_PASSTHROUGH_LAYER_CREATE_INFO_FB};
layerInfo.passthrough = passthrough;
layerInfo.purpose = XR_PASSTHROUGH_LAYER_PURPOSE_RECONSTRUCTION_FB;
xrCreatePassthroughLayerFB(session, &layerInfo, &passthroughLayer);
// Add layer to frame
XrCompositionLayerPassthroughFB passthroughCompLayer{XR_TYPE_COMPOSITION_LAYER_PASSTHROUGH_FB};
passthroughCompLayer.layerHandle = passthroughLayer;
passthroughCompLayer.flags = XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT;
const XrCompositionLayerBaseHeader* layers[] = {
(XrCompositionLayerBaseHeader*)&passthroughCompLayer,
(XrCompositionLayerBaseHeader*)&projectionLayer
};
endInfo.layers = layers;
endInfo.layerCount = 2;🎮 Unity & Unreal Integration
🎯 OpenXR in game engines
Unity and Unreal Engine have built-in OpenXR support, abstracting the low-level API into high-level components. You get cross-platform XR without writing C++ OpenXR code.
Unity & Unreal OpenXR
Unity: XR Plugin Management with OpenXR provider. XR Interaction Toolkit for input. Unreal: OpenXR plugin, built-in support since UE4.26. Features: Hand tracking, passthrough, eye tracking via extensions. Deployment: Build once, run on Quest, SteamVR, WMR, etc.
// Unity OpenXR: enabling features
using UnityEngine.XR.OpenXR;
using UnityEngine.XR.OpenXR.Features;
using UnityEngine.XR.OpenXR.Features.MetaQuestSupport;
public class OpenXRSetup : MonoBehaviour {
void Start() {
// Enable Meta Quest support
var questFeature = OpenXRSettings.Instance.GetFeature();
if (questFeature != null) {
questFeature.enabled = true;
}
// Enable hand tracking
var handFeature = OpenXRSettings.Instance.GetFeature();
if (handFeature != null) {
handFeature.enabled = true;
}
}
}
// Unreal OpenXR: enabling extensions
// In DefaultEngine.ini:
// [/Script/OpenXRPlugin.OpenXRPlugin]
// +EnabledExtensions=XR_EXT_hand_tracking
// +EnabledExtensions=XR_FB_passthrough ✅ Validation & Debugging
🎯 Catching errors early
OpenXR provides validation layers and debugging tools: XR_EXT_debug_utils for error messages, API layers for validation, and runtime debug tools.
Debugging Tools
XR_EXT_debug_utils: Debug messages, object naming, labels. Validation layer: Catches API misuse. API dump layer: Logs all calls. Runtime logs: Oculus, SteamVR, WMR logs. RenderDoc: GPU debugging for XR.
// Enable debug utils
const char* extensions[] = {XR_EXT_DEBUG_UTILS_EXTENSION_NAME};
XrInstanceCreateInfo createInfo{XR_TYPE_INSTANCE_CREATE_INFO};
createInfo.enabledExtensionCount = 1;
createInfo.enabledExtensionNames = extensions;
// Set up debug messenger
XrDebugUtilsMessengerEXT debugMessenger;
XrDebugUtilsMessengerCreateInfoEXT debugInfo{XR_TYPE_DEBUG_UTILS_MESSENGER_CREATE_INFO_EXT};
debugInfo.messageSeverities = XR_DEBUG_UTILS_MESSAGE_SEVERITY_ERROR_BIT_EXT |
XR_DEBUG_UTILS_MESSAGE_SEVERITY_WARNING_BIT_EXT;
debugInfo.messageTypes = XR_DEBUG_UTILS_MESSAGE_TYPE_GENERAL_BIT_EXT |
XR_DEBUG_UTILS_MESSAGE_TYPE_VALIDATION_BIT_EXT |
XR_DEBUG_UTILS_MESSAGE_TYPE_PERFORMANCE_BIT_EXT;
debugInfo.userCallback = debugCallback;
debugInfo.userData = nullptr;
xrCreateDebugUtilsMessengerEXT(instance, &debugInfo, &debugMessenger);
// Debug callback
XrBool32 debugCallback(
XrDebugUtilsMessageSeverityFlagsEXT severity,
XrDebugUtilsMessageTypeFlagsEXT type,
const XrDebugUtilsMessengerCallbackDataEXT* data,
void* userData)
{
printf("[OpenXR] %s: %s\n", data->functionName, data->message);
return XR_FALSE; // don't abort
}
// Name objects for debugging
XrDebugUtilsObjectNameInfoEXT nameInfo{XR_TYPE_DEBUG_UTILS_OBJECT_NAME_INFO_EXT};
nameInfo.objectType = XR_OBJECT_TYPE_SESSION;
nameInfo.objectHandle = (uint64_t)session;
nameInfo.objectName = "MainSession";
xrSetDebugUtilsObjectNameEXT(instance, &nameInfo);🚀 Future: OpenXR + AI
🎯 AI-powered OpenXR experiences
OpenXR + AI will enable intelligent XR: generative content, conversational agents, semantic understanding, and adaptive experiences across all devices.
OpenXR + AI Integration
AI extensions: Neural network inference for hand/eye tracking. Generative: Create 3D content from text. LLM agents: Conversational NPCs in XR. Semantic understanding: Real-time scene interpretation. Neural rendering: NeRF/Gaussian Splatting via OpenXR.
// OpenXR + AI: neural hand tracking (conceptual)
// Use an AI model to predict hand joints from camera images
// then feed into OpenXR hand tracking extension
XrHandTrackerEXT handTracker;
XrHandTrackerCreateInfoEXT handInfo{XR_TYPE_HAND_TRACKER_CREATE_INFO_EXT};
handInfo.hand = XR_HAND_LEFT_EXT;
handInfo.handJointSet = XR_HAND_JOINT_SET_DEFAULT_EXT;
xrCreateHandTrackerEXT(session, &handInfo, &handTracker);
// AI model inference (e.g., MediaPipe, HaMeR)
// Process camera image with neural network
// Get 21 hand landmarks
// Map to OpenXR 26 joints
// Update hand joint locations
// Use for interaction in XR app
// Future: OpenXR ML extension
// XR_EXT_ml_inference - run neural networks via OpenXR
// XR_EXT_semantic_understanding - scene understanding