Adjusting Projection Based on IPD Values in Meta Quest 3 with Unity CloudXR Plugin

Gaming and Visualization Technologies XR – VR/AR/MR CloudXR (VR and AR Streaming)
1

Hello everyone,

I am working on a Unity project with Meta Quest 3 using the NVIDIA CloudXR SDK and the CloudXR Client Unity Plugin. I need to adjust the projection settings based on the Interpupillary Distance (IPD) value of the headset to improve the visual experience. After reviewing the NVIDIA CloudXR documentation, particularly the section on updating headset properties, I’m still unsure about how to implement this in Unity.

Problem Description:

I am trying to determine if there is built-in functionality in the CloudXR Client Unity Plugin to automatically adjust projection settings based on the IPD value of the headset. If such functionality is not available, I would appreciate guidance on how to manually compute and apply the necessary projection adjustments within Unity.

Relevant Information:

Headset: Meta Quest 3
Unity Version: 2022.2.12f1
CloudXR SDK Version: 4.0
CloudXR Client Unity Plugin Version: 0.1.1
Documentation Referenced: NVIDIA CloudXR SDK Documentation

Question:

  1. Does the CloudXR Client Unity Plugin provide built-in functionality to adjust projection settings based on the IPD value?
  2. If not, can anyone provide guidance on manually computing and applying projection adjustments based on the IPD value within Unity?
  3. Are there specific steps or best practices for integrating IPD-based adjustments in a Unity project using CloudXR?

I would greatly appreciate any insights or advice on achieving IPD-based projection adjustments with Meta Quest 3 using Unity and the NVIDIA CloudXR SDK. Thank you in advance for your assistance!

2

Hi, @muhyeon_shin, great question!

Getting IPD information out of Unity has been challenging. I believe the Unity plugin is attempting to set the IPD correctly based on what Unity thinks it is during startup. If you want to investigate more, feel free to crack open the Unity plugin tarball, which contains all the code that does this.

In the file CloudXRClient.cs, there is a line:

_dd.ipd = m_xr_manager.getIpd();

This is where the IPD is set when the connection is first made. This is determined using the code of the getIpd() method in CxrUnityXRManager.cs.

You can see that it generates a debug log if Unity’s stereo separation value is 0.22, which is a hard-coded default constant in Unity from what I can tell. If you get that number there, you don’t have the real IPD. What follows is some code to get the IPD by using the Unity XR APIs to get the left and right eye positions and calculate the distance between them.

Remember, the Unity plugin is only offered as a beta right now, and that should factor into decisions about adopting it for deployments.

3

Hi, @AndreusNvidius

Thank you for your response.

I successfully retrieved the HMD’s IPD value using getIpd(). However, what I’m interested in is whether it’s possible to configure the projection settings based on this IPD value.

In the CloudXRClient.cs code, there is a segment as follows:

if (false) { // CXRUNITY-97 m_cxrHmdState.flags = m_cxrVrState.hmd.flags | (ulong)cxrHmdTrackingFlags.cxrHmdTrackingFlags_HasProjection; // m_cxrHmdState.projection = do some fancy stuff with float pointers }

I’m unsure whether I need to manually calculate and set these projection values, or if there is an existing function to handle this. If manual calculation is necessary, could you provide any guidelines or examples on how to do this?

Thank you!

4

All CloudXR clients are required to provide at least one projection during startup, otherwise the server has nothing valid to render. Typically this is done through the cxrDeviceDesc struct passed through cxrReceiverDesc (docs). In the Unity client plugin code, this is done (rather complicatedly) by this code:

// Per eye raw projection extents: tangents of the half-angles from the center view axis. // 0|1 for left|right eye, then 0-3 for left, right, top, bottom edges. float[] p = GetProjectionFlattened(); Log.I($"Projection: {p[0]},{p[1]},{p[2]},{p[3]} {p[4]},{p[5]},{p[6]},{p[7]}"); System.IntPtr floatptr = (System.IntPtr)(SWIGTYPE_p_a_4__float.getCPtr(_dd.proj)); Marshal.Copy(p,0,floatptr,8);

You can follow the code in GetProjectionFlattened() to see how this plugin is getting the projection from Unity’s camera, which (due to how XR cameras are set up in Unity) is automatically set up by Unity’s XR subsystems via whatever XR provider you select vie Project Settings > XR.