Reality kit Entities Appearing to Lag in a Full or Progressive Style Immersive Space When Opened with Environment Turned On

Spatial Computing ARKit
Nathan_0707 OP
Created
Replies 2
Boosts 1
Views 632
Participants 3

PLATFORM AND VERSION Vision OS

Development environment: Xcode 16.2, macOS 15.2 Run-time configuration: visionOS 2.3 (On Real Device, Not simulator)

Please someone confirm I'm not crazy and this issue is actually out of my control.

Spent hours trying to fix my app and running profiles because thought it was an issue related to my apps performance. Finally considered chance it was issue with API itself and made sample app to isolate problem, and it still existed in it. The issue is when a model entity moves around in a full space that was launched when the system environment immersion was turned up before opening it, the entities looks very choppy as they move around. If you take off the headset while still in the space, and put it back on, this fixes it and then they move smoothly as they should. In addition, you can also leave the space, and then turn the system environment immersion all the way down before launching the full space again, this will also make the entity moves smoothly as it should. If you launch a mixed immersion style instead of a full immersion style, this issue never arrises. The issue only arrises if you launch the space with either a full style, or progressive style, while the system immersion level is turned on.

STEPS TO REPRODUCE

https://github.com/nathan-707/ChoppyEntitySample

Open my test project, its a small, modified vision os project template that shows it clearly.

otherwise:

  • create immersive space with either full or progressive immersion style.
  • setup a entity in kinematic mode, apply a velocity to it to make it pass over your head when the space appears.

if you opened the space while the Apple Vision Pros system environment was turned up, the entity will look choppy.

if you take the headset off while in the same space, and put it back on, it will fix the issue and it will look smooth.

alternatively if you open the space with the system immersion environment all the way down, you will also not run into the issue. Again, issue also does not happen if space launched is in mixed style.

Replies  2
Boosts 1
Views  632
Participants  3
Vision Pro Engineer OP
Apple

These are the steps I have followed:

  1. Created an Xcode project with your code
  2. Play to device (I see your button and text prompt)
  3. Before opening the immersive space, I turn the immersion level all the way up by turning the Digital Crown (I no longer see passthrough, I see Lake Vrangla).
  4. Press the button to open the immersive space (note: this hides the system immersive environment, I see passthrough for a brief moment, and then I see your app's immersive environment, which is a black void).
  5. The green cube flies over my head

Unfortunately I do not perceive any lag, and the cube flies overhead smoothly.

Could you describe more what you mean by "choppy" and "lag"? Is the rest of the operating system responsive to button taps, swipes, etc. or are those interactions also choppy? If it is a drop in frame rate, could you estimate what that drop is?

Thanks for your question and detailed write up (it really does help a lot!). You are also welcome to file a bug report using Feedback Assistant in order to get more eyes on this.
Lafan OP

I'm seeing what may be the same issue on visionOS 27.0.1 (Apple Vision Pro), and I have
measurements that may help narrow it down.

Setup: SwiftUI app, RealityView in a full ImmersiveSpace. The world moves around the user
every frame, so any change in content latency shows as judder during turns.

Symptom: when the space is opened without a tap in the app's UI (system launch via
UIApplicationPreferredDefaultSceneSessionRole, or openImmersiveSpace from code, even after
a delay), RealityKit content judders while moving. When the space is opened by tapping a
button in a window, or after the user taps any SwiftUI control inside the space (e.g. a
ViewAttachmentComponent), motion becomes smooth and stays smooth for the rest of the session.
A pinch in empty air doesn't help.

What changes: frame rate, CPU and GPU load are the same in both states. Only the timing
of the app's UI update cycle changes. With a UIUpdateLink on the immersive UIWindowScene,
in .beforeCATransactionCommit:

estimatedPresentationTime - now: ~23.9 ms (judder) -> ~17.1 ms (smooth) at 100 Hz
completionDeadlineTime - now: ~13.9 ms -> ~7.1 ms

The switch happens exactly on the first tap delivered to the app's UI. In RealityKit Trace,
the app's entity commits land 3–6 ms after the render server's frame start before the tap
(close to its latch, so content latency alternates between frames) and 0–2 ms after it.
RealityKit's app-side update runs inside UIKit's update cycle (_UIUpdateSequenceRunNext ->
CA commit -> re::Engine::doUpdate), so the UI update scheduler seems to stay in a
"waiting for input" timing mode until the first digitizer event arrives.

Workarounds: open the space from a window button, or require a tap on an in-space SwiftUI
control at start