RealityView AR - anchored to the screen not the floor

Question

Created 3w

Replies 1

Boosts 0

Participants 2

This started out as a plea for help, but in preparing this post I discovered the root cause. I'm posting it as a lesson learned in hopes it will help someone.

I've spent a good chunk of March trying to get AR-mode working again in my unreleased game. I had it working with SceneKit and ARView 5 years ago, but since 2024 I've been converting the game to use RealityKit and RealityView on iOS, macOS, visionOS, and tvOS. I've been having no joy getting AR mode to work on iOS. I get the pass-through device video but the game content isn't anchored to the floor but rather anchored to the screen.

I made a simple project with just a simple shape in the middle of a RealityView and an overlay with a SwiftUI toggle to go in and out of AR-mode. At first, my simple project worked, and I couldn't figure out what was different in the logic. Both projects used the same logic:

func transitionToXR(_ content: inout RealityViewCameraContent) {
    content.remove(gameBoard.rootEntity)
    content.add(xrAnchor)
    content.camera = .spatialTracking
    
    Self.anchorStateChangedSubscription = content.subscribe(to: SceneEvents.AnchoredStateChanged.self)  { event in
        if event.anchor == xrAnchor, event.isAnchored {
            xrAnchor.addChild(gameBoard.rootEntity)
        }
    }
}

Then I made an alternate version of my view, and reproduced the same "anchored to the screen not the floor" issue.

I compared the code side-by-side and finally saw the difference! The one that didn't work, like my game, had a property 'cameraEntity' which is initialized with PerspectiveCamera(), position and look-at configured, then added as a child of the root entity. So, the simple fix was to remove 'cameraEntity' from the root entity before adding it to the detected AnchorEntity when going into AR-mode. Then when leaving AR-mode, I add back 'cameraEntity' as a child of the root entity and configure it again.

So the lesson learned is: make sure there isn't a PerspectiveCamera in the tree of Entities added to an AnchorEntity with a .spatialTracking content camera.

Apple: let me know if you think this is a bug or if I was being dumb. If a bug, I can use Feedback Assistant to report this. If I was being dumb, it wouldn't be the first time. :-)

Boost

Answer 1

DTS Engineer OP

Apple

1h

Thanks for the excellent write-up and for sharing your findings — this will definitely help others.

We were able to reproduce the issue using a minimal project that mirrors your setup: a PerspectiveCamera added as a child of the root entity, then transitioning to AR with .spatialTracking and an AnchoredStateChanged subscription. With the camera in the tree, the content floats with the device instead of anchoring to the floor. Removing the PerspectiveCamera before entering AR mode resolves it, just as you described.

The interaction between a PerspectiveCamera entity in the hierarchy and content.camera = .spatialTracking isn't documented — the API doesn't prohibit having both, but it also doesn't describe what should happen when they coexist. At minimum, the framework should produce a diagnostic warning when it detects the conflict rather than silently falling back to screen-anchored behavior.

Please file a feedback report through Feedback Assistant with your test project and post the feedback number here. Your workaround — removing the PerspectiveCamera before entering AR mode and restoring it when leaving — is the right approach in the meantime.

0