After updating to visionOS 27.0, WorldTrackingProvider.allAnchors returns WorldAnchor.originFromAnchorTransform values that do not match what anchorUpdates delivers for the same anchor, read at the same moment, both isTracked == true, with no .updated event in between. The difference is a rigid frame change, not drift: Same vertical offset on every anchor: -1.173 m. Identical across two apps and two days with the head at 1.13 m and 1.22 m, so a constant, not head height. Horizontal deltas vary per anchor, but inter-anchor distances are preserved, so it is a yaw plus translation. Stable over time, same across launches. anchorUpdates matches the ImmersiveSpace frame RealityKit draws in. allAnchors does not. On visionOS 26 both agreed. Same anchor, same instant: allAnchors (0.1486, -0.7143, 0.2231) anchorUpdates (0.1316, 0.4590, -1.0955) device pose (0.0073, 1.2195, -0.0038) Four other anchors, delta = allAnchors - anchorUpdates: ( 2.6933, -1.1733, -0.2514) (-0.0810, -1.1733, 1.1339) ( 0.1878, -1.1733, 0.9891) ( 2.6289, -1.1733, -0.1809) In our app this showed up as a placed model vanishing after a scene switch: we re-pinned the model root from allAnchors, and it landed below the floor behind the user. A standalone sample (one provider, one anchor, a green sphere from anchorUpdates, a red sphere from allAnchors) shows two spheres on visionOS 27.0, the red one below the floor: added F21942C4-… at (-0.031, 0.712, -0.976) update added F21942C4-… tracked=true at (-0.031, 0.712, -0.976) MISMATCH delta (-0.211, -1.173, 1.444) m allAnchors (-0.243, -0.461, 0.468) anchorUpdates (-0.031, 0.712, -0.976) tracked=true device (0.000, 1.129, -0.001) Questions: Is allAnchors now intentionally expressed in a different frame on visionOS 27, for example related to the new ARKitCoordinateSpaceProviding / coordinateSpace(correction:) API? The documentation for allAnchors does not mention it. If so, which frame is it, and how should it be converted to the ImmersiveSpace frame? Workaround we use: read transforms only from anchorUpdates into a per-id cache and never position from allAnchors. Filed as FB24799304. Minimal repro project attached to the feedback.

As a 3D artist specializing in spatial installations, I faced the constant risk of designs looking balanced on screen but failing in physical proportions on the production floor. Starting with no traditional coding background, I embraced AI-assisted development and the CORE learning framework to master Swift, ARKit, and RealityKit. Real ini was built out of real practitioner necessity: turning the iPhone and iPad into on-site spatial validation studios that bridge digital art into physical reality. Download my app for free : https://apps.apple.com/id/app/real-ini/id6805716872

Summary A SpotLightComponent with SpotLightComponent.SurroundingsLight() (and a ProjectiveTexture) lights the flat part of my ceiling and the walls, but never the two sloped ceiling surfaces of a converted attic room, at any angle, intensity, attenuation radius or falloff. The lit area stops in a hard straight line exactly where the flat ceiling meets each slope, even though the beam clearly extends past that line. The sloped surfaces are fully present in SceneReconstructionProvider's mesh, and their faces point into the room. So this doesn't look like a scene understanding coverage problem. It looks as if physical space lighting only considers surfaces classified as horizontal ceiling, wall or floor, and skips slanted ones. Steps to reproduce Open a mixed immersive space in a room with a sloped ceiling (a converted attic, or any vaulted ceiling). Add an entity with SpotLightComponent(color: .white, intensity: 40_000, innerAngleInDegrees: 44.5, outerAngleInDegrees: 45, attenuationRadius: 10), SpotLightComponent.SurroundingsLight(), and a SpotLightComponent.ProjectiveTexture carrying a labelled grid texture so the beam's extent is visible. Place it near the floor under the wearer and aim it straight up. Run SceneReconstructionProvider alongside and draw each MeshAnchor as a wireframe (or as a solid material with faceCulling = .back) to confirm the mesh covers the slopes and faces inward. Expected The projected pattern conforms to every surface in the scene understanding mesh inside the cone, including the sloped ceiling, as it does for the flat ceiling and walls. Actual The pattern appears only on the flat ceiling strip and on the walls. Both slopes stay completely dark, with a hard straight cut at the crease. Tilting the light to aim directly at a slope, raising intensity to 200,000, and changing attenuation radius and falloff make no difference. The wireframe shows continuous mesh across the crease and onto the slopes; the back-face-culled solid view shows the slope faces pointing into the room. Screenshots attached: (1) the projected grid stopping at the crease, (2) the reconstruction wireframe covering the slopes, (3) the solid front-faces-only mesh showing the slopes are front-facing. (4) an additional image of the projected grid with both ceiling and walls in view Code let spot = Entity() spot.components.set(SpotLightComponent.SurroundingsLight()) var light = SpotLightComponent(color: .white, intensity: 40_000, innerAngleInDegrees: 44.5, outerAngleInDegrees: 45, attenuationRadius: 10) light.attenuationFalloffExponent = 0.5 spot.components.set(light) spot.components.set(SpotLightComponent.ProjectiveTexture(texture: gridTexture)) spot.look(at: from + SIMD3(0, 1, 0), from: from, upVector: SIMD3(0, 0, -1), relativeTo: nil) root.addChild(spot) Questions Is physical space lighting limited to horizontally and vertically classified surfaces by design? If so, could this be documented on SurroundingsLight, and is there a way to include slanted surfaces? If not by design, is there anything an app can do to have the light use the full reconstruction mesh?

Summary We have identified a reproducible world tracking drift regression in ARKit on LiDAR-equipped iOS devices running iOS 26.4 and later. A static virtual node anchored at the world origin visually drifts from its initial position as the user moves around a real-world scene, despite the scene remaining physically static. The same code produces stable, drift-free results on non-LiDAR devices running identical OS versions. Device & OS Observations Testing was performed across four devices on iOS 26.4 using the same application build and ARWorldTrackingConfiguration settings. Non-LiDAR devices — iPhone 14 and iPhone 15 — produced stable, drift-free tracking in all test runs. No world origin displacement was observed regardless of how long or how far the user walked. LiDAR-equipped devices — iPhone 14 Pro and iPhone 16 Pro — exhibited consistent, reproducible drift. A static node placed at the world origin visually shifted from its initial position as the user moved through the scene. The same devices were stable on earlier iOS versions, confirming this is a regression introduced in iOS 26.4. Technical Observations Nature of drift: A SCNNode placed statically at the ARKit world origin (SCNVector3(0, 0, 0)) visually displaces from its original position as the user walks around a static real-world scene. The displacement is not random — it accumulates directionally as the user moves, consistent with a sensor fusion or coordinate anchoring error. Trigger condition: The drift occurs during normal walking motion around a fixed point of interest, such as circling a parked vehicle. It does not appear when the device is held still. LiDAR specificity: The drift is exclusive to devices with a LiDAR scanner. Identical hardware configurations — same iOS build, same ARWorldTrackingConfiguration settings — on non-LiDAR devices produce no drift whatsoever. This isolates the regression to the LiDAR sensor's contribution to ARKit's internal Visual-Inertial Odometry (VIO) fusion pipeline. No API-level workaround found: There is currently no public ARKit API to selectively disable the LiDAR scanner's contribution to VIO. All available configuration-level options have been evaluated without resolving the drift. ARWorldTrackingConfiguration Options Evaluated The following configuration changes were applied individually and in combination. None resolved the drift on LiDAR devices: isAutoFocusEnabled = false — No improvement videoHDRAllowed = false (disabled) — No improvement planeDetection = [] (disabled) — No improvement sceneReconstruction = .mesh — No improvement worldAlignment: .gravity vs .gravityAndHeading — No improvement Minimal Reproduction Case The drift can be reproduced with a minimal ARKit scene: Create an ARSCNView with ARWorldTrackingConfiguration using default settings. Add a single static SCNNode (e.g., a small sphere or axes geometry) at SCNVector3(0, 0, 0) when the session starts. Run the app on a LiDAR-equipped device (iPhone Pro, iPad Pro with LiDAR) on iOS 26.4 or later. Walk in a circle around the node's approximate real-world position. Expected: The node remains visually fixed at its world position throughout the walkthrough. Actual: The node drifts from its initial position, increasingly displaced from its world origin anchor as walking continues. We want to know if Apple has made any internal updates to ARKit, particularly after the OS 26.4 upgrade. Thanks!

[PhotogrammetrySample]) with objectMask set and traps on my ios26.5.2, see the attached screenshot on feedback FB24379913 , it gets to the function and hangs . Even the folder reconstruction with lazy sequence as recommended from your video sample also doesn;t complete as it can't find alignment and displays the CoreOC.PhotogrammetrySession.Error error 6 means alignment failed. What can be done or do you guys expose any functions that can be used to check or trace or handle these internally The ObjectMasks are actually segmentation masks from an segmentation algorithm I will appreciate a timely response and willing to provide more clarity and informations, thank you so much for your understanding

Xcode hangs when I call PhotogrammetrySession(input: [PhotogrammetrySample]) with objectMask set and traps on some devices, see the attached screenshot, it gets to the function and hangs. Even the folder reconstruction also doesn't complete as it can't find alignment and displays the CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. In this case it failed while object masking was ON, so RealityKit could not find enough consistent feature tracks inside the masked pixels across the image set. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing. I will appreciate a timely response and willing to provide more clarity and informations, thank you so much for your understanding

I am implementing a custom generic spatial accessory according to Section 20 of the Accessory Design Guidelines dated 2026-06-08. Apple Vision Pro discovers the accessory, completes BLE pairing and encryption, reads the Apple Spatial Accessory Version and Uniform Type Identifiers characteristics, and recognizes the device in the accessory tracking debug view. However, visionOS writes ASA protocol version 00.00 to the Apple Spatial Accessory Host Version characteristic instead of selecting version 01.01. Here is the log message when I connected my spatial accessory with apple vision pro with os27 beta 3. The error is the ASA part. BLE advertising: ZemScannerASA15; BLE ATT MTU tx=23,rx=23; BLE conn timing connected: interval=30000; us,latency=0,timeout=72; TimeSync anchor report enabled: h=0x011f; BLE connected TimeSync anchor #1 ce=0,int=30000,delay=0 TimeSync anchor #2 ce=3,int=30000,delay=0 TimeSync anchor #3 ce=4,int=30000,delay=0 BLE ATT MTU tx=293,rx=65 BLE security level 2 BLE pairing complete, bonded ASA version read #1 off=0 req=64 hex=02 01 01 01 01 ASA UTI read #2 off=0 req=64 total=49 ASA host wr #3 o=0 l=4 f=00 hex=00 00 01 31 ASA host raw: off=0,write=4,total=4,hex=00 00 01 31 ASA host version: >len=4,ver=0.0,features_len=1,features=0x31 ASA host selected no common protocol HID direct input report sent HID notif #4 report=3 enabled HID IMU stream started HID notif #5 report=2 enabled HID IMU report sent BLE conn params updated: interval=7500 us,latency=4,timeout=100

I have a few questions about the recommended approach: Should a large area like this be scanned in a single RoomCaptureSession, or is it recommended to split it into multiple sessions? If multiple sessions are preferred, should the decision be based on the device's available RAM/memory or the size of the scanned area (square footage)? Is there a recommended maximum square footage, number of rooms, or session duration for a single RoomCaptureSession? Are there any documented limitations or best practices for handling large-area scans with RoomPlan? I'm looking for the recommended approach from the RoomPlan/ARKit developer for managing large scans efficiently.

Hello, I'm a VTuber, or a content creator who uses a virtual avatar instead of my real life face for presentation. I've been trying to use the iOS implementation of ARKit to get face tracking working, but noticed a major issue, especially for my purposes. ARKit appears to move both eyebrows the same, regardless of whether or not I am physically moving my eyebrows the same. Every ARKit based application I've used simply mirrors eyebrow movement. Is there a fix to this, or is this just a bug currently in the software?

Bug: When SpatialTrackingSession and ARKitSession + WorldTrackingProvider are running concurrently, any WorldAnchor added via WorldTrackingProvider.addAnchor() triggers .added followed immediately by .removed—without any user call to removeAnchor(). The anchor never persists in allAnchors. import SwiftUI import RealityKit import ARKit struct ImmersiveView: View { @State private var worldTracking: WorldTrackingProvider? @State private var arSession: ARKitSession? @State private var processWorldTrackingUpdatesTask: Task<Void, Never>? var body: some View { RealityView { content in let configuration = SpatialTrackingSession.Configuration(tracking: [.world]) if let unavailableCapabilities = await SpatialTrackingSession().run(configuration) { if unavailableCapabilities.anchor.contains(.world) { fatalError("World tracking is not available on this device.") } } let worldTracking = WorldTrackingProvider() let arSession = ARKitSession() self.arSession = arSession try! await arSession.run([worldTracking]) self.worldTracking = worldTracking processWorldTrackingUpdatesTask = Task { @MainActor [weak worldTracking] in guard let worldTracking else { return } for await update in worldTracking.anchorUpdates { let worldAnchor = update.anchor switch update.event { case .added: print("Anchor added: \(worldAnchor.id)") case .updated: print("Anchor updated: \(worldAnchor.id)") case .removed: fatalError("Anchor removed unexpectedly — this should not happen in this demo scenario.") } } } } .task { try? await Task.sleep(for: .seconds(3)) guard let worldTracking else { return } do { try await worldTracking.addAnchor(.init(originFromAnchorTransform: Transform.identity.matrix)) } catch { print("Error adding anchor: \(error)") } } } } Expected: Anchors persist until explicitly removed or out of range. Actual: SpatialTrackingSession interferes with WorldTrackingProvider's anchor lifecycle, causing immediate removal. This was originally reported in 2025 (https://developer.apple.com/forums/thread/773351) , but remains unfixed in visionOS 27.0 beta. I've re-filed as FB23420195.

Kind folks, Was all setup for online access to several Q&A sessions at this WWDC, even posted some questions. But other priorities on my project prevented me from attending. Did any of you hear any news on direction finding with 3rd party devices at this year's WWDC? My current understanding is that (for iPhones later than 15) the NI framework returns null for direction. Apple recommends use of the ARkit in conjunction with NI (tech session way back in WWDC22!) FYI, I've been trying hard to get any information on this topic. Would be happy to share what public information/speculation I've found (including how Apple get's such good range and direction info in their Find My app for their AirTag 2 😉). Perhaps we can set up a small discussion group for sharing insights and experiences with emerging UWB technology. Comments appreciated. Mike

Providing a response and feedback to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 I have a created a feedback/bug report with ID FB22823733 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing.

Dear Apple Developer Team, I would like to ask a few questions related to RoomPlan, Room API, USDZ export, and possible future spatial scanning workflows: Can the RoomPlan or Room API load an existing USDZ room model and compare it in real time with a new live scan of the same space? Is Apple considering extending RoomPlan beyond indoor rooms, for example to scan outdoor areas, house exteriors, terrain, and simple building volumes with rough dimensions? Also in indoor to scan slopes with angle room for rooms in roof floor? Will RoomPlan support multi-floor continuous scanning, custom object detection elements, and reliable export of the scanned result to USDZ for further CAD/BIM workflows? Is it possible to create a real-time comparator between a reference object in USDZ or STEP format and a physical object being scanned live, so that deviations in geometry or dimensions can be detected during scanning? Best regards, Ivo Saina

Reconstruction with Object Mask in PhotogrammetrySession Fails and doesn't complete because of alignment and object masks with thing segmentation even after increasing the mask region . Also the PhotogrammetrySample hangs with EXC_BREAKPOINTwhen I usePhotogrammetrySession(input: [PhotogrammetrySample])withobjectMask set and the Apple Developer Forums report crashes (“HelloPhotogrammetry” / sample+mask), and (EXC_BREAKPOINT` was being observed at ~4 GB) with freezing the execution and Xcode interface

We have developed an iPad application using the ARCL (AR + CoreLocation) library to render Point of Interest (POI) annotations in both an AR view and a standard MapView. The application performs as expected on standard devices. However, we have some iPad covered with strong magnet. This creates significant magnetic interference, resulting in a 90° to 180° heading offset, rendering the AR POI placement and MapView orientation unusable. Technical Challenges & Constraints: Hardware Lock: The magnetic cover is a mandatory business requirement and cannot be removed during field use. Sensor Failure: The internal magnetometer cannot provide an accurate North reference due to the proximity of the cover’s magnets. While CoreLocation and CoreMotion use sensor fusion, the magnetometer remains the primary source for absolute heading. Alternative Orientation Tracking: Is there a documented method to bypass the magnetometer and derive device orientation using only the Gyroscope and Accelerometer (e.g., relative tracking) while still maintaining alignment with geographic coordinates in CoreLocation? Programmatic Offsets: Are there known APIs or mathematical workarounds to programmatically "nullify" or offset a constant magnetic bias once the device is inside the cover? so we can use that offset for ARView and in Mapview as well.

I want to: Run ARKit on the main rear camera, and while it's running shoot high resolution pictures on the wide camera, without disturbing the AR tracking. Is this possible?

I’ve seen, mainly in discussions with AIs, that ARFaceTrackingConfiguration uses the same technology as Face ID and therefore should work in complete darkness. However, I haven’t been able to achieve this. Does anyone know if this is actually true? I'm using an iPhone 16 to test, and the Face ID works well in darkness.

We are building an ARKit application targeting iPhone Pro devices on iOS 26.4+. During testing, we consistently observe the following log message at a specific point in our AR session (approximately 90° into a walking arc around a vehicle): ARWorldTrackingTechnique: resource constraints [33] What specifically does resource constraint code [33] represent? Is it a VIO constraint type, a sensor fusion budget limit, or something else entirely? How does it differ from code [1] (which appears to be a general CPU starvation signal for the ARKit sensor fusion thread)? Is there a public API to detect or respond to this specific constraint condition, short of waiting for a full ARCamera.TrackingState downgrade? Was this constraint type introduced or changed in iOS 26.4?

I'm trying to track identified objects in realtime with bounding rects, with no 3D integration, but still has poor update performance. I'm trying to understand how to optimize frame updates. (I'm a new programmer) Using: Foundation, AVFoundation, ARKit, CoreVideo, Vision, CoreImage, CoreVideo with YOLOE-11s object detection currently throttled to 2fps. (target iOS, testing on 16pro) YOLOE-11S CoreML model detects objects with class labels + bounding boxes Labels are matched against ObjectCatalog.json for relevance Matched objects are promoted from blue (detected) to green (identified) Log warnings: ARSession <0x110afdb80>: The delegate of ARSession is retaining 13 ARFrames. The camera will stop delivering camera images if the delegate keeps holding on to too many ARFrames. This could be a threading or memory management issue in the delegate and should be fixed. Skipping integration due to poor slam at time: 619447.208339 vio_initialized(1) map_size(0) tracking_state_is_nominal(0) is_3dof(0) reinitialize_attempts(6) slam_mode(RegularSLAM)

I’m exploring a visionOS interaction pattern where nearby Apple Vision Pro users have opted in ahead of time, such as when entering a shared venue, and a physical gesture like a handshake triggers a lightweight exchange of user-approved information between their devices. I’m not asking about identifying strangers, accessing raw camera data, or tracking another person without consent. I’m trying to understand the most automatic Apple-supported architecture for this kind of privacy-preserving nearby interaction. What is the recommended approach on visionOS? Specifically: Can a visionOS app use ARKit hand tracking to detect the current user’s own gesture, then combine that with nearby peer discovery or ranging through Nearby Interaction, Network framework, Multipeer Connectivity, Bluetooth, Wi-Fi, or another supported API? Is there a supported way for two nearby Vision Pro devices to exchange a small user-approved payload after prior opt-in, without requiring users to manually start SharePlay or confirm every individual exchange? If SharePlay or Group Activities is the recommended path for shared spatial context, is there a supported alternative for venue-scale or multi-user interactions that should not be limited to a small active SharePlay group? What are the privacy and App Review boundaries for this pattern? Should developers assume the app cannot identify nearby people, observe another person’s body or hands, or trigger an exchange unless both users have explicit opt-in and clear awareness? If a mostly passive gesture-triggered exchange is not supported today, what is the closest Apple-recommended design pattern?