Posts under Spatial Computing topic

Post

Replies

Boosts

Views

Activity

visionOS 27: allAnchors and anchorUpdates disagree on WorldAnchor transforms (different coordinate frame)
After updating to visionOS 27.0, WorldTrackingProvider.allAnchors returns WorldAnchor.originFromAnchorTransform values that do not match what anchorUpdates delivers for the same anchor, read at the same moment, both isTracked == true, with no .updated event in between. The difference is a rigid frame change, not drift: Same vertical offset on every anchor: -1.173 m. Identical across two apps and two days with the head at 1.13 m and 1.22 m, so a constant, not head height. Horizontal deltas vary per anchor, but inter-anchor distances are preserved, so it is a yaw plus translation. Stable over time, same across launches. anchorUpdates matches the ImmersiveSpace frame RealityKit draws in. allAnchors does not. On visionOS 26 both agreed. Same anchor, same instant: allAnchors (0.1486, -0.7143, 0.2231) anchorUpdates (0.1316, 0.4590, -1.0955) device pose (0.0073, 1.2195, -0.0038) Four other anchors, delta = allAnchors - anchorUpdates: ( 2.6933, -1.1733, -0.2514) (-0.0810, -1.1733, 1.1339) ( 0.1878, -1.1733, 0.9891) ( 2.6289, -1.1733, -0.1809) In our app this showed up as a placed model vanishing after a scene switch: we re-pinned the model root from allAnchors, and it landed below the floor behind the user. A standalone sample (one provider, one anchor, a green sphere from anchorUpdates, a red sphere from allAnchors) shows two spheres on visionOS 27.0, the red one below the floor: added F21942C4-… at (-0.031, 0.712, -0.976) update added F21942C4-… tracked=true at (-0.031, 0.712, -0.976) MISMATCH delta (-0.211, -1.173, 1.444) m allAnchors (-0.243, -0.461, 0.468) anchorUpdates (-0.031, 0.712, -0.976) tracked=true device (0.000, 1.129, -0.001) Questions: Is allAnchors now intentionally expressed in a different frame on visionOS 27, for example related to the new ARKitCoordinateSpaceProviding / coordinateSpace(correction:) API? The documentation for allAnchors does not mention it. If so, which frame is it, and how should it be converted to the ImmersiveSpace frame? Workaround we use: read transforms only from anchorUpdates into a per-id cache and never position from allAnchors. Filed as FB24799304. Minimal repro project attached to the feedback.
0
0
22
3h
Camera specs of world-facing tracking cameras.
We are working on a very space constrained custom spatial accessory. Since the form factor of the device is small, we need to add small LEDs producing small blob diameters with accordingly small distance between them. To better estimate the amount, size, and distance of the LEDs it would help to know which image sensors and field of view the six world‑facing tracking cameras have.
1
0
721
13h
Update entity state across devices through SharePlay on VisionOS
I am trying to create an app where I have multiple users modifying an entity at runtime similar to collaborative digital sculpting using SharePlay on VisionOS. When creating a collaborative experience using other 3D engines I have utilized an external server that devices send and update their state from serving as a global "source of truth". This doesn't seem to be the intended workflow for SharePlay which seemingly only allows for device to device messaging. While this works for a variety of apps it doesn't provide a smooth experience when multiple users are interacting with and manipulating the same 3D entity. Is there an intended way to do this for SharePlay? Would the best practice be to have one user serve as a host and establish a "source of truth" based on their app state? The closest example in documentation I have found is an example for using SharePlay in a drawing app. When a user is drawing they send the information for their realtime stroke through unreliable SharePlay messages and then, when they've finished, they send a reliable message with the final line. This technique makes sense for the given example since there is no scenario where multiple users would be drawing the same line, but it seemingly falls short for my use case since multiple users can sculpt in the same area of a given mesh. What would the intended approach be to achieve this with SharePlay? Is it simply a limitation of this type of networking? Thanks!
1
0
1.1k
14h
RealityKit cloth: reducing mesh collider air gap without penetration
I’m testing RealityKit cloth for garment-like cloth over an authored human-shaped triangle-mesh collider. The issue also occurs with a static collider, so it does not appear to depend on character animation.. Contact is stable and runs interactively, but I cannot achieve close surface contact without sacrificing robustness. The cloth mid-surface remains visibly several millimetres from the authored collider, especially around curved or narrow areas such as the head, arms, and hands. Transparent rendering of visualMesh confirms this is not just render thickness. Reducing ClothMeshShape.bias reduces the gap, but eventually causes interpenetration around difficult geometry. Increasing it prevents penetration but makes the cloth visibly float. I’ve also checked scale, winding, normals, reasonably even triangles, timestep, and solver iterations. I want to avoid inflating the collider or adding anatomical proxy shapes. Is bias the only physical-clearance control, or does RealityKit apply an additional contact radius derived from simulation-mesh resolution? Is local conforming refinement the recommended way to achieve roughly 1–2 mm clearance while retaining robust real-time contact? Are there any specific topology requirements beyond evenly sized triangles? I can provide a minimal reproduction and screenshots if helpful although not sure I can share screenshots here?
3
0
3.1k
16h
Questions about RoomPlan, Room API, USDZ/STEP comparison, and extended spatial scanning
Dear Apple Developer Team, I would like to ask a few questions related to RoomPlan, Room API, USDZ export, and possible future spatial scanning workflows: Can the RoomPlan or Room API load an existing USDZ room model and compare it in real time with a new live scan of the same space? Is Apple considering extending RoomPlan beyond indoor rooms, for example to scan outdoor areas, house exteriors, terrain, and simple building volumes with rough dimensions? Will RoomPlan support multi-floor continuous scanning, custom object detection elements, and reliable export of the scanned result to USDZ for further CAD/BIM workflows? Is it possible to create a real-time comparator between a reference object in USDZ or STEP format and a physical object being scanned live, so that deviations in geometry or dimensions can be detected during scanning? Best regards, Ivo Saina
4
0
836
1d
Is there any technical documentation available for the iPhone’s LiDAR scanner?
Hi, I’m looking to develop an app that uses the iPhone’s LiDAR scanner and am trying to decide which model to buy. Is there any relevant technical documentation available on its capabilities, such as accuracy, resolution and range? I’d also like to understand whether there have been any improvements between generations, particularly with the iPhone 18 Pro. Any documentation or pointers in the right direction would be appreciated. Thanks!
0
0
246
1d
Best practice for connecting Mac-side USDZ preparation with exact-asset Vision Pro verification?
I’m developing two complementary developer tools: Reality Prep Pro on macOS and Reality Prep Preview on visionOS. The workflow I’m trying to solve is: Mac: prepare and validate a USDZ Vision Pro: verify that exact USDZ on-device Mac: bring the device evidence back and associate it with the exact asset revision I currently use the USDZ’s SHA-256 as the authoritative identity. If the asset bytes later change, the previous Vision Pro evidence is treated as stale and a re-test is recommended. Existing USDZ→USDZ preparation is package-preserving, so non-texture payloads such as animation/rigging data are not unnecessarily re-authored. I’d be interested in hearing from other RealityKit / Reality Composer Pro developers: Is this similar to how you currently handle asset provenance between desktop preparation and real-device testing? Are there any Apple frameworks or established visionOS practices you would recommend for strengthening this kind of revision-to-device-evidence workflow?
0
0
457
3d
Generic spatial accessory receives LED reports but never produces an AccessoryAnchor
We are developing a generic spatial accessory for Apple Vision Pro using the “Working with generic spatial accessories” sample and a trained .referenceaccessory file. Environment: visionOS: [version/build] Xcode: [version/build] Hardware: Apple Vision Pro + nRF52840-based BLE accessory IMU: 6-axis IMU, 400 Hz HID reports LED constellation: 8 x 850 nm IR LEDs, driven through MOSFETs The BLE/HID and timing path appears to be working: The accessory connects successfully over BLE. ASA/HID enumeration succeeds. The firmware receives continuous LED Output Reports from visionOS. The Accessory Tracking Clock reaches READY. PWM scheduling reports no late events and no queue overflow. In the Accessory Tracking Debug View, multiple independent IR LED blobs are visible and stable at approximately 0.7 m. At least 4 LEDs are visible from normal viewing angles. The app resolves the .referenceaccessory and starts AccessoryTrackingProvider. However, ARKit never produces an AccessoryAnchor. We also have no diagnostic signal indicating whether the failure occurs at: LED candidate-point detection, constellation matching against the .referenceaccessory, or IMU/optical fusion. Questions: Is there a supported diagnostic API, log category, or Debug View indicator that distinguishes candidate-point rejection from constellation-match or fusion rejection? Are there known acquisition prerequisites in the current visionOS beta beyond the published Accessory Design Guidelines? Can Apple recommend a minimal validation procedure using the generic spatial accessory sample to confirm that the LED constellation is accepted by the tracking system? Is this expected to work in a volumetric window, or are there additional lifecycle/session requirements that can prevent initial acquisition? I can provide sanitized HID LED-report logs, timing data, Debug View captures, and a minimal reproducible project if useful.
10
0
2.8k
6d
Physical space lighting (SurroundingsLight) never lands on sloped ceiling surfaces that are present in the scene reconstruction mesh
Summary A SpotLightComponent with SpotLightComponent.SurroundingsLight() (and a ProjectiveTexture) lights the flat part of my ceiling and the walls, but never the two sloped ceiling surfaces of a converted attic room, at any angle, intensity, attenuation radius or falloff. The lit area stops in a hard straight line exactly where the flat ceiling meets each slope, even though the beam clearly extends past that line. The sloped surfaces are fully present in SceneReconstructionProvider's mesh, and their faces point into the room. So this doesn't look like a scene understanding coverage problem. It looks as if physical space lighting only considers surfaces classified as horizontal ceiling, wall or floor, and skips slanted ones. Steps to reproduce Open a mixed immersive space in a room with a sloped ceiling (a converted attic, or any vaulted ceiling). Add an entity with SpotLightComponent(color: .white, intensity: 40_000, innerAngleInDegrees: 44.5, outerAngleInDegrees: 45, attenuationRadius: 10), SpotLightComponent.SurroundingsLight(), and a SpotLightComponent.ProjectiveTexture carrying a labelled grid texture so the beam's extent is visible. Place it near the floor under the wearer and aim it straight up. Run SceneReconstructionProvider alongside and draw each MeshAnchor as a wireframe (or as a solid material with faceCulling = .back) to confirm the mesh covers the slopes and faces inward. Expected The projected pattern conforms to every surface in the scene understanding mesh inside the cone, including the sloped ceiling, as it does for the flat ceiling and walls. Actual The pattern appears only on the flat ceiling strip and on the walls. Both slopes stay completely dark, with a hard straight cut at the crease. Tilting the light to aim directly at a slope, raising intensity to 200,000, and changing attenuation radius and falloff make no difference. The wireframe shows continuous mesh across the crease and onto the slopes; the back-face-culled solid view shows the slope faces pointing into the room. Screenshots attached: (1) the projected grid stopping at the crease, (2) the reconstruction wireframe covering the slopes, (3) the solid front-faces-only mesh showing the slopes are front-facing. (4) an additional image of the projected grid with both ceiling and walls in view Code let spot = Entity() spot.components.set(SpotLightComponent.SurroundingsLight()) var light = SpotLightComponent(color: .white, intensity: 40_000, innerAngleInDegrees: 44.5, outerAngleInDegrees: 45, attenuationRadius: 10) light.attenuationFalloffExponent = 0.5 spot.components.set(light) spot.components.set(SpotLightComponent.ProjectiveTexture(texture: gridTexture)) spot.look(at: from + SIMD3(0, 1, 0), from: from, upVector: SIMD3(0, 0, -1), relativeTo: nil) root.addChild(spot) Questions Is physical space lighting limited to horizontally and vertically classified surfaces by design? If so, could this be documented on SurroundingsLight, and is there a way to include slanted surfaces? If not by design, is there anything an app can do to have the light use the full reconstruction mesh?
2
0
1.1k
1w
Is there currently a way to import 3D Gaussian Splatting data into Reality Composer Pro?
Hello, As far as I understand, RealityKit now supports 3D Gaussian Splatting through GaussianSplatComponent. I would like to ask whether there is currently a way to use 3D Gaussian Splatting data directly in Reality Composer Pro, for example by importing a .ply file containing Gaussian splats. More specifically: Can Reality Composer Pro import a 3D Gaussian Splatting .ply file? Is there a way to create or assign a GaussianSplatComponent to an entity directly within Reality Composer Pro? If this is not currently supported, is there any recommended workflow for creating a .usdz containing Gaussian Splatting data that can be opened in the Preview app on iPhone/iPad and displayed as an AR object? I tried opening the sample plant.usdz on an iPhone using the Preview app. However, the AR preview appears to be empty, so it seems that the Preview app may not currently support rendering the Gaussian Splatting content contained in that USDZ. My desired workflow would be: Gaussian Splatting .ply → Reality Composer Pro / USDZ → iPhone Preview → AR Is this workflow currently possible? If not, is there an official or recommended way to package a GaussianSplatComponent and its associated Gaussian splat data into a USD/USDZ file for AR preview on iOS? If the above functionality is not currently supported, does the development team have any plans to add or improve support for these features in future versions of Reality Composer Pro or the Preview app? Any clarification regarding the current capabilities, limitations, or future plans would be greatly appreciated. Thank you!
3
0
2.3k
1w
Anchoring Invalid
Hello. I trained an ObjectTracker.referenceobject using ML, and am using it in RCP. After adding Anchoring to the entity, I set the Target mode to Object and applied the trained model data ObjectTracker.referenceobject. However, there is no tracking effect after building for VisionPro. Are any permissions required? What should I do?
3
0
2.5k
1w
PresentationComponent loses gaze targeting or dismisses in a volumetric window
On visionOS 26.6 and 27 latest beta: PresentationComponent in a volumetric WindowGroup, when looking at the volume from a specific lower/back region create two bugs: buttons inside the presentation have their gaze obstructed by hand presence, no button shows any hover highlight hand movement in that specific area auto dismisses the presentation The focused public repro uses a thin ModelEntity with InputTargetComponent and ordinary SwiftUI buttons. Its README includes two clearer videos: https://github.com/mesqueeb/PresentationAutoCloseInBackOfVolume 1. Gaze obstructed on buttons when hand is at specific location (see "Test Action" has no hover highlight, even though i look at it) 2. Hover state restored on buttons when hand is gone (see "Test Action" has hover highlight) 3. Presentation auto closed when hand comes back The weirdest part: This only happens at a specific location, when walking to the back of the volume, then turning around and interacting with the volume from the lower right back side. Any workaround or fix guidance would be appreciated. Feedback Assistant: FB24569418.
2
0
1.3k
2w
Configure AI Provider via Web login
In Xcode it's extremely convenient to be able to sign into AI providers by having an authentication workflow opened in a browser. Reality Composer Pro 3 Beta 5 requires you to manually configure the AI provider (Claude for example). Please provide the Web-based authentication workflow in Reality Composer Pro that is also available in Xcode.
2
0
1.2k
2w
Request for better Gaussian Splat examples
I noticed that there is a Level of Detail component in beta in RealityKit, along with the new GaussianSplatResource component. Apple Maps 3D clearly uses some combination of these things, in addition to some form of streaming and compression most likely, in order to make performance acceptable. It would be nice if Apple could give us some examples of combining file compression + LevelofDetail + GaussianSplatResource in an example project. This is what will be required to make any kind of immersive 3D walkthroughs possible with these frameworks. I just randomly stumbled upon the LevelOfDetail component bc it was mentioned in a Google AI response, but otherwise there seem to be no examples that use it with respect to GaussianSplatResource (and so it’s not clear if it’s meant to be used in this way)
0
0
679
2w
ARKit World Tracking Drift Regression on LiDAR-Equipped Devices - iOS 26.4+
Summary We have identified a reproducible world tracking drift regression in ARKit on LiDAR-equipped iOS devices running iOS 26.4 and later. A static virtual node anchored at the world origin visually drifts from its initial position as the user moves around a real-world scene, despite the scene remaining physically static. The same code produces stable, drift-free results on non-LiDAR devices running identical OS versions. Device & OS Observations Testing was performed across four devices on iOS 26.4 using the same application build and ARWorldTrackingConfiguration settings. Non-LiDAR devices — iPhone 14 and iPhone 15 — produced stable, drift-free tracking in all test runs. No world origin displacement was observed regardless of how long or how far the user walked. LiDAR-equipped devices — iPhone 14 Pro and iPhone 16 Pro — exhibited consistent, reproducible drift. A static node placed at the world origin visually shifted from its initial position as the user moved through the scene. The same devices were stable on earlier iOS versions, confirming this is a regression introduced in iOS 26.4. Technical Observations Nature of drift: A SCNNode placed statically at the ARKit world origin (SCNVector3(0, 0, 0)) visually displaces from its original position as the user walks around a static real-world scene. The displacement is not random — it accumulates directionally as the user moves, consistent with a sensor fusion or coordinate anchoring error. Trigger condition: The drift occurs during normal walking motion around a fixed point of interest, such as circling a parked vehicle. It does not appear when the device is held still. LiDAR specificity: The drift is exclusive to devices with a LiDAR scanner. Identical hardware configurations — same iOS build, same ARWorldTrackingConfiguration settings — on non-LiDAR devices produce no drift whatsoever. This isolates the regression to the LiDAR sensor's contribution to ARKit's internal Visual-Inertial Odometry (VIO) fusion pipeline. No API-level workaround found: There is currently no public ARKit API to selectively disable the LiDAR scanner's contribution to VIO. All available configuration-level options have been evaluated without resolving the drift. ARWorldTrackingConfiguration Options Evaluated The following configuration changes were applied individually and in combination. None resolved the drift on LiDAR devices: isAutoFocusEnabled = false — No improvement videoHDRAllowed = false (disabled) — No improvement planeDetection = [] (disabled) — No improvement sceneReconstruction = .mesh — No improvement worldAlignment: .gravity vs .gravityAndHeading — No improvement Minimal Reproduction Case The drift can be reproduced with a minimal ARKit scene: Create an ARSCNView with ARWorldTrackingConfiguration using default settings. Add a single static SCNNode (e.g., a small sphere or axes geometry) at SCNVector3(0, 0, 0) when the session starts. Run the app on a LiDAR-equipped device (iPhone Pro, iPad Pro with LiDAR) on iOS 26.4 or later. Walk in a circle around the node's approximate real-world position. Expected: The node remains visually fixed at its world position throughout the walkthrough. Actual: The node drifts from its initial position, increasingly displaced from its world origin anchor as walking continues. We want to know if Apple has made any internal updates to ARKit, particularly after the OS 26.4 upgrade. Thanks!
6
8
3.4k
2w
RCP 3 Beta 2–4: Create/Open silently fails before editor launch
Reality Composer Pro 3 cannot create a new project or open an existing project on my Mac. After I select Create New Project or Open Existing Project, the operation begins but the project editor never appears. The Reality Composer Pro process remains alive and idle, and no standard crash report is generated. Environment: MacBook Pro Mac15,9 Apple M3 Max, 48 GB unified memory macOS 26.6 (25G72) Reality Composer Pro 3 Beta 4, version 3.080.0.1.500.1 Built-in display only Steps to reproduce: Quit Reality Composer Pro. Launch /Applications/RealityComposerPro.app. Select Create New Project. Observe that the editor never appears. Open Existing Project fails in the same way. The problem is reproducible 100% of the time with Reality Composer Pro 3 Beta 2, Beta 3, and Beta 4. I have also reproduced it: With a completely blank new project With an existing project In a clean local macOS user account In Safe Mode With no external displays With local paths and iCloud disabled After verifying adequate free disk space and separately testing free-space conditions After resetting Reality Composer Pro state and preferences The application passes code-signature and Gatekeeper verification. Its temporary loader copy is created successfully, with no ENOSPC, failed-copy, or failed-write error. The failure occurs around libtm-welcome_window.dylib and libtm-plugin_support.dylib prepareToLoad. A small modal NSPanel repeatedly performs AppKit/SwiftUI constraint updates and reports: The window has been marked as needing another Update Constraints in Window pass, but it has already had more Update Constraints in Window passes than there are views in the window. The exception appears to be swallowed instead of crashing the process. A process sample then reports SOME_OTHER_THREAD_SWALLOWED_AT_LEAST_ONE_EXCEPTION while the main thread remains idle in the normal NSApplication event loop. This appears to be a constraint-update loop in the welcome-window loader/progress panel, rather than a problem with a particular project or its assets. Feedback Assistant report: FB24460905 Has anyone reproduced this on macOS 26 with Reality Composer Pro 3? Is there a supported workaround that bypasses the failing loader/progress panel? Is RCP 3 Beta 4 expected to work on macOS 26.6, or does it require macOS 27?Reality Composer Pro 3 cannot create a new project or open an existing project on my Mac. After I select Create New Project or Open Existing Project, the operation begins but the project editor never appears. The Reality Composer Pro process remains alive and idle, and no standard crash report is generated. Environment: MacBook Pro Mac15,9 Apple M3 Max, 48 GB unified memory macOS 26.6 (25G72) Reality Composer Pro 3 Beta 4, version 3.080.0.1.500.1 Built-in display only Steps to reproduce: Quit Reality Composer Pro. Launch /Applications/RealityComposerPro.app. Select Create New Project. Observe that the editor never appears. Open Existing Project fails in the same way. The problem is reproducible 100% of the time with Reality Composer Pro 3 Beta 2, Beta 3, and Beta 4. I have also reproduced it: With a completely blank new project With an existing project In a clean local macOS user account In Safe Mode With no external displays With local paths and iCloud disabled After verifying adequate free disk space and separately testing free-space conditions After resetting Reality Composer Pro state and preferences The application passes code-signature and Gatekeeper verification. Its temporary loader copy is created successfully, with no ENOSPC, failed-copy, or failed-write error. The failure occurs around libtm-welcome_window.dylib and libtm-plugin_support.dylib prepareToLoad. A small modal NSPanel repeatedly performs AppKit/SwiftUI constraint updates and reports: The window has been marked as needing another Update Constraints in Window pass, but it has already had more Update Constraints in Window passes than there are views in the window. The exception appears to be swallowed instead of crashing the process. A process sample then reports SOME_OTHER_THREAD_SWALLOWED_AT_LEAST_ONE_EXCEPTION while the main thread remains idle in the normal NSApplication event loop. This appears to be a constraint-update loop in the welcome-window loader/progress panel, rather than a problem with a particular project or its assets. Feedback Assistant report: FB24460905 Has anyone reproduced this on macOS 26 with Reality Composer Pro 3? Is there a supported workaround that bypasses the failing loader/progress panel? Is RCP 3 Beta 4 expected to work on macOS 26.6, or does it require macOS 27?
2
0
1.2k
2w
ARKit: Is there a refined historical camera trajectory available after world mapping?
Hi, I'm building an iOS LiDAR scanning app using ARWorldTrackingConfiguration with sceneReconstruction = .mesh. During scanning I save the RGB image, ARFrame.camera.transform, intrinsics and per-frame scene depth. At the end of the scan I also have the final ARMeshAnchor mesh. When I place the saved depth observations into the ARSession world using their original ARCamera.transform, they broadly form the same scene, but over a longer walk I can see several-centimetre discrepancies between observations of the same surfaces. My question is very specific: After ARKit has built its world map / reconstructed the environment, is there any public API that provides a globally refined or corrected historical pose for previously captured ARFrames, rather than the original ARCamera.transform recorded at each frame? In other words, is there an API or ARKit data structure from which I can obtain something equivalent to: frame timestamp → refined world-space camera transform after the session has accumulated more information? I am aware of ARWorldMap and worldMappingStatus, but I'm unclear whether the world map contains information that can be used to recover or refine the historical camera trajectory, or whether ARCamera.transform is the only historical pose available to an app. I'm not looking for advice on implementing ICP or my own SLAM/registration. I just want to know whether ARKit already exposes a refined trajectory or the underlying information needed to obtain one. Thanks.
6
0
1.5k
3w
OpacityComponent does not blend a GaussianSplatComponent on iOS/macOS, behaves correct on visionOS
I have a RealityKit scene with one container entity and two children: an entity carrying a GaussianSplatComponent, and a plain Entity with a ModelComponent (an unlit white cube). A single OpacityComponent goes on the container, so one value covers both children in the same frame. container.components.set(OpacityComponent(opacity: value)) On visionOS 27 this does exactly what I expect: both children fade together, continuously, across the whole range. On iOS 27 and Mac Catalyst 27 the cube fades continuously but the splat does not. It renders at full opacity for every value above 0 and disappears at exactly 0. At 0.01 the cube beside it is already almost invisible while the splat is still drawing at full strength, so entity opacity behaves as a visibility switch on a splat rather than as a blend factor. Things I have already ruled out: Not a propagation issue. Moving the OpacityComponent off the container and onto the splat's own entity (and the cube's own entity, to keep the A/B) changes nothing. Not the container failing to receive the value. The cube under the same component, in the same frame, fades correctly. Not the splat data. Per-splat alpha through GaussianSplatResource's opacity buffer is honoured, and projectionMode, sortingMode, scaleActivation and opacityActivation all take effect. Not a difference in scene construction per platform. The same file builds the container on all three platforms in my sample. Confirmed on: iOS 27.0 (24A5418b), iPhone 16 Pro — reproduces macOS 27.0 (26A5416b), Mac Catalyst — reproduces visionOS 27.0 (24M5326g), Apple Vision Pro — correct Filed as FB24431214 with a sample project that builds for all three from one target. Would be great to see this fixed!
1
0
449
3w
How does PhotogrammetrySession use LiDAR depth provided through PhotogrammetrySample?
We are developing a LiDAR-based spatial capture application using ARKit and RealityKit. We are investigating whether PhotogrammetrySession can be used as part of a capture pipeline where we collect: RGB frames LiDAR depth camera intrinsics camera transforms IMU/device motion data and then reconstruct a textured 3D model. The documentation states that PhotogrammetrySample can include depthDataMap, depthConfidenceMap, gravity, and camera metadata. Could someone clarify how PhotogrammetrySession uses the depth information when supplied? Specifically: Is LiDAR depth used during reconstruction to improve geometry (for example as a constraint during reconstruction), or is it primarily used for scale/orientation metadata? When camera transforms and intrinsics are available from ARKit, does PhotogrammetrySession consume this information, or does it independently estimate camera poses from the image sequence? Is the recommended Apple workflow for RGB + LiDAR capture: ARKit capture → PhotogrammetrySample → PhotogrammetrySession or is ObjectCaptureSession the intended capture pipeline for photogrammetry workflows? We are trying to understand the intended relationship between ARKit, ObjectCaptureSession, and PhotogrammetrySession before choosing an architecture. Thanks.
2
0
1.4k
4w
visionOS 27: allAnchors and anchorUpdates disagree on WorldAnchor transforms (different coordinate frame)
After updating to visionOS 27.0, WorldTrackingProvider.allAnchors returns WorldAnchor.originFromAnchorTransform values that do not match what anchorUpdates delivers for the same anchor, read at the same moment, both isTracked == true, with no .updated event in between. The difference is a rigid frame change, not drift: Same vertical offset on every anchor: -1.173 m. Identical across two apps and two days with the head at 1.13 m and 1.22 m, so a constant, not head height. Horizontal deltas vary per anchor, but inter-anchor distances are preserved, so it is a yaw plus translation. Stable over time, same across launches. anchorUpdates matches the ImmersiveSpace frame RealityKit draws in. allAnchors does not. On visionOS 26 both agreed. Same anchor, same instant: allAnchors (0.1486, -0.7143, 0.2231) anchorUpdates (0.1316, 0.4590, -1.0955) device pose (0.0073, 1.2195, -0.0038) Four other anchors, delta = allAnchors - anchorUpdates: ( 2.6933, -1.1733, -0.2514) (-0.0810, -1.1733, 1.1339) ( 0.1878, -1.1733, 0.9891) ( 2.6289, -1.1733, -0.1809) In our app this showed up as a placed model vanishing after a scene switch: we re-pinned the model root from allAnchors, and it landed below the floor behind the user. A standalone sample (one provider, one anchor, a green sphere from anchorUpdates, a red sphere from allAnchors) shows two spheres on visionOS 27.0, the red one below the floor: added F21942C4-… at (-0.031, 0.712, -0.976) update added F21942C4-… tracked=true at (-0.031, 0.712, -0.976) MISMATCH delta (-0.211, -1.173, 1.444) m allAnchors (-0.243, -0.461, 0.468) anchorUpdates (-0.031, 0.712, -0.976) tracked=true device (0.000, 1.129, -0.001) Questions: Is allAnchors now intentionally expressed in a different frame on visionOS 27, for example related to the new ARKitCoordinateSpaceProviding / coordinateSpace(correction:) API? The documentation for allAnchors does not mention it. If so, which frame is it, and how should it be converted to the ImmersiveSpace frame? Workaround we use: read transforms only from anchorUpdates into a per-id cache and never position from allAnchors. Filed as FB24799304. Minimal repro project attached to the feedback.
Replies
0
Boosts
0
Views
22
Activity
3h
Camera specs of world-facing tracking cameras.
We are working on a very space constrained custom spatial accessory. Since the form factor of the device is small, we need to add small LEDs producing small blob diameters with accordingly small distance between them. To better estimate the amount, size, and distance of the LEDs it would help to know which image sensors and field of view the six world‑facing tracking cameras have.
Replies
1
Boosts
0
Views
721
Activity
13h
Update entity state across devices through SharePlay on VisionOS
I am trying to create an app where I have multiple users modifying an entity at runtime similar to collaborative digital sculpting using SharePlay on VisionOS. When creating a collaborative experience using other 3D engines I have utilized an external server that devices send and update their state from serving as a global "source of truth". This doesn't seem to be the intended workflow for SharePlay which seemingly only allows for device to device messaging. While this works for a variety of apps it doesn't provide a smooth experience when multiple users are interacting with and manipulating the same 3D entity. Is there an intended way to do this for SharePlay? Would the best practice be to have one user serve as a host and establish a "source of truth" based on their app state? The closest example in documentation I have found is an example for using SharePlay in a drawing app. When a user is drawing they send the information for their realtime stroke through unreliable SharePlay messages and then, when they've finished, they send a reliable message with the final line. This technique makes sense for the given example since there is no scenario where multiple users would be drawing the same line, but it seemingly falls short for my use case since multiple users can sculpt in the same area of a given mesh. What would the intended approach be to achieve this with SharePlay? Is it simply a limitation of this type of networking? Thanks!
Replies
1
Boosts
0
Views
1.1k
Activity
14h
RealityKit cloth: reducing mesh collider air gap without penetration
I’m testing RealityKit cloth for garment-like cloth over an authored human-shaped triangle-mesh collider. The issue also occurs with a static collider, so it does not appear to depend on character animation.. Contact is stable and runs interactively, but I cannot achieve close surface contact without sacrificing robustness. The cloth mid-surface remains visibly several millimetres from the authored collider, especially around curved or narrow areas such as the head, arms, and hands. Transparent rendering of visualMesh confirms this is not just render thickness. Reducing ClothMeshShape.bias reduces the gap, but eventually causes interpenetration around difficult geometry. Increasing it prevents penetration but makes the cloth visibly float. I’ve also checked scale, winding, normals, reasonably even triangles, timestep, and solver iterations. I want to avoid inflating the collider or adding anatomical proxy shapes. Is bias the only physical-clearance control, or does RealityKit apply an additional contact radius derived from simulation-mesh resolution? Is local conforming refinement the recommended way to achieve roughly 1–2 mm clearance while retaining robust real-time contact? Are there any specific topology requirements beyond evenly sized triangles? I can provide a minimal reproduction and screenshots if helpful although not sure I can share screenshots here?
Replies
3
Boosts
0
Views
3.1k
Activity
16h
Questions about RoomPlan, Room API, USDZ/STEP comparison, and extended spatial scanning
Dear Apple Developer Team, I would like to ask a few questions related to RoomPlan, Room API, USDZ export, and possible future spatial scanning workflows: Can the RoomPlan or Room API load an existing USDZ room model and compare it in real time with a new live scan of the same space? Is Apple considering extending RoomPlan beyond indoor rooms, for example to scan outdoor areas, house exteriors, terrain, and simple building volumes with rough dimensions? Will RoomPlan support multi-floor continuous scanning, custom object detection elements, and reliable export of the scanned result to USDZ for further CAD/BIM workflows? Is it possible to create a real-time comparator between a reference object in USDZ or STEP format and a physical object being scanned live, so that deviations in geometry or dimensions can be detected during scanning? Best regards, Ivo Saina
Replies
4
Boosts
0
Views
836
Activity
1d
Is there any technical documentation available for the iPhone’s LiDAR scanner?
Hi, I’m looking to develop an app that uses the iPhone’s LiDAR scanner and am trying to decide which model to buy. Is there any relevant technical documentation available on its capabilities, such as accuracy, resolution and range? I’d also like to understand whether there have been any improvements between generations, particularly with the iPhone 18 Pro. Any documentation or pointers in the right direction would be appreciated. Thanks!
Replies
0
Boosts
0
Views
246
Activity
1d
Best practice for connecting Mac-side USDZ preparation with exact-asset Vision Pro verification?
I’m developing two complementary developer tools: Reality Prep Pro on macOS and Reality Prep Preview on visionOS. The workflow I’m trying to solve is: Mac: prepare and validate a USDZ Vision Pro: verify that exact USDZ on-device Mac: bring the device evidence back and associate it with the exact asset revision I currently use the USDZ’s SHA-256 as the authoritative identity. If the asset bytes later change, the previous Vision Pro evidence is treated as stale and a re-test is recommended. Existing USDZ→USDZ preparation is package-preserving, so non-texture payloads such as animation/rigging data are not unnecessarily re-authored. I’d be interested in hearing from other RealityKit / Reality Composer Pro developers: Is this similar to how you currently handle asset provenance between desktop preparation and real-device testing? Are there any Apple frameworks or established visionOS practices you would recommend for strengthening this kind of revision-to-device-evidence workflow?
Replies
0
Boosts
0
Views
457
Activity
3d
Generic spatial accessory receives LED reports but never produces an AccessoryAnchor
We are developing a generic spatial accessory for Apple Vision Pro using the “Working with generic spatial accessories” sample and a trained .referenceaccessory file. Environment: visionOS: [version/build] Xcode: [version/build] Hardware: Apple Vision Pro + nRF52840-based BLE accessory IMU: 6-axis IMU, 400 Hz HID reports LED constellation: 8 x 850 nm IR LEDs, driven through MOSFETs The BLE/HID and timing path appears to be working: The accessory connects successfully over BLE. ASA/HID enumeration succeeds. The firmware receives continuous LED Output Reports from visionOS. The Accessory Tracking Clock reaches READY. PWM scheduling reports no late events and no queue overflow. In the Accessory Tracking Debug View, multiple independent IR LED blobs are visible and stable at approximately 0.7 m. At least 4 LEDs are visible from normal viewing angles. The app resolves the .referenceaccessory and starts AccessoryTrackingProvider. However, ARKit never produces an AccessoryAnchor. We also have no diagnostic signal indicating whether the failure occurs at: LED candidate-point detection, constellation matching against the .referenceaccessory, or IMU/optical fusion. Questions: Is there a supported diagnostic API, log category, or Debug View indicator that distinguishes candidate-point rejection from constellation-match or fusion rejection? Are there known acquisition prerequisites in the current visionOS beta beyond the published Accessory Design Guidelines? Can Apple recommend a minimal validation procedure using the generic spatial accessory sample to confirm that the LED constellation is accepted by the tracking system? Is this expected to work in a volumetric window, or are there additional lifecycle/session requirements that can prevent initial acquisition? I can provide sanitized HID LED-report logs, timing data, Debug View captures, and a minimal reproducible project if useful.
Replies
10
Boosts
0
Views
2.8k
Activity
6d
Physical space lighting (SurroundingsLight) never lands on sloped ceiling surfaces that are present in the scene reconstruction mesh
Summary A SpotLightComponent with SpotLightComponent.SurroundingsLight() (and a ProjectiveTexture) lights the flat part of my ceiling and the walls, but never the two sloped ceiling surfaces of a converted attic room, at any angle, intensity, attenuation radius or falloff. The lit area stops in a hard straight line exactly where the flat ceiling meets each slope, even though the beam clearly extends past that line. The sloped surfaces are fully present in SceneReconstructionProvider's mesh, and their faces point into the room. So this doesn't look like a scene understanding coverage problem. It looks as if physical space lighting only considers surfaces classified as horizontal ceiling, wall or floor, and skips slanted ones. Steps to reproduce Open a mixed immersive space in a room with a sloped ceiling (a converted attic, or any vaulted ceiling). Add an entity with SpotLightComponent(color: .white, intensity: 40_000, innerAngleInDegrees: 44.5, outerAngleInDegrees: 45, attenuationRadius: 10), SpotLightComponent.SurroundingsLight(), and a SpotLightComponent.ProjectiveTexture carrying a labelled grid texture so the beam's extent is visible. Place it near the floor under the wearer and aim it straight up. Run SceneReconstructionProvider alongside and draw each MeshAnchor as a wireframe (or as a solid material with faceCulling = .back) to confirm the mesh covers the slopes and faces inward. Expected The projected pattern conforms to every surface in the scene understanding mesh inside the cone, including the sloped ceiling, as it does for the flat ceiling and walls. Actual The pattern appears only on the flat ceiling strip and on the walls. Both slopes stay completely dark, with a hard straight cut at the crease. Tilting the light to aim directly at a slope, raising intensity to 200,000, and changing attenuation radius and falloff make no difference. The wireframe shows continuous mesh across the crease and onto the slopes; the back-face-culled solid view shows the slope faces pointing into the room. Screenshots attached: (1) the projected grid stopping at the crease, (2) the reconstruction wireframe covering the slopes, (3) the solid front-faces-only mesh showing the slopes are front-facing. (4) an additional image of the projected grid with both ceiling and walls in view Code let spot = Entity() spot.components.set(SpotLightComponent.SurroundingsLight()) var light = SpotLightComponent(color: .white, intensity: 40_000, innerAngleInDegrees: 44.5, outerAngleInDegrees: 45, attenuationRadius: 10) light.attenuationFalloffExponent = 0.5 spot.components.set(light) spot.components.set(SpotLightComponent.ProjectiveTexture(texture: gridTexture)) spot.look(at: from + SIMD3(0, 1, 0), from: from, upVector: SIMD3(0, 0, -1), relativeTo: nil) root.addChild(spot) Questions Is physical space lighting limited to horizontally and vertically classified surfaces by design? If so, could this be documented on SurroundingsLight, and is there a way to include slanted surfaces? If not by design, is there anything an app can do to have the light use the full reconstruction mesh?
Replies
2
Boosts
0
Views
1.1k
Activity
1w
Is there currently a way to import 3D Gaussian Splatting data into Reality Composer Pro?
Hello, As far as I understand, RealityKit now supports 3D Gaussian Splatting through GaussianSplatComponent. I would like to ask whether there is currently a way to use 3D Gaussian Splatting data directly in Reality Composer Pro, for example by importing a .ply file containing Gaussian splats. More specifically: Can Reality Composer Pro import a 3D Gaussian Splatting .ply file? Is there a way to create or assign a GaussianSplatComponent to an entity directly within Reality Composer Pro? If this is not currently supported, is there any recommended workflow for creating a .usdz containing Gaussian Splatting data that can be opened in the Preview app on iPhone/iPad and displayed as an AR object? I tried opening the sample plant.usdz on an iPhone using the Preview app. However, the AR preview appears to be empty, so it seems that the Preview app may not currently support rendering the Gaussian Splatting content contained in that USDZ. My desired workflow would be: Gaussian Splatting .ply → Reality Composer Pro / USDZ → iPhone Preview → AR Is this workflow currently possible? If not, is there an official or recommended way to package a GaussianSplatComponent and its associated Gaussian splat data into a USD/USDZ file for AR preview on iOS? If the above functionality is not currently supported, does the development team have any plans to add or improve support for these features in future versions of Reality Composer Pro or the Preview app? Any clarification regarding the current capabilities, limitations, or future plans would be greatly appreciated. Thank you!
Replies
3
Boosts
0
Views
2.3k
Activity
1w
Anchoring Invalid
Hello. I trained an ObjectTracker.referenceobject using ML, and am using it in RCP. After adding Anchoring to the entity, I set the Target mode to Object and applied the trained model data ObjectTracker.referenceobject. However, there is no tracking effect after building for VisionPro. Are any permissions required? What should I do?
Replies
3
Boosts
0
Views
2.5k
Activity
1w
PresentationComponent loses gaze targeting or dismisses in a volumetric window
On visionOS 26.6 and 27 latest beta: PresentationComponent in a volumetric WindowGroup, when looking at the volume from a specific lower/back region create two bugs: buttons inside the presentation have their gaze obstructed by hand presence, no button shows any hover highlight hand movement in that specific area auto dismisses the presentation The focused public repro uses a thin ModelEntity with InputTargetComponent and ordinary SwiftUI buttons. Its README includes two clearer videos: https://github.com/mesqueeb/PresentationAutoCloseInBackOfVolume 1. Gaze obstructed on buttons when hand is at specific location (see "Test Action" has no hover highlight, even though i look at it) 2. Hover state restored on buttons when hand is gone (see "Test Action" has hover highlight) 3. Presentation auto closed when hand comes back The weirdest part: This only happens at a specific location, when walking to the back of the volume, then turning around and interacting with the volume from the lower right back side. Any workaround or fix guidance would be appreciated. Feedback Assistant: FB24569418.
Replies
2
Boosts
0
Views
1.3k
Activity
2w
Configure AI Provider via Web login
In Xcode it's extremely convenient to be able to sign into AI providers by having an authentication workflow opened in a browser. Reality Composer Pro 3 Beta 5 requires you to manually configure the AI provider (Claude for example). Please provide the Web-based authentication workflow in Reality Composer Pro that is also available in Xcode.
Replies
2
Boosts
0
Views
1.2k
Activity
2w
Request for better Gaussian Splat examples
I noticed that there is a Level of Detail component in beta in RealityKit, along with the new GaussianSplatResource component. Apple Maps 3D clearly uses some combination of these things, in addition to some form of streaming and compression most likely, in order to make performance acceptable. It would be nice if Apple could give us some examples of combining file compression + LevelofDetail + GaussianSplatResource in an example project. This is what will be required to make any kind of immersive 3D walkthroughs possible with these frameworks. I just randomly stumbled upon the LevelOfDetail component bc it was mentioned in a Google AI response, but otherwise there seem to be no examples that use it with respect to GaussianSplatResource (and so it’s not clear if it’s meant to be used in this way)
Replies
0
Boosts
0
Views
679
Activity
2w
ARKit World Tracking Drift Regression on LiDAR-Equipped Devices - iOS 26.4+
Summary We have identified a reproducible world tracking drift regression in ARKit on LiDAR-equipped iOS devices running iOS 26.4 and later. A static virtual node anchored at the world origin visually drifts from its initial position as the user moves around a real-world scene, despite the scene remaining physically static. The same code produces stable, drift-free results on non-LiDAR devices running identical OS versions. Device & OS Observations Testing was performed across four devices on iOS 26.4 using the same application build and ARWorldTrackingConfiguration settings. Non-LiDAR devices — iPhone 14 and iPhone 15 — produced stable, drift-free tracking in all test runs. No world origin displacement was observed regardless of how long or how far the user walked. LiDAR-equipped devices — iPhone 14 Pro and iPhone 16 Pro — exhibited consistent, reproducible drift. A static node placed at the world origin visually shifted from its initial position as the user moved through the scene. The same devices were stable on earlier iOS versions, confirming this is a regression introduced in iOS 26.4. Technical Observations Nature of drift: A SCNNode placed statically at the ARKit world origin (SCNVector3(0, 0, 0)) visually displaces from its original position as the user walks around a static real-world scene. The displacement is not random — it accumulates directionally as the user moves, consistent with a sensor fusion or coordinate anchoring error. Trigger condition: The drift occurs during normal walking motion around a fixed point of interest, such as circling a parked vehicle. It does not appear when the device is held still. LiDAR specificity: The drift is exclusive to devices with a LiDAR scanner. Identical hardware configurations — same iOS build, same ARWorldTrackingConfiguration settings — on non-LiDAR devices produce no drift whatsoever. This isolates the regression to the LiDAR sensor's contribution to ARKit's internal Visual-Inertial Odometry (VIO) fusion pipeline. No API-level workaround found: There is currently no public ARKit API to selectively disable the LiDAR scanner's contribution to VIO. All available configuration-level options have been evaluated without resolving the drift. ARWorldTrackingConfiguration Options Evaluated The following configuration changes were applied individually and in combination. None resolved the drift on LiDAR devices: isAutoFocusEnabled = false — No improvement videoHDRAllowed = false (disabled) — No improvement planeDetection = [] (disabled) — No improvement sceneReconstruction = .mesh — No improvement worldAlignment: .gravity vs .gravityAndHeading — No improvement Minimal Reproduction Case The drift can be reproduced with a minimal ARKit scene: Create an ARSCNView with ARWorldTrackingConfiguration using default settings. Add a single static SCNNode (e.g., a small sphere or axes geometry) at SCNVector3(0, 0, 0) when the session starts. Run the app on a LiDAR-equipped device (iPhone Pro, iPad Pro with LiDAR) on iOS 26.4 or later. Walk in a circle around the node's approximate real-world position. Expected: The node remains visually fixed at its world position throughout the walkthrough. Actual: The node drifts from its initial position, increasingly displaced from its world origin anchor as walking continues. We want to know if Apple has made any internal updates to ARKit, particularly after the OS 26.4 upgrade. Thanks!
Replies
6
Boosts
8
Views
3.4k
Activity
2w
RCP 3 Beta 2–4: Create/Open silently fails before editor launch
Reality Composer Pro 3 cannot create a new project or open an existing project on my Mac. After I select Create New Project or Open Existing Project, the operation begins but the project editor never appears. The Reality Composer Pro process remains alive and idle, and no standard crash report is generated. Environment: MacBook Pro Mac15,9 Apple M3 Max, 48 GB unified memory macOS 26.6 (25G72) Reality Composer Pro 3 Beta 4, version 3.080.0.1.500.1 Built-in display only Steps to reproduce: Quit Reality Composer Pro. Launch /Applications/RealityComposerPro.app. Select Create New Project. Observe that the editor never appears. Open Existing Project fails in the same way. The problem is reproducible 100% of the time with Reality Composer Pro 3 Beta 2, Beta 3, and Beta 4. I have also reproduced it: With a completely blank new project With an existing project In a clean local macOS user account In Safe Mode With no external displays With local paths and iCloud disabled After verifying adequate free disk space and separately testing free-space conditions After resetting Reality Composer Pro state and preferences The application passes code-signature and Gatekeeper verification. Its temporary loader copy is created successfully, with no ENOSPC, failed-copy, or failed-write error. The failure occurs around libtm-welcome_window.dylib and libtm-plugin_support.dylib prepareToLoad. A small modal NSPanel repeatedly performs AppKit/SwiftUI constraint updates and reports: The window has been marked as needing another Update Constraints in Window pass, but it has already had more Update Constraints in Window passes than there are views in the window. The exception appears to be swallowed instead of crashing the process. A process sample then reports SOME_OTHER_THREAD_SWALLOWED_AT_LEAST_ONE_EXCEPTION while the main thread remains idle in the normal NSApplication event loop. This appears to be a constraint-update loop in the welcome-window loader/progress panel, rather than a problem with a particular project or its assets. Feedback Assistant report: FB24460905 Has anyone reproduced this on macOS 26 with Reality Composer Pro 3? Is there a supported workaround that bypasses the failing loader/progress panel? Is RCP 3 Beta 4 expected to work on macOS 26.6, or does it require macOS 27?Reality Composer Pro 3 cannot create a new project or open an existing project on my Mac. After I select Create New Project or Open Existing Project, the operation begins but the project editor never appears. The Reality Composer Pro process remains alive and idle, and no standard crash report is generated. Environment: MacBook Pro Mac15,9 Apple M3 Max, 48 GB unified memory macOS 26.6 (25G72) Reality Composer Pro 3 Beta 4, version 3.080.0.1.500.1 Built-in display only Steps to reproduce: Quit Reality Composer Pro. Launch /Applications/RealityComposerPro.app. Select Create New Project. Observe that the editor never appears. Open Existing Project fails in the same way. The problem is reproducible 100% of the time with Reality Composer Pro 3 Beta 2, Beta 3, and Beta 4. I have also reproduced it: With a completely blank new project With an existing project In a clean local macOS user account In Safe Mode With no external displays With local paths and iCloud disabled After verifying adequate free disk space and separately testing free-space conditions After resetting Reality Composer Pro state and preferences The application passes code-signature and Gatekeeper verification. Its temporary loader copy is created successfully, with no ENOSPC, failed-copy, or failed-write error. The failure occurs around libtm-welcome_window.dylib and libtm-plugin_support.dylib prepareToLoad. A small modal NSPanel repeatedly performs AppKit/SwiftUI constraint updates and reports: The window has been marked as needing another Update Constraints in Window pass, but it has already had more Update Constraints in Window passes than there are views in the window. The exception appears to be swallowed instead of crashing the process. A process sample then reports SOME_OTHER_THREAD_SWALLOWED_AT_LEAST_ONE_EXCEPTION while the main thread remains idle in the normal NSApplication event loop. This appears to be a constraint-update loop in the welcome-window loader/progress panel, rather than a problem with a particular project or its assets. Feedback Assistant report: FB24460905 Has anyone reproduced this on macOS 26 with Reality Composer Pro 3? Is there a supported workaround that bypasses the failing loader/progress panel? Is RCP 3 Beta 4 expected to work on macOS 26.6, or does it require macOS 27?
Replies
2
Boosts
0
Views
1.2k
Activity
2w
ARKit: Is there a refined historical camera trajectory available after world mapping?
Hi, I'm building an iOS LiDAR scanning app using ARWorldTrackingConfiguration with sceneReconstruction = .mesh. During scanning I save the RGB image, ARFrame.camera.transform, intrinsics and per-frame scene depth. At the end of the scan I also have the final ARMeshAnchor mesh. When I place the saved depth observations into the ARSession world using their original ARCamera.transform, they broadly form the same scene, but over a longer walk I can see several-centimetre discrepancies between observations of the same surfaces. My question is very specific: After ARKit has built its world map / reconstructed the environment, is there any public API that provides a globally refined or corrected historical pose for previously captured ARFrames, rather than the original ARCamera.transform recorded at each frame? In other words, is there an API or ARKit data structure from which I can obtain something equivalent to: frame timestamp → refined world-space camera transform after the session has accumulated more information? I am aware of ARWorldMap and worldMappingStatus, but I'm unclear whether the world map contains information that can be used to recover or refine the historical camera trajectory, or whether ARCamera.transform is the only historical pose available to an app. I'm not looking for advice on implementing ICP or my own SLAM/registration. I just want to know whether ARKit already exposes a refined trajectory or the underlying information needed to obtain one. Thanks.
Replies
6
Boosts
0
Views
1.5k
Activity
3w
GaussianSplatResource - any support for spz and sog formats soon?
While loading PLY and USDZ is nice, it would be great to support more compressed formats like SPZ and SOG: https://www.nianticspatial.com/en/blog/spz4 Is this something that might be in the works? Or if Apple releases their own compression, that could work also. PLY tends to be quite large for models of any size.
Replies
4
Boosts
0
Views
1.3k
Activity
3w
OpacityComponent does not blend a GaussianSplatComponent on iOS/macOS, behaves correct on visionOS
I have a RealityKit scene with one container entity and two children: an entity carrying a GaussianSplatComponent, and a plain Entity with a ModelComponent (an unlit white cube). A single OpacityComponent goes on the container, so one value covers both children in the same frame. container.components.set(OpacityComponent(opacity: value)) On visionOS 27 this does exactly what I expect: both children fade together, continuously, across the whole range. On iOS 27 and Mac Catalyst 27 the cube fades continuously but the splat does not. It renders at full opacity for every value above 0 and disappears at exactly 0. At 0.01 the cube beside it is already almost invisible while the splat is still drawing at full strength, so entity opacity behaves as a visibility switch on a splat rather than as a blend factor. Things I have already ruled out: Not a propagation issue. Moving the OpacityComponent off the container and onto the splat's own entity (and the cube's own entity, to keep the A/B) changes nothing. Not the container failing to receive the value. The cube under the same component, in the same frame, fades correctly. Not the splat data. Per-splat alpha through GaussianSplatResource's opacity buffer is honoured, and projectionMode, sortingMode, scaleActivation and opacityActivation all take effect. Not a difference in scene construction per platform. The same file builds the container on all three platforms in my sample. Confirmed on: iOS 27.0 (24A5418b), iPhone 16 Pro — reproduces macOS 27.0 (26A5416b), Mac Catalyst — reproduces visionOS 27.0 (24M5326g), Apple Vision Pro — correct Filed as FB24431214 with a sample project that builds for all three from one target. Would be great to see this fixed!
Replies
1
Boosts
0
Views
449
Activity
3w
How does PhotogrammetrySession use LiDAR depth provided through PhotogrammetrySample?
We are developing a LiDAR-based spatial capture application using ARKit and RealityKit. We are investigating whether PhotogrammetrySession can be used as part of a capture pipeline where we collect: RGB frames LiDAR depth camera intrinsics camera transforms IMU/device motion data and then reconstruct a textured 3D model. The documentation states that PhotogrammetrySample can include depthDataMap, depthConfidenceMap, gravity, and camera metadata. Could someone clarify how PhotogrammetrySession uses the depth information when supplied? Specifically: Is LiDAR depth used during reconstruction to improve geometry (for example as a constraint during reconstruction), or is it primarily used for scale/orientation metadata? When camera transforms and intrinsics are available from ARKit, does PhotogrammetrySession consume this information, or does it independently estimate camera poses from the image sequence? Is the recommended Apple workflow for RGB + LiDAR capture: ARKit capture → PhotogrammetrySample → PhotogrammetrySession or is ObjectCaptureSession the intended capture pipeline for photogrammetry workflows? We are trying to understand the intended relationship between ARKit, ObjectCaptureSession, and PhotogrammetrySession before choosing an architecture. Thanks.
Replies
2
Boosts
0
Views
1.4k
Activity
4w