RealityKit

RSS for tag

Simulate and render 3D content for use in your augmented reality apps using RealityKit.

Posts under RealityKit tag

200 Posts

Post

Replies

Boosts

Views

Activity

Immersive space sometimes opens with a broken world origin (world anchors fail with error 200, recenter times out)
Hi all, We're seeing an intermittent issue in a native SwiftUI + RealityKit app on Vision Pro (visionOS 27.0.1, still there on 27.2). When we open a mixed ImmersiveSpace, roughly 1 in 10–30 opens, everything ends up offset from where it should be. Sometimes it's 50 cm, sometimes several meters, sometimes it's rotated. Holding the Digital Crown to recenter fixes it immediately. We are attaching things to the camera, like we have a window following the user and that is offset in these rare cases which makes the app not usable. In the device logs, a good open looks like this: backboardd (recenter) Setting new world origin with anchor ... backboardd ARWorldTrackingProcessor: Added world anchor ... successfully (x4) backboardd (recenter) Finished (error: (null)) setting set new world origin ... A bad open looks like this: backboardd (recenter) Setting new world origin with anchor ... arkitd SLAMAnchorUpdate: Failed to add anchor request ... NSPOSIXErrorDomain Code=4 backboardd ARWorldTrackingProcessor: Failed to add world anchor ... com.apple.arkit Code=200 "Adding an object failed." (x4) backboardd (recenter) timeout exceeded. Forcing fade in. Like whenever the bad state occurs it is because adding a world anchor seems to fail. I haven't tested with visionOS26 but the same app didnt run into that problem ever. So the system seems to fail adding its own origin anchors, gives up after 2 seconds and shows the space anyway. Has anyone else run into this? And is there any way to detect it from the app or trigger a recenter in code? Thanks!
2
0
609
2d
High fidelity gaming on Apple Vision Pro
I'm looking to find the best setup for a game on AVP that gives the best quality environment with gameplay but there seems to be a mix of possible solutions each with their own drawback. My ideal scenario is a high fidelity room/environment while making use of the PS VR2 Sense controllers to track the user's hands and give them more inputs via the controllers. Eg they hold a weapon tracked to the controllers with the buttons able to use the weapon, shooting and hitting enemies in the room. I want to aim more for highest qualtiy gaming than cartoony so opted for foveated streaming from the Mac as my ideal, but this seems limited to only having some base inputs from the hands and no controller inputs for hand and positional tracking from the controller movement (as they need the data from the Vision Pro camera but the game is technically running on the Mac and streamed to the VP) The only real solution for what I want here is to go via RealityKit as a native Vision Pro app but this reduces the game quality I had in mind as the entire thing is now rendering on the VP instead of the Mac. Am I missing something here or is this a real limitation today? To reitarate I'm looking for: highest fidelity game quality (rendering as much as possible on the Mac) tracking the VR2 Sense controller's positions (to track a weapon) in 3D space
1
0
998
3d
Async Entity(contentsOf:) is 15 to 20x slower while an ARView is rendering
While a non-AR ARView is on screen and rendering, await Entity(contentsOf:) takes about 26 s to load a 4.5 MB USDZ with 300 textured meshes. The same file loads in about 1.5 s with the synchronous Entity.load(contentsOf:), or with the async API while the view is hidden. iPhone 16 Pro, iOS 27.2Mac Catalyst, macOS 27.2await Entity(contentsOf:), view rendering22 to 26 s26 sEntity.load(contentsOf:), view rendering1.6 s1.1 to 1.7 sawait Entity(contentsOf:), view hidden1.4 s1.3 to 1.5 sModelEntity(contentsOf:) and Entity.loadAsync(contentsOf:) behave the same. A .reality file loads fast either way, and the iOS Simulator doesn't show it. A Time Profiler trace shows the main thread blocked in re::RenderThread::waitForFramePacing on every frame, so the @MainActor import only advances between frames. Neither workaround fits loading content into a scene the user is looking at: the sync load blocks the main thread, and hiding the view blanks it. Filed as FB25340264, with a sample project.
0
0
138
3d
visionOS : any non-nil preferredSurroundingsEffect enables a light-spill halo around VideoPlayerComponent video
Hi all — sharing a platform behavior I isolated while building a custom immersive video player, in case others hit it (and to ask whether it's intended). Finding On visionOS 27 (Xcode 27), an entity playing video via VideoPlayerComponent(videoRenderer:) gets a light-spill "screen glow" halo around its edges whenever the immersive space declares any non-nil preferredSurroundingsEffect. The halo follows the video's colors frame by frame. With no surroundings effect (nil), there is no halo. Quick repro Progressive immersive space + VideoPlayerComponent playing. Add .preferredSurroundingsEffect(.dim(intensity: 0.0)) → halo appears (even though the effect is visually a no-op). Remove it → halo disappears. Same with .colorMultiply(_:) and preset effects. Things I ruled out on device (Vision Pro) ImmersiveContentBrightness (.custom(1.0) vs .automatic): no difference. VideoPlayerComponent configuration (.stereo, spatial video mode, isPassthroughTintingEnabled) and component attach timing: no difference. Presenting the same video via VideoMaterial: no halo. Bright white SwiftUI content: no halo. Questions Is this coupling intended? The preferredSurroundingsEffect docs don't mention any effect on video playback. Is there a public way to keep a surroundings effect (e.g. room dimming) while avoiding the video light spill? I found none on VideoPlayerComponent. Current workaround: don't declare any surroundings effect in the modes where the halo is undesirable. Also filed via Feedback Assistant(FB25130748). Curious if anyone else has seen this or knows a switch.
2
1
2.9k
1w
RCP .reality file in iPadOS with behavior tree
I am having troubles with a .reality file in iPadOS 27.0.1. I have a Reality Composer Pro 3 project with an Animation Graph and Behavior Tree. The animation simply switches between a character walking and running, and the Behavior Tree simply changes the boolean to trigger the running state, waits, flips the boolean back, waits, and repeats. When I run the Simulate in RCP 3, it behaves as I expected - walks, runs, walks, runs, etc. I export the World scene as a .reality file, and open the .reality file in Xcode on macOS 27.2, and it behaves as I expected. But when I load the .reality file in an iPadOS app (iPadOS 27.0.1), the character does not switch between states. It starts the walking animation, but it never switches to the running animation. It appears the Behavior Tree isn't working as expected. Any idea what is wrong? Should I switch the iPad to the iPadOS 27 betas? Should I set the minimum OS target for the app to iPadOS 27? Is the Behavior Tree a known limitation with iOS or iPadOS 27?
1
0
570
1w
Photogrammetry with Object masks hangs and terminates with masks of objects
[PhotogrammetrySample]) with objectMask set and traps on my ios26.5.2, see the attached screenshot on feedback FB24379913 , it gets to the function and hangs . Even the folder reconstruction with lazy sequence as recommended from your video sample also doesn;t complete as it can't find alignment and displays the CoreOC.PhotogrammetrySession.Error error 6 means alignment failed. What can be done or do you guys expose any functions that can be used to check or trace or handle these internally The ObjectMasks are actually segmentation masks from an segmentation algorithm I will appreciate a timely response and willing to provide more clarity and informations, thank you so much for your understanding
3
0
1.4k
2w
On-screen RealityView starves CADisplayLink to 30 Hz on ProMotion (Mac Catalyst)
FB24536235 On Mac Catalyst under macOS 27, a plain CADisplayLink asking for CAFrameRateRange(minimum: 60, maximum: 60, preferred: 60) gets serviced at 30 Hz for as long as a RealityView is on screen in the same window. The link does nothing per tick but count, so there's nothing of mine to blame it on. RealityKit's own statistics overlay reads 60.41 fps in the same frame. Click a segmented control that removes the RealityView and the same link goes straight back to 60. Nothing else changes. That's the whole reproducer, and I've attached it to the radar. It only happens while the display panel is in ProMotion mode. Set the built-in to a fixed 60 Hz and it's correct again. With an external 60 Hz display attached the roles swap: the built-in is fine and the external drops to somewhere between 18 and 30, and setting the built-in to 60 Hz fixes that one too without touching the external's own settings. A raw MTKView presenting continuously at 60, at 120, and on a 120 Hz link presenting every second callback are all fine, so it isn't continuous presentation and it isn't the requested rate. It's RealityKit specifically. Worth knowing if you're testing: RealityView on Catalyst is an ARView underneath, so both paths give you the same answer. This is VERY rough for anything that puts RealityKit next to a UI. In an editor that's the sidebar, the inspector, gizmos, drag handles, every display-link-driven or UIKit animation in the window running at half rate around a viewport that stays smooth. Likely Related to FB24091347, which is the same defect seen as SwiftUI scroll judder. If you can reproduce either, please file a duplicate. Attached two screenshots; first with promotion enabled, second with promotion off. PLEASE fix this, it drives me crazy and there seems to be no workaround. On release day of macOS 27 our app will likely be blamed for it by users and my hands are tied. Thank you!
2
1
1.3k
3w
Spatial Scene API for iOS Apps
As part of the WWDC25 Keynote, a technology was announced that can present 2D images as 3D spatial scenes. This announcement is supported by a Press Release. ...developers can use the Spatial Scene API to make their app experience even more immersive. Zillow is taking advantage of the API for their Zillow Immersive app, allowing users to see images of homes and apartments with the rich depth and dimension that spatial scenes offer. The feature also appears in the Photos App on iOS 26 Developer Beta 1. Tapping "Spatial Scene" on any photo opens a view of that photo with a parallax effect. I've searched the WWDC sessions and new documentation and have come up short. Reaching out here for help. Is there any documentation for Spatial Scene API? Or any guidance on how to implement the spatial scene in iOS?
2
2
882
3w
RealityKit VideoMaterial renders 8-bit HEVC HLG/BT.2020 360° video with a purple tint in the visionOS Simulator (device is fine)
We paint a looping 360° video on the inside of a sphere with VideoMaterial(avPlayer:): an AVQueuePlayer plus AVPlayerLooper, a ModelEntity with an inward-facing equirectangular mesh, inside a .mixed ImmersiveSpace. No shaders, no VideoPlayerComponent. We could not find an official Apple sample that puts a VideoMaterial on a 360° sphere; this is the plain pattern from the VideoMaterial(avPlayer:) reference and the one used in threads like 734081. On Apple Vision Pro hardware the video renders correctly. In the visionOS Simulator (27.0, 26.5 and earlier 26.x, Xcode 27.0) the whole frame has a strong purple/magenta cast. Audio and timing are fine. The video (which we cannot share, so here is its metadata, image sample of issue on simulator provided below): MP4, HEVC Main, 8-bit 4:2:0, 3840x1920, tagged BT.2020 primaries, HLG transfer (ARIB STD-B67), BT.2020 non-constant matrix, both in the colr atom and in the SPS VUI. ffprobe -show_format -show_streams (trimmed): [STREAM] index=0 codec_name=hevc profile=Main codec_type=video codec_tag_string=hvc1 width=3840 height=1920 pix_fmt=yuv420p color_range=tv color_space=bt2020nc color_transfer=arib-std-b67 color_primaries=bt2020 r_frame_rate=30000/1001 avg_frame_rate=679200/22663 duration=37.771667 bit_rate=25030607 [SIDE_DATA] side_data_type=Stereo 3D type=2D horizontal_field_of_view=360000/1000 [/SIDE_DATA] [/STREAM] [STREAM] index=1 codec_name=aac profile=LC codec_type=audio codec_tag_string=mp4a sample_fmt=fltp sample_rate=48000 channels=2 channel_layout=stereo duration=37.758333 bit_rate=128176 [/STREAM] [FORMAT] nb_streams=2 format_name=mov,mp4,m4a,3gp,3g2,mj2 duration=37.778333 bit_rate=25160348 TAG:major_brand=mp42 TAG:minor_version=1 TAG:compatible_brands=isommp41mp42 [/FORMAT] AVFoundation's view of the track, from CMFormatDescriptionGetExtension on the loaded video track: primaries=ITU_R_2020 transfer=ITU_R_2100_HLG matrix=ITU_R_2020 bitsPerComponent=8 The playback code, reduced to the essentials: let item = AVPlayerItem(url: fileURL) let looper = AVPlayerLooper(player: player, templateItem: item) // retained player.actionAtItemEnd = .advance let material = VideoMaterial(avPlayer: player) let sphere = ModelEntity( mesh: try equirectSphere(radius: 12), // lat/long grid, normals inward, materials: [material] // UVs (u, 1 - v), inward winding ) sphere.position = [0, 1.5, 0] rootEntity.addChild(sphere) // rootEntity was added in RealityView { } player.play() What we have ruled out in a minimal sample project: Apple's own 360° HLS example (360 Lighthouse, HEVC Main 10, 10-bit, BT.709 SDR) plays with correct colours in the same sphere, same code, same simulator session. Rewriting our file's colr atom to BT.709 (three bytes, no re-encode) is honoured by AVFoundation (the track's CMFormatDescription then reports ITU_R_709_2 for primaries, transfer and matrix) but the tint stays. So the remaining differences between the failing file and Apple's working stream are 8-bit Main vs 10-bit Main 10, HLG/BT.2020 in the bitstream VUI vs BT.709, and local MP4 vs HLS. The minimal sample project (without the video) is available on request. Is this a known limitation of the simulator's VideoMaterial path with 8-bit HLG content, and is there a recommended way to get device-equivalent colour in the simulator? Feedback filed as FB24997147. Here is a sample of the result on the simulator of a cement column, painted white, but rendered purple on simulator:
1
0
3.3k
4w
Revealed keyboard drifts in progressive immersion, and no immersion style gives windows-in-front + Crown + passthrough cutouts
Environment: Apple Vision Pro, visionOS 27.0.1 (24M372). Magic Keyboard, with Settings › Awareness & Safety › Reveal Keyboards set to Always. Our app puts the user in an opaque virtual room while they work at their real desk, using app windows and Mac Virtual Display. A virtual desk is placed on the real desk. It is world-locked: positioned once from a PlaneAnchor (PlaneDetectionProvider), then a plain entity at fixed ImmersiveSpace coordinates. No AnchorEntity or WorldAnchor is involved. We offer two modes: .mixed, and .progressive for Digital Crown control. Each gives us only half of what desk work needs. 1. The revealed keyboard drifts in progressive immersion (FB25006789) In .progressive, the physical keyboard shown by keyboard awareness does not stay registered with the virtual desk. After the keyboard has been out of view for a few seconds (for example, while looking at a Mac Virtual Display window), it looks "floaty" when you look back: as the head turns, it slides against the desk. Turning right, the keyboard moves left farther than the desk does. What we checked: It happens in .progressive at any Crown amount, and in .full. With identical content in .mixed (no app passthrough cutout, so again only keyboard awareness shows the keyboard), it is much steadier. On 27.0.1, PlaneAnchor and DeviceAnchor coordinateSpace(correction: .rendered) are exactly equal to their .none transforms: 0 mm, 0°, scale 1. That held over 835 per-frame samples with ±50° yaw and up to 36 cm of head movement. So the rendered correction gives the app nothing to compensate with. The app's desk transform is constant while the drift is visible. upperLimbVisibility(.automatic) vs .visible and a .portrait progressive aspect ratio make no difference. Possibly related: https://developer.apple.com/forums/thread/796557 (FB19666209). 2. No immersion style combines what desk work needs (FB25006837) We need, at the same time: windows (our own and Mac Virtual Display) never covered by the room's walls, furniture or trees; the Digital Crown to bring the real surroundings back in; a passthrough cutout over the real desk, and a well-registered keyboard. .mixed: ✓ OcclusionMaterial cutouts reveal passthrough. ✓ The keyboard stays registered. ✗ RealityKit entities can render in front of windows, including Mac Virtual Display and window chrome. breakthroughEffect doesn't cover whole windows or Mac Virtual Display. ✗ The Crown can't adjust immersion. .progressive: ✓ Windows always stay in front of immersive content (as an Apple engineer confirmed in https://developer.apple.com/forums/thread/764556). ✓ The Crown adjusts immersion. ✗ OcclusionMaterial reveals nothing inside the portal, so there is no desk cutout. ✗ The keyboard drifts (issue 1). Because of issue 1 we have switched our default back to .mixed, and lost windows-in-front. Questions Is keyboard awareness in .progressive/.full expected to be registered less accurately than in .mixed? Is there a supported way to keep it registered with world-locked content, or to get the pose at which the system draws it? In .mixed, is there a supported way to keep windows (including Mac Virtual Display) in front of an app's immersive content? Or, in .progressive, a supported way to show passthrough in an app-defined region inside the portal? Any pointers or workarounds are appreciated.
0
0
1.6k
Sep ’26
How to use ClothGrabComponent?
I've simulated a piece of cloth using https://developer.apple.com/documentation/realitykit/physics-cloth-simulation I would like to grab the cloth like what I can in Marvellous Designer. I found the ClothGrabComponent at https://developer.apple.com/documentation/realitykit/clothgrabcomponent However, after I added the ClothGrabComponent to the cloth entity, I am still not able to grab it. Is there any sample code to show me how to grab the cloth? Thank you.
2
0
1.5k
Sep ’26
RealityKit asset compilation gives zero bounds to explicit meshes at matching USD prim paths
I filed Feedback Assistant report FB24967734 for this issue. When two USD scenes contain explicit polygon meshes at the same prim path and are compiled together with xcrun realitytool, one scene's mesh loads with zero bounds. Renaming the mesh prims so their paths are unique makes both meshes load correctly. Reproduce Download and expand the source ZIP from https://github.com/mesqueeb/RealitykitUsdMeshDisappearanceOnMatchingPaths/archive/refs/heads/main.zip, then run these commands from the extracted directory: xcrun realitytool compile --output-reality Fixtures/Compiled/inline-same-path.bundle/Contents/Resources/RealityKitContent.reality --platform macosx --deployment-target 27.0 Fixtures/Source/inline-same-path.rkassets xcrun realitytool compile --output-reality Fixtures/Compiled/inline-unique-path.bundle/Contents/Resources/RealityKitContent.reality --platform macosx --deployment-target 27.0 Fixtures/Source/inline-unique-path.rkassets swift run The same-path case has a mesh at /Root/MeshCube in both scenes. The control changes only the mesh paths to /Root/MeshCube_A and /Root/MeshCube_B. Current output inline-same-path: SceneA.usda / MeshCube: bounds=SIMD3<Float>(0.19999999, 0.2, 0.2) SceneB.usda / MeshCube: bounds=SIMD3<Float>(0.0, 0.0, 0.0) inline-unique-path: SceneA.usda / MeshCube_A: bounds=SIMD3<Float>(0.19999999, 0.2, 0.2) SceneB.usda / MeshCube_B: bounds=SIMD3<Float>(0.19999999, 0.2, 0.2) Expected output Both meshes in the matching-path case should have nonzero bounds: inline-same-path: SceneA.usda / MeshCube: bounds=SIMD3<Float>(0.19999999, 0.2, 0.2) SceneB.usda / MeshCube: bounds=SIMD3<Float>(0.19999999, 0.2, 0.2) inline-unique-path: SceneA.usda / MeshCube_A: bounds=SIMD3<Float>(0.19999999, 0.2, 0.2) SceneB.usda / MeshCube_B: bounds=SIMD3<Float>(0.19999999, 0.2, 0.2) Reproduced on macOS 27.0 (26A428), Xcode 27.0 (27A266a). The repo includes the USDA inputs, compiled RealityKitContent assets, and the Swift code that prints these bounds. Source and downloadable ZIP: https://github.com/mesqueeb/RealitykitUsdMeshDisappearanceOnMatchingPaths
0
0
613
Sep ’26
RealityKit cloth: reducing mesh collider air gap without penetration
I’m testing RealityKit cloth for garment-like cloth over an authored human-shaped triangle-mesh collider. The issue also occurs with a static collider, so it does not appear to depend on character animation.. Contact is stable and runs interactively, but I cannot achieve close surface contact without sacrificing robustness. The cloth mid-surface remains visibly several millimetres from the authored collider, especially around curved or narrow areas such as the head, arms, and hands. Transparent rendering of visualMesh confirms this is not just render thickness. Reducing ClothMeshShape.bias reduces the gap, but eventually causes interpenetration around difficult geometry. Increasing it prevents penetration but makes the cloth visibly float. I’ve also checked scale, winding, normals, reasonably even triangles, timestep, and solver iterations. I want to avoid inflating the collider or adding anatomical proxy shapes. Is bias the only physical-clearance control, or does RealityKit apply an additional contact radius derived from simulation-mesh resolution? Is local conforming refinement the recommended way to achieve roughly 1–2 mm clearance while retaining robust real-time contact? Are there any specific topology requirements beyond evenly sized triangles? I can provide a minimal reproduction and screenshots if helpful although not sure I can share screenshots here?
4
0
5.2k
Sep ’26
BloomComponent lags behind the camera on iOS/macOS but not visionOS
BloomComponent (RealityKit 27) lags behind the geometry that produces it while the camera moves. The halo trails the emissive geometry by roughly one to three frames (eyeballed) and snaps back once the camera stops. Video attached — it's obvious at normal playback speed. visionOS renders the identical scene correctly. Only iOS and macOS / Mac Catalyst lag, which points at the non-visionOS compositing path rather than the effect itself. Filed as FB23960052. This should be fixed before 27 ships — bloom is unusable for anything with a moving camera in its current state on those platforms. Ruled out: Scope. Identical with BloomComponent(scope: .hierarchical) and .unbounded. .unbounded computes no per-entity screen-space bounds, so stale bounds are not the cause. Input handling. The camera is RealityKit's own .realityViewCameraControls(.orbit) — no gesture code of mine involved. Per-frame component writes. BloomOptionsComponent is set once and untouched during the drag. Geometry and camera transform. Both track perfectly; only the glow lags. Reproducer is ~130 lines, no assets — five emissive spheres inside a large inward-facing dark sphere: var material = PhysicallyBasedMaterial() material.baseColor = .init(tint: .black) material.emissiveColor = .init(color: .cyan) material.emissiveIntensity = 4 // ... root.components.set(BloomComponent(scope: .unbounded)) var options = BloomOptionsComponent() options.strength = 1 options.threshold = 1 options.blurRadius = 1 root.components.set(options) shown in: RealityView { content in content.camera = .virtual content.add(root) } .realityViewCameraControls(.orbit) Xcode 27.0 beta 4, iOS 27.0 SDK, Apple Silicon. Affected: iOS 27, macOS 27 / Mac Catalyst 27. Not affected: visionOS 27 (immersive space). Full project and screen recording attached to my radar.
4
0
2.1k
Sep ’26
GaussianSplatComponent — stale visibility set after close-range camera approach
Title: GaussianSplatComponent — stale visibility set after close-range camera approach (iOS 27 beta 4, still unfixed) Filed this as a radar during beta 3; still reproduces unchanged on beta 4, so posting here in case anyone from the RealityKit team can confirm it's tracked before this ships. Radar: FB23749559 (filed against beta 3, re-verified on beta 4) Summary On iOS, GaussianSplatComponent keeps a stale visibility set after the camera moves close to and then away from the splat. Moving the camera in close causes a large portion of splats to be culled — expected. But pulling the camera back out does not repopulate them. The visible set stays stale for as long as the current camera transform is held. The missing splats only return once the camera transform changes again — any pan/orbit/zoom delta triggers a recompute and the full splat snaps back. Net effect: after any close inspection, the splat renders permanently partially-culled until the user happens to nudge the camera. The recovery-on-movement / no-recovery-when-stationary signature suggests the iOS render path recomputes splat residency on camera-transform change events rather than every frame. This also reproduces in AR mode, with a completely different camera driver. Since the only thing the two paths share is the splat residency code, that's where the stale set has to be coming from — which also means there's no app-side fix available to me. The camera transform in AR is driven by ARKit, not by my code, so I can't force the perturbation that would otherwise work around it. visionOS 27, running the same component, is unaffected — the visible set is re-evaluated per frame and the splat stays complete. Steps to reproduce Attach a GaussianSplatComponent (~400k splats) to an entity in a bounded RealityView with orbit controls. Move the camera close until a visible portion of the splats is culled. Move the camera back out so the full splat is within the frustum. Hold the camera still. Expected: full splat visible once pulled back. Actual: previously-culled splats stay missing while the camera is stationary; they reappear only on the next camera transform delta. Configuration iOS 27 beta 4 (originally filed against beta 3) iPhone 16 Pro ~400k splats Bounded RealityView + orbit controls; also reproduces in AR Screenshots attached: full splat, stale-culled state after pull-back, and the close-range view that triggers it. This is still present in beta 4 and in the public beta. I'd like to know whether it's tracked, or whether I should plan around splats being unreliable on iOS 27. Thanks!
5
0
1.6k
Sep ’26
visionOS 27: allAnchors and anchorUpdates disagree on WorldAnchor transforms (different coordinate frame)
After updating to visionOS 27.0, WorldTrackingProvider.allAnchors returns WorldAnchor.originFromAnchorTransform values that do not match what anchorUpdates delivers for the same anchor, read at the same moment, both isTracked == true, with no .updated event in between. The difference is a rigid frame change, not drift: Same vertical offset on every anchor: -1.173 m. Identical across two apps and two days with the head at 1.13 m and 1.22 m, so a constant, not head height. Horizontal deltas vary per anchor, but inter-anchor distances are preserved, so it is a yaw plus translation. Stable over time, same across launches. anchorUpdates matches the ImmersiveSpace frame RealityKit draws in. allAnchors does not. On visionOS 26 both agreed. Same anchor, same instant: allAnchors (0.1486, -0.7143, 0.2231) anchorUpdates (0.1316, 0.4590, -1.0955) device pose (0.0073, 1.2195, -0.0038) Four other anchors, delta = allAnchors - anchorUpdates: ( 2.6933, -1.1733, -0.2514) (-0.0810, -1.1733, 1.1339) ( 0.1878, -1.1733, 0.9891) ( 2.6289, -1.1733, -0.1809) In our app this showed up as a placed model vanishing after a scene switch: we re-pinned the model root from allAnchors, and it landed below the floor behind the user. A standalone sample (one provider, one anchor, a green sphere from anchorUpdates, a red sphere from allAnchors) shows two spheres on visionOS 27.0, the red one below the floor: added F21942C4-… at (-0.031, 0.712, -0.976) update added F21942C4-… tracked=true at (-0.031, 0.712, -0.976) MISMATCH delta (-0.211, -1.173, 1.444) m allAnchors (-0.243, -0.461, 0.468) anchorUpdates (-0.031, 0.712, -0.976) tracked=true device (0.000, 1.129, -0.001) Questions: Is allAnchors now intentionally expressed in a different frame on visionOS 27, for example related to the new ARKitCoordinateSpaceProviding / coordinateSpace(correction:) API? The documentation for allAnchors does not mention it. If so, which frame is it, and how should it be converted to the ImmersiveSpace frame? Workaround we use: read transforms only from anchorUpdates into a per-id cache and never position from allAnchors. Filed as FB24799304. Minimal repro project attached to the feedback.
1
0
309
Sep ’26
RealityKit: How to read the current audio playback position, and sync audio across multiple entities?
Hi, in RealityKit, AudioPlaybackController exposes duration, gain, speed, play/pause/stop, and a completion handler, but I can't find a way to read the current playhead position while a resource is playing. I need this to trigger animations and other timed events in sync with the audio. Is there a supported way to read the current playback position on AudioPlaybackController? If not, is this planned any time soon? Also, is there a sample-accurate way to start/keep audio playback in sync across multiple entities in RealityKit? Appreciate any guidance, thanks.
1
0
1.1k
Sep ’26
Update entity state across devices through SharePlay on VisionOS
I am trying to create an app where I have multiple users modifying an entity at runtime similar to collaborative digital sculpting using SharePlay on VisionOS. When creating a collaborative experience using other 3D engines I have utilized an external server that devices send and update their state from serving as a global "source of truth". This doesn't seem to be the intended workflow for SharePlay which seemingly only allows for device to device messaging. While this works for a variety of apps it doesn't provide a smooth experience when multiple users are interacting with and manipulating the same 3D entity. Is there an intended way to do this for SharePlay? Would the best practice be to have one user serve as a host and establish a "source of truth" based on their app state? The closest example in documentation I have found is an example for using SharePlay in a drawing app. When a user is drawing they send the information for their realtime stroke through unreliable SharePlay messages and then, when they've finished, they send a reliable message with the final line. This technique makes sense for the given example since there is no scenario where multiple users would be drawing the same line, but it seemingly falls short for my use case since multiple users can sculpt in the same area of a given mesh. What would the intended approach be to achieve this with SharePlay? Is it simply a limitation of this type of networking? Thanks!
1
0
1.6k
Sep ’26
My first app as apple developer acadamey
As a 3D artist specializing in spatial installations, I faced the constant risk of designs looking balanced on screen but failing in physical proportions on the production floor. Starting with no traditional coding background, I embraced AI-assisted development and the CORE learning framework to master Swift, ARKit, and RealityKit. Real ini was built out of real practitioner necessity: turning the iPhone and iPad into on-site spatial validation studios that bridge digital art into physical reality. Download my app for free : https://apps.apple.com/id/app/real-ini/id6805716872
0
0
272
Sep ’26
Immersive space sometimes opens with a broken world origin (world anchors fail with error 200, recenter times out)
Hi all, We're seeing an intermittent issue in a native SwiftUI + RealityKit app on Vision Pro (visionOS 27.0.1, still there on 27.2). When we open a mixed ImmersiveSpace, roughly 1 in 10–30 opens, everything ends up offset from where it should be. Sometimes it's 50 cm, sometimes several meters, sometimes it's rotated. Holding the Digital Crown to recenter fixes it immediately. We are attaching things to the camera, like we have a window following the user and that is offset in these rare cases which makes the app not usable. In the device logs, a good open looks like this: backboardd (recenter) Setting new world origin with anchor ... backboardd ARWorldTrackingProcessor: Added world anchor ... successfully (x4) backboardd (recenter) Finished (error: (null)) setting set new world origin ... A bad open looks like this: backboardd (recenter) Setting new world origin with anchor ... arkitd SLAMAnchorUpdate: Failed to add anchor request ... NSPOSIXErrorDomain Code=4 backboardd ARWorldTrackingProcessor: Failed to add world anchor ... com.apple.arkit Code=200 "Adding an object failed." (x4) backboardd (recenter) timeout exceeded. Forcing fade in. Like whenever the bad state occurs it is because adding a world anchor seems to fail. I haven't tested with visionOS26 but the same app didnt run into that problem ever. So the system seems to fail adding its own origin anchors, gives up after 2 seconds and shows the space anyway. Has anyone else run into this? And is there any way to detect it from the app or trigger a recenter in code? Thanks!
Replies
2
Boosts
0
Views
609
Activity
2d
High fidelity gaming on Apple Vision Pro
I'm looking to find the best setup for a game on AVP that gives the best quality environment with gameplay but there seems to be a mix of possible solutions each with their own drawback. My ideal scenario is a high fidelity room/environment while making use of the PS VR2 Sense controllers to track the user's hands and give them more inputs via the controllers. Eg they hold a weapon tracked to the controllers with the buttons able to use the weapon, shooting and hitting enemies in the room. I want to aim more for highest qualtiy gaming than cartoony so opted for foveated streaming from the Mac as my ideal, but this seems limited to only having some base inputs from the hands and no controller inputs for hand and positional tracking from the controller movement (as they need the data from the Vision Pro camera but the game is technically running on the Mac and streamed to the VP) The only real solution for what I want here is to go via RealityKit as a native Vision Pro app but this reduces the game quality I had in mind as the entire thing is now rendering on the VP instead of the Mac. Am I missing something here or is this a real limitation today? To reitarate I'm looking for: highest fidelity game quality (rendering as much as possible on the Mac) tracking the VR2 Sense controller's positions (to track a weapon) in 3D space
Replies
1
Boosts
0
Views
998
Activity
3d
Async Entity(contentsOf:) is 15 to 20x slower while an ARView is rendering
While a non-AR ARView is on screen and rendering, await Entity(contentsOf:) takes about 26 s to load a 4.5 MB USDZ with 300 textured meshes. The same file loads in about 1.5 s with the synchronous Entity.load(contentsOf:), or with the async API while the view is hidden. iPhone 16 Pro, iOS 27.2Mac Catalyst, macOS 27.2await Entity(contentsOf:), view rendering22 to 26 s26 sEntity.load(contentsOf:), view rendering1.6 s1.1 to 1.7 sawait Entity(contentsOf:), view hidden1.4 s1.3 to 1.5 sModelEntity(contentsOf:) and Entity.loadAsync(contentsOf:) behave the same. A .reality file loads fast either way, and the iOS Simulator doesn't show it. A Time Profiler trace shows the main thread blocked in re::RenderThread::waitForFramePacing on every frame, so the @MainActor import only advances between frames. Neither workaround fits loading content into a scene the user is looking at: the sync load blocks the main thread, and hiding the view blanks it. Filed as FB25340264, with a sample project.
Replies
0
Boosts
0
Views
138
Activity
3d
visionOS : any non-nil preferredSurroundingsEffect enables a light-spill halo around VideoPlayerComponent video
Hi all — sharing a platform behavior I isolated while building a custom immersive video player, in case others hit it (and to ask whether it's intended). Finding On visionOS 27 (Xcode 27), an entity playing video via VideoPlayerComponent(videoRenderer:) gets a light-spill "screen glow" halo around its edges whenever the immersive space declares any non-nil preferredSurroundingsEffect. The halo follows the video's colors frame by frame. With no surroundings effect (nil), there is no halo. Quick repro Progressive immersive space + VideoPlayerComponent playing. Add .preferredSurroundingsEffect(.dim(intensity: 0.0)) → halo appears (even though the effect is visually a no-op). Remove it → halo disappears. Same with .colorMultiply(_:) and preset effects. Things I ruled out on device (Vision Pro) ImmersiveContentBrightness (.custom(1.0) vs .automatic): no difference. VideoPlayerComponent configuration (.stereo, spatial video mode, isPassthroughTintingEnabled) and component attach timing: no difference. Presenting the same video via VideoMaterial: no halo. Bright white SwiftUI content: no halo. Questions Is this coupling intended? The preferredSurroundingsEffect docs don't mention any effect on video playback. Is there a public way to keep a surroundings effect (e.g. room dimming) while avoiding the video light spill? I found none on VideoPlayerComponent. Current workaround: don't declare any surroundings effect in the modes where the halo is undesirable. Also filed via Feedback Assistant(FB25130748). Curious if anyone else has seen this or knows a switch.
Replies
2
Boosts
1
Views
2.9k
Activity
1w
RCP .reality file in iPadOS with behavior tree
I am having troubles with a .reality file in iPadOS 27.0.1. I have a Reality Composer Pro 3 project with an Animation Graph and Behavior Tree. The animation simply switches between a character walking and running, and the Behavior Tree simply changes the boolean to trigger the running state, waits, flips the boolean back, waits, and repeats. When I run the Simulate in RCP 3, it behaves as I expected - walks, runs, walks, runs, etc. I export the World scene as a .reality file, and open the .reality file in Xcode on macOS 27.2, and it behaves as I expected. But when I load the .reality file in an iPadOS app (iPadOS 27.0.1), the character does not switch between states. It starts the walking animation, but it never switches to the running animation. It appears the Behavior Tree isn't working as expected. Any idea what is wrong? Should I switch the iPad to the iPadOS 27 betas? Should I set the minimum OS target for the app to iPadOS 27? Is the Behavior Tree a known limitation with iOS or iPadOS 27?
Replies
1
Boosts
0
Views
570
Activity
1w
Photogrammetry with Object masks hangs and terminates with masks of objects
[PhotogrammetrySample]) with objectMask set and traps on my ios26.5.2, see the attached screenshot on feedback FB24379913 , it gets to the function and hangs . Even the folder reconstruction with lazy sequence as recommended from your video sample also doesn;t complete as it can't find alignment and displays the CoreOC.PhotogrammetrySession.Error error 6 means alignment failed. What can be done or do you guys expose any functions that can be used to check or trace or handle these internally The ObjectMasks are actually segmentation masks from an segmentation algorithm I will appreciate a timely response and willing to provide more clarity and informations, thank you so much for your understanding
Replies
3
Boosts
0
Views
1.4k
Activity
2w
On-screen RealityView starves CADisplayLink to 30 Hz on ProMotion (Mac Catalyst)
FB24536235 On Mac Catalyst under macOS 27, a plain CADisplayLink asking for CAFrameRateRange(minimum: 60, maximum: 60, preferred: 60) gets serviced at 30 Hz for as long as a RealityView is on screen in the same window. The link does nothing per tick but count, so there's nothing of mine to blame it on. RealityKit's own statistics overlay reads 60.41 fps in the same frame. Click a segmented control that removes the RealityView and the same link goes straight back to 60. Nothing else changes. That's the whole reproducer, and I've attached it to the radar. It only happens while the display panel is in ProMotion mode. Set the built-in to a fixed 60 Hz and it's correct again. With an external 60 Hz display attached the roles swap: the built-in is fine and the external drops to somewhere between 18 and 30, and setting the built-in to 60 Hz fixes that one too without touching the external's own settings. A raw MTKView presenting continuously at 60, at 120, and on a 120 Hz link presenting every second callback are all fine, so it isn't continuous presentation and it isn't the requested rate. It's RealityKit specifically. Worth knowing if you're testing: RealityView on Catalyst is an ARView underneath, so both paths give you the same answer. This is VERY rough for anything that puts RealityKit next to a UI. In an editor that's the sidebar, the inspector, gizmos, drag handles, every display-link-driven or UIKit animation in the window running at half rate around a viewport that stays smooth. Likely Related to FB24091347, which is the same defect seen as SwiftUI scroll judder. If you can reproduce either, please file a duplicate. Attached two screenshots; first with promotion enabled, second with promotion off. PLEASE fix this, it drives me crazy and there seems to be no workaround. On release day of macOS 27 our app will likely be blamed for it by users and my hands are tied. Thank you!
Replies
2
Boosts
1
Views
1.3k
Activity
3w
Spatial Scene API for iOS Apps
As part of the WWDC25 Keynote, a technology was announced that can present 2D images as 3D spatial scenes. This announcement is supported by a Press Release. ...developers can use the Spatial Scene API to make their app experience even more immersive. Zillow is taking advantage of the API for their Zillow Immersive app, allowing users to see images of homes and apartments with the rich depth and dimension that spatial scenes offer. The feature also appears in the Photos App on iOS 26 Developer Beta 1. Tapping "Spatial Scene" on any photo opens a view of that photo with a parallax effect. I've searched the WWDC sessions and new documentation and have come up short. Reaching out here for help. Is there any documentation for Spatial Scene API? Or any guidance on how to implement the spatial scene in iOS?
Replies
2
Boosts
2
Views
882
Activity
3w
RealityKit VideoMaterial renders 8-bit HEVC HLG/BT.2020 360° video with a purple tint in the visionOS Simulator (device is fine)
We paint a looping 360° video on the inside of a sphere with VideoMaterial(avPlayer:): an AVQueuePlayer plus AVPlayerLooper, a ModelEntity with an inward-facing equirectangular mesh, inside a .mixed ImmersiveSpace. No shaders, no VideoPlayerComponent. We could not find an official Apple sample that puts a VideoMaterial on a 360° sphere; this is the plain pattern from the VideoMaterial(avPlayer:) reference and the one used in threads like 734081. On Apple Vision Pro hardware the video renders correctly. In the visionOS Simulator (27.0, 26.5 and earlier 26.x, Xcode 27.0) the whole frame has a strong purple/magenta cast. Audio and timing are fine. The video (which we cannot share, so here is its metadata, image sample of issue on simulator provided below): MP4, HEVC Main, 8-bit 4:2:0, 3840x1920, tagged BT.2020 primaries, HLG transfer (ARIB STD-B67), BT.2020 non-constant matrix, both in the colr atom and in the SPS VUI. ffprobe -show_format -show_streams (trimmed): [STREAM] index=0 codec_name=hevc profile=Main codec_type=video codec_tag_string=hvc1 width=3840 height=1920 pix_fmt=yuv420p color_range=tv color_space=bt2020nc color_transfer=arib-std-b67 color_primaries=bt2020 r_frame_rate=30000/1001 avg_frame_rate=679200/22663 duration=37.771667 bit_rate=25030607 [SIDE_DATA] side_data_type=Stereo 3D type=2D horizontal_field_of_view=360000/1000 [/SIDE_DATA] [/STREAM] [STREAM] index=1 codec_name=aac profile=LC codec_type=audio codec_tag_string=mp4a sample_fmt=fltp sample_rate=48000 channels=2 channel_layout=stereo duration=37.758333 bit_rate=128176 [/STREAM] [FORMAT] nb_streams=2 format_name=mov,mp4,m4a,3gp,3g2,mj2 duration=37.778333 bit_rate=25160348 TAG:major_brand=mp42 TAG:minor_version=1 TAG:compatible_brands=isommp41mp42 [/FORMAT] AVFoundation's view of the track, from CMFormatDescriptionGetExtension on the loaded video track: primaries=ITU_R_2020 transfer=ITU_R_2100_HLG matrix=ITU_R_2020 bitsPerComponent=8 The playback code, reduced to the essentials: let item = AVPlayerItem(url: fileURL) let looper = AVPlayerLooper(player: player, templateItem: item) // retained player.actionAtItemEnd = .advance let material = VideoMaterial(avPlayer: player) let sphere = ModelEntity( mesh: try equirectSphere(radius: 12), // lat/long grid, normals inward, materials: [material] // UVs (u, 1 - v), inward winding ) sphere.position = [0, 1.5, 0] rootEntity.addChild(sphere) // rootEntity was added in RealityView { } player.play() What we have ruled out in a minimal sample project: Apple's own 360° HLS example (360 Lighthouse, HEVC Main 10, 10-bit, BT.709 SDR) plays with correct colours in the same sphere, same code, same simulator session. Rewriting our file's colr atom to BT.709 (three bytes, no re-encode) is honoured by AVFoundation (the track's CMFormatDescription then reports ITU_R_709_2 for primaries, transfer and matrix) but the tint stays. So the remaining differences between the failing file and Apple's working stream are 8-bit Main vs 10-bit Main 10, HLG/BT.2020 in the bitstream VUI vs BT.709, and local MP4 vs HLS. The minimal sample project (without the video) is available on request. Is this a known limitation of the simulator's VideoMaterial path with 8-bit HLG content, and is there a recommended way to get device-equivalent colour in the simulator? Feedback filed as FB24997147. Here is a sample of the result on the simulator of a cement column, painted white, but rendered purple on simulator:
Replies
1
Boosts
0
Views
3.3k
Activity
4w
Revealed keyboard drifts in progressive immersion, and no immersion style gives windows-in-front + Crown + passthrough cutouts
Environment: Apple Vision Pro, visionOS 27.0.1 (24M372). Magic Keyboard, with Settings › Awareness & Safety › Reveal Keyboards set to Always. Our app puts the user in an opaque virtual room while they work at their real desk, using app windows and Mac Virtual Display. A virtual desk is placed on the real desk. It is world-locked: positioned once from a PlaneAnchor (PlaneDetectionProvider), then a plain entity at fixed ImmersiveSpace coordinates. No AnchorEntity or WorldAnchor is involved. We offer two modes: .mixed, and .progressive for Digital Crown control. Each gives us only half of what desk work needs. 1. The revealed keyboard drifts in progressive immersion (FB25006789) In .progressive, the physical keyboard shown by keyboard awareness does not stay registered with the virtual desk. After the keyboard has been out of view for a few seconds (for example, while looking at a Mac Virtual Display window), it looks "floaty" when you look back: as the head turns, it slides against the desk. Turning right, the keyboard moves left farther than the desk does. What we checked: It happens in .progressive at any Crown amount, and in .full. With identical content in .mixed (no app passthrough cutout, so again only keyboard awareness shows the keyboard), it is much steadier. On 27.0.1, PlaneAnchor and DeviceAnchor coordinateSpace(correction: .rendered) are exactly equal to their .none transforms: 0 mm, 0°, scale 1. That held over 835 per-frame samples with ±50° yaw and up to 36 cm of head movement. So the rendered correction gives the app nothing to compensate with. The app's desk transform is constant while the drift is visible. upperLimbVisibility(.automatic) vs .visible and a .portrait progressive aspect ratio make no difference. Possibly related: https://developer.apple.com/forums/thread/796557 (FB19666209). 2. No immersion style combines what desk work needs (FB25006837) We need, at the same time: windows (our own and Mac Virtual Display) never covered by the room's walls, furniture or trees; the Digital Crown to bring the real surroundings back in; a passthrough cutout over the real desk, and a well-registered keyboard. .mixed: ✓ OcclusionMaterial cutouts reveal passthrough. ✓ The keyboard stays registered. ✗ RealityKit entities can render in front of windows, including Mac Virtual Display and window chrome. breakthroughEffect doesn't cover whole windows or Mac Virtual Display. ✗ The Crown can't adjust immersion. .progressive: ✓ Windows always stay in front of immersive content (as an Apple engineer confirmed in https://developer.apple.com/forums/thread/764556). ✓ The Crown adjusts immersion. ✗ OcclusionMaterial reveals nothing inside the portal, so there is no desk cutout. ✗ The keyboard drifts (issue 1). Because of issue 1 we have switched our default back to .mixed, and lost windows-in-front. Questions Is keyboard awareness in .progressive/.full expected to be registered less accurately than in .mixed? Is there a supported way to keep it registered with world-locked content, or to get the pose at which the system draws it? In .mixed, is there a supported way to keep windows (including Mac Virtual Display) in front of an app's immersive content? Or, in .progressive, a supported way to show passthrough in an app-defined region inside the portal? Any pointers or workarounds are appreciated.
Replies
0
Boosts
0
Views
1.6k
Activity
Sep ’26
How to use ClothGrabComponent?
I've simulated a piece of cloth using https://developer.apple.com/documentation/realitykit/physics-cloth-simulation I would like to grab the cloth like what I can in Marvellous Designer. I found the ClothGrabComponent at https://developer.apple.com/documentation/realitykit/clothgrabcomponent However, after I added the ClothGrabComponent to the cloth entity, I am still not able to grab it. Is there any sample code to show me how to grab the cloth? Thank you.
Replies
2
Boosts
0
Views
1.5k
Activity
Sep ’26
RealityKit asset compilation gives zero bounds to explicit meshes at matching USD prim paths
I filed Feedback Assistant report FB24967734 for this issue. When two USD scenes contain explicit polygon meshes at the same prim path and are compiled together with xcrun realitytool, one scene's mesh loads with zero bounds. Renaming the mesh prims so their paths are unique makes both meshes load correctly. Reproduce Download and expand the source ZIP from https://github.com/mesqueeb/RealitykitUsdMeshDisappearanceOnMatchingPaths/archive/refs/heads/main.zip, then run these commands from the extracted directory: xcrun realitytool compile --output-reality Fixtures/Compiled/inline-same-path.bundle/Contents/Resources/RealityKitContent.reality --platform macosx --deployment-target 27.0 Fixtures/Source/inline-same-path.rkassets xcrun realitytool compile --output-reality Fixtures/Compiled/inline-unique-path.bundle/Contents/Resources/RealityKitContent.reality --platform macosx --deployment-target 27.0 Fixtures/Source/inline-unique-path.rkassets swift run The same-path case has a mesh at /Root/MeshCube in both scenes. The control changes only the mesh paths to /Root/MeshCube_A and /Root/MeshCube_B. Current output inline-same-path: SceneA.usda / MeshCube: bounds=SIMD3<Float>(0.19999999, 0.2, 0.2) SceneB.usda / MeshCube: bounds=SIMD3<Float>(0.0, 0.0, 0.0) inline-unique-path: SceneA.usda / MeshCube_A: bounds=SIMD3<Float>(0.19999999, 0.2, 0.2) SceneB.usda / MeshCube_B: bounds=SIMD3<Float>(0.19999999, 0.2, 0.2) Expected output Both meshes in the matching-path case should have nonzero bounds: inline-same-path: SceneA.usda / MeshCube: bounds=SIMD3<Float>(0.19999999, 0.2, 0.2) SceneB.usda / MeshCube: bounds=SIMD3<Float>(0.19999999, 0.2, 0.2) inline-unique-path: SceneA.usda / MeshCube_A: bounds=SIMD3<Float>(0.19999999, 0.2, 0.2) SceneB.usda / MeshCube_B: bounds=SIMD3<Float>(0.19999999, 0.2, 0.2) Reproduced on macOS 27.0 (26A428), Xcode 27.0 (27A266a). The repo includes the USDA inputs, compiled RealityKitContent assets, and the Swift code that prints these bounds. Source and downloadable ZIP: https://github.com/mesqueeb/RealitykitUsdMeshDisappearanceOnMatchingPaths
Replies
0
Boosts
0
Views
613
Activity
Sep ’26
RealityKit cloth: reducing mesh collider air gap without penetration
I’m testing RealityKit cloth for garment-like cloth over an authored human-shaped triangle-mesh collider. The issue also occurs with a static collider, so it does not appear to depend on character animation.. Contact is stable and runs interactively, but I cannot achieve close surface contact without sacrificing robustness. The cloth mid-surface remains visibly several millimetres from the authored collider, especially around curved or narrow areas such as the head, arms, and hands. Transparent rendering of visualMesh confirms this is not just render thickness. Reducing ClothMeshShape.bias reduces the gap, but eventually causes interpenetration around difficult geometry. Increasing it prevents penetration but makes the cloth visibly float. I’ve also checked scale, winding, normals, reasonably even triangles, timestep, and solver iterations. I want to avoid inflating the collider or adding anatomical proxy shapes. Is bias the only physical-clearance control, or does RealityKit apply an additional contact radius derived from simulation-mesh resolution? Is local conforming refinement the recommended way to achieve roughly 1–2 mm clearance while retaining robust real-time contact? Are there any specific topology requirements beyond evenly sized triangles? I can provide a minimal reproduction and screenshots if helpful although not sure I can share screenshots here?
Replies
4
Boosts
0
Views
5.2k
Activity
Sep ’26
BloomComponent lags behind the camera on iOS/macOS but not visionOS
BloomComponent (RealityKit 27) lags behind the geometry that produces it while the camera moves. The halo trails the emissive geometry by roughly one to three frames (eyeballed) and snaps back once the camera stops. Video attached — it's obvious at normal playback speed. visionOS renders the identical scene correctly. Only iOS and macOS / Mac Catalyst lag, which points at the non-visionOS compositing path rather than the effect itself. Filed as FB23960052. This should be fixed before 27 ships — bloom is unusable for anything with a moving camera in its current state on those platforms. Ruled out: Scope. Identical with BloomComponent(scope: .hierarchical) and .unbounded. .unbounded computes no per-entity screen-space bounds, so stale bounds are not the cause. Input handling. The camera is RealityKit's own .realityViewCameraControls(.orbit) — no gesture code of mine involved. Per-frame component writes. BloomOptionsComponent is set once and untouched during the drag. Geometry and camera transform. Both track perfectly; only the glow lags. Reproducer is ~130 lines, no assets — five emissive spheres inside a large inward-facing dark sphere: var material = PhysicallyBasedMaterial() material.baseColor = .init(tint: .black) material.emissiveColor = .init(color: .cyan) material.emissiveIntensity = 4 // ... root.components.set(BloomComponent(scope: .unbounded)) var options = BloomOptionsComponent() options.strength = 1 options.threshold = 1 options.blurRadius = 1 root.components.set(options) shown in: RealityView { content in content.camera = .virtual content.add(root) } .realityViewCameraControls(.orbit) Xcode 27.0 beta 4, iOS 27.0 SDK, Apple Silicon. Affected: iOS 27, macOS 27 / Mac Catalyst 27. Not affected: visionOS 27 (immersive space). Full project and screen recording attached to my radar.
Replies
4
Boosts
0
Views
2.1k
Activity
Sep ’26
GaussianSplatComponent — stale visibility set after close-range camera approach
Title: GaussianSplatComponent — stale visibility set after close-range camera approach (iOS 27 beta 4, still unfixed) Filed this as a radar during beta 3; still reproduces unchanged on beta 4, so posting here in case anyone from the RealityKit team can confirm it's tracked before this ships. Radar: FB23749559 (filed against beta 3, re-verified on beta 4) Summary On iOS, GaussianSplatComponent keeps a stale visibility set after the camera moves close to and then away from the splat. Moving the camera in close causes a large portion of splats to be culled — expected. But pulling the camera back out does not repopulate them. The visible set stays stale for as long as the current camera transform is held. The missing splats only return once the camera transform changes again — any pan/orbit/zoom delta triggers a recompute and the full splat snaps back. Net effect: after any close inspection, the splat renders permanently partially-culled until the user happens to nudge the camera. The recovery-on-movement / no-recovery-when-stationary signature suggests the iOS render path recomputes splat residency on camera-transform change events rather than every frame. This also reproduces in AR mode, with a completely different camera driver. Since the only thing the two paths share is the splat residency code, that's where the stale set has to be coming from — which also means there's no app-side fix available to me. The camera transform in AR is driven by ARKit, not by my code, so I can't force the perturbation that would otherwise work around it. visionOS 27, running the same component, is unaffected — the visible set is re-evaluated per frame and the splat stays complete. Steps to reproduce Attach a GaussianSplatComponent (~400k splats) to an entity in a bounded RealityView with orbit controls. Move the camera close until a visible portion of the splats is culled. Move the camera back out so the full splat is within the frustum. Hold the camera still. Expected: full splat visible once pulled back. Actual: previously-culled splats stay missing while the camera is stationary; they reappear only on the next camera transform delta. Configuration iOS 27 beta 4 (originally filed against beta 3) iPhone 16 Pro ~400k splats Bounded RealityView + orbit controls; also reproduces in AR Screenshots attached: full splat, stale-culled state after pull-back, and the close-range view that triggers it. This is still present in beta 4 and in the public beta. I'd like to know whether it's tracked, or whether I should plan around splats being unreliable on iOS 27. Thanks!
Replies
5
Boosts
0
Views
1.6k
Activity
Sep ’26
visionOS 27: allAnchors and anchorUpdates disagree on WorldAnchor transforms (different coordinate frame)
After updating to visionOS 27.0, WorldTrackingProvider.allAnchors returns WorldAnchor.originFromAnchorTransform values that do not match what anchorUpdates delivers for the same anchor, read at the same moment, both isTracked == true, with no .updated event in between. The difference is a rigid frame change, not drift: Same vertical offset on every anchor: -1.173 m. Identical across two apps and two days with the head at 1.13 m and 1.22 m, so a constant, not head height. Horizontal deltas vary per anchor, but inter-anchor distances are preserved, so it is a yaw plus translation. Stable over time, same across launches. anchorUpdates matches the ImmersiveSpace frame RealityKit draws in. allAnchors does not. On visionOS 26 both agreed. Same anchor, same instant: allAnchors (0.1486, -0.7143, 0.2231) anchorUpdates (0.1316, 0.4590, -1.0955) device pose (0.0073, 1.2195, -0.0038) Four other anchors, delta = allAnchors - anchorUpdates: ( 2.6933, -1.1733, -0.2514) (-0.0810, -1.1733, 1.1339) ( 0.1878, -1.1733, 0.9891) ( 2.6289, -1.1733, -0.1809) In our app this showed up as a placed model vanishing after a scene switch: we re-pinned the model root from allAnchors, and it landed below the floor behind the user. A standalone sample (one provider, one anchor, a green sphere from anchorUpdates, a red sphere from allAnchors) shows two spheres on visionOS 27.0, the red one below the floor: added F21942C4-… at (-0.031, 0.712, -0.976) update added F21942C4-… tracked=true at (-0.031, 0.712, -0.976) MISMATCH delta (-0.211, -1.173, 1.444) m allAnchors (-0.243, -0.461, 0.468) anchorUpdates (-0.031, 0.712, -0.976) tracked=true device (0.000, 1.129, -0.001) Questions: Is allAnchors now intentionally expressed in a different frame on visionOS 27, for example related to the new ARKitCoordinateSpaceProviding / coordinateSpace(correction:) API? The documentation for allAnchors does not mention it. If so, which frame is it, and how should it be converted to the ImmersiveSpace frame? Workaround we use: read transforms only from anchorUpdates into a per-id cache and never position from allAnchors. Filed as FB24799304. Minimal repro project attached to the feedback.
Replies
1
Boosts
0
Views
309
Activity
Sep ’26
RealityKit: How to read the current audio playback position, and sync audio across multiple entities?
Hi, in RealityKit, AudioPlaybackController exposes duration, gain, speed, play/pause/stop, and a completion handler, but I can't find a way to read the current playhead position while a resource is playing. I need this to trigger animations and other timed events in sync with the audio. Is there a supported way to read the current playback position on AudioPlaybackController? If not, is this planned any time soon? Also, is there a sample-accurate way to start/keep audio playback in sync across multiple entities in RealityKit? Appreciate any guidance, thanks.
Replies
1
Boosts
0
Views
1.1k
Activity
Sep ’26
Update entity state across devices through SharePlay on VisionOS
I am trying to create an app where I have multiple users modifying an entity at runtime similar to collaborative digital sculpting using SharePlay on VisionOS. When creating a collaborative experience using other 3D engines I have utilized an external server that devices send and update their state from serving as a global "source of truth". This doesn't seem to be the intended workflow for SharePlay which seemingly only allows for device to device messaging. While this works for a variety of apps it doesn't provide a smooth experience when multiple users are interacting with and manipulating the same 3D entity. Is there an intended way to do this for SharePlay? Would the best practice be to have one user serve as a host and establish a "source of truth" based on their app state? The closest example in documentation I have found is an example for using SharePlay in a drawing app. When a user is drawing they send the information for their realtime stroke through unreliable SharePlay messages and then, when they've finished, they send a reliable message with the final line. This technique makes sense for the given example since there is no scenario where multiple users would be drawing the same line, but it seemingly falls short for my use case since multiple users can sculpt in the same area of a given mesh. What would the intended approach be to achieve this with SharePlay? Is it simply a limitation of this type of networking? Thanks!
Replies
1
Boosts
0
Views
1.6k
Activity
Sep ’26
My first app as apple developer acadamey
As a 3D artist specializing in spatial installations, I faced the constant risk of designs looking balanced on screen but failing in physical proportions on the production floor. Starting with no traditional coding background, I embraced AI-assisted development and the CORE learning framework to master Swift, ARKit, and RealityKit. Real ini was built out of real practitioner necessity: turning the iPhone and iPad into on-site spatial validation studios that bridge digital art into physical reality. Download my app for free : https://apps.apple.com/id/app/real-ini/id6805716872
Replies
0
Boosts
0
Views
272
Activity
Sep ’26