Okay so what I've discovered so far is that for some reason the toolbars NSToolBarDelegate is being reset to nil (in the code above you can see that I indeed set it) and thus no items are being added. I have absolutely no idea why though and tested countless different setups. The same exact app runs fine on Sonoma, so what could have changed here?

Hi, you likely forgot to reapply the material to the mesh after updating its parameters. RealityKit Materials are value types. I always use a helper function like this: extension Entity { func modifyMaterials(_ closure: (Material) throws -> Material) rethrows { try children.forEach { try $0.modifyMaterials(closure) } guard var comp = components[ModelComponent.self] as? ModelComponent else { return } comp.materials = try comp.materials.map { try closure($0) } components[ModelComponent.self] = comp } }

You could use a SwiftUI 3D Transform and apply it on the view (https://developer.apple.com/documentation/swiftui/view/rotation3deffect(_:axis:anchor:anchorz:perspective:)) or put your view into a RealityKit Attachment and then rotate the Attachment Entity (https://developer.apple.com/documentation/realitykit/transform/rotation).

Could you share the console crash log for more context? One idea coming to my mind: updateImage() might be called on a non-main thread. And afaik ModelEntity should be created on the main- tread. So you could try marking the function as @MainActor.

Same exact issue here, I think this is a bug on Apples side. Any workarounds to fix this?

Instead of context.scene.performQuery did you try context.entities(matching: query, updatingSystemWhen: .rendering)?

Feedback filed: FB13659588 Thank you!

Hi, thanks so much for the quick reply! I was afraid that would be the case, seems kinda wasteful with the mesh copy :/ Will definitely file an enhancement request! I got it working now: https://streamable.com/7nakkb But instead of using the ShaderGraphMaterial I put a plane behind the portal with an occlusion material. AFAIK ShaderGraphMaterial does not allow me to discard fragments though, correct? And I should probably be able to still use PBR materials instead of Unlit if I give both models the same ImageBasedLightReceiverComponent? Regarding ClippingPlane I'm curious: what would be a sample use case for the clipping inside of the portal world?

Hi, I've found that you can retrieve the RealityKit Scene from an entity, once added (subscribe to the event). Then you can use the Scene's Raycast methods, e.g: https://developer.apple.com/documentation/realitykit/scene/raycast(from:to:query:mask:relativeto:) For that to work though you would need to create CollisionShapes for either the detected planes or the reconstructed world mesh geometry.

I just found this: https://developer.apple.com/documentation/arkit/worldtrackingprovider/4218774-querypose which sounds promising if I enable ARKit tracking. Will give it a go. Can someone from the RealityKit confirm that this would be the way to go? Also there is https://developer.apple.com/documentation/realitykit/anchoringcomponent/target-swift.enum/head Does this also only work when ARKit is enabled? So far I wasn't able to run it successfully in the Simulator.

You could create a custom component containing your data (https://developer.apple.com/documentation/realitykit/component) and assign it to your entity. Then on tap check if such a component is available and read/display the data. Alternatively you could also subclass Entity and store some data within that custom class.

Hi, on first glimpse that looks to me like either you misspelled the function name of your geometry modifier or your metal file isn't being compiled. If you are using the default metal library make sure that you have your geometry modifier function available like so: #include <metal_stdlib> #include <RealityKit/RealityKit.h> using namespace metal; [[visible]] void nameOfYourGeometryModifier(realitykit::geometry_parameters params) { // … your shader }

You could use https://developer.apple.com/documentation/realitykit/arview/debugoptions-swift.struct/showfeaturepoints but that's rather aimed for debugging purposes. The other option I see would be to draw the points manually on top of your rendered scene with a custom post processing effect.

Looks to me like you haven't configured EnvironmentTexturing in your ARConfiguration https://developer.apple.com/documentation/arkit/arworldtrackingconfiguration/2977509-environmenttexturing or you've set disableAREnvironmentLighting in the ARViews RenderOptions – https://developer.apple.com/documentation/realitykit/arview/renderoptions-swift.struct/disablearenvironmentlighting

I think RealityKit is unfortunately not supported in the Simulator, so it can't find the initializer. If you have an Apple Silicon may you could run RealityKit there though via the »My Mac (Designed for iPad)« option.