How to Achieve Volumetric Lighting (Light Shafts) in RealityKit on visionOS?

Hello,

Not aware of any Apple samples or documentation covering ray-marching techniques with RealityKit.

You might want to try a web search for "ray marching RealityKit" and see what you find.

Also, it's likely that other developers are interested in ray marching so please consider sending us a request for such a sample using the Feedback Assistant.

Lastly, given that Metal interoperates with RealityKit consider using Metal ray tracing to render parts of your scene to achieve the effect.

@DTS Engineer Thank you for your reply. I understand that your advice about using Metal ray tracing to render parts of the scene suggests integrating it with RealityKit. Are you referring to using low-level meshes in this context? To be honest, this answer alone doesn’t give me any clear idea of what to do. I have very little experience with Metal itself.

Allow me to clarify with some suggested resources:

Altering RealityKit Rendering with Shader Functions

Explore advanced rendering with RealityKit 2

Note: there may be additional sessions and documentation so take a look around. This should provide a good starting point.

Note: Metal ray tracing may not work directly given RealityKit's support for Metal Surface Shaders and Geometry Modifiers via CustomMaterial.

Note: re: "You might want to try a web search for "ray marching RealityKit" and see what you find." we encourage you to do this if you haven't already.

@DTS Engineer On visionOS, RealityKit does not support CustomMaterial (Surface Shader / Geometry Modifier), so your advice isn’t applicable and is off-target. Please provide the optimal guidance for visionOS, assuming Shader Graph Material (ShaderGraphMaterial), and include concrete API names, sample code, and references to the latest official documentation / WWDC session numbers.

We're unable to provide what you seek as there isn't something akin to a "ray marching recipe" for visionOS with RealityKit.

Instead consider ray marching (an exercise for the reader) or ray tracing with Metal which you can do using the Compositor Services framework.

@DTS Engineer Thank you for your response. From your explanation, I understand that RealityKit itself does not support ray marching and that the Compositor Services framework is only applicable for apps that leverage Metal-based rendering.

I also acknowledge that my original question lacked details regarding the intended use of ARKit features. My assumption is to utilize ARKit capabilities other than hand tracking or world tracking.

To confirm, is it correct to understand from your official guidance that, under these conditions, there is currently no supported approach to implement ray marching in visionOS with RealityKit?

Re: "... there is currently no supported approach to implement ray marching in visionOS with RealityKit?"

It's likely that you can achieve ray marching with ShaderGraphMaterial but we're unable to provide the exact implementation details, and don't have sample code or documentation on that particular technique.

In other words, we don't have a recipe to share so it's something you'll need to figure it out. Asking questions here is a good way to continue building up a recipe.

Alternatively, Compositor Services supports Metal ray tracing which is capable of rendering excellent results likely better than a ray marching technique.

FYI have been asking some colleagues for additional suggestions re: RK and ray marching.

Will share back here if they're able to provide more guidance.

Also, Metal can render for ARKit. See Displaying an AR Experience with Metal (an iOS sample but same principles apply).

I have also submitted this request via Feedback Assistant. FB Number: FB19350167