Rendering scene in RealityView to an Image

Is there any way to render a RealityView to an Image/UIImage like we used to be able to do using SCNView.snapshot() ?

  • ImageRenderer doesn't work because it renders a SwiftUI view hierarchy, and I need the currently presented RealityView with camera background and 3D scene content the way the user sees it
  • I tried UIHostingController and UIGraphicsImageRenderer like
extension View {
    func snapshot() -> UIImage {
        let controller = UIHostingController(rootView: self)
        let view = controller.view

        let targetSize = controller.view.intrinsicContentSize
        view?.bounds = CGRect(origin: .zero, size: targetSize)
        view?.backgroundColor = .clear

        let renderer = UIGraphicsImageRenderer(size: targetSize)
        return renderer.image { _ in
            view?.drawHierarchy(in: view!.bounds, afterScreenUpdates: true)
        }
    }
}

but that leads to the app freezing and sending an infinite loop of

[CAMetalLayer nextDrawable] returning nil because allocation failed.

Same thing happens when I try

        return renderer.image { ctx in
            view.layer.render(in: ctx.cgContext)
        }

Now that SceneKit is deprecated, I didn't want to start a new app using deprecated APIs.

Answered by DTS Engineer in 857122022

Hello,

Take a look at ARView's snapshot(saveToHDR:completion:).

You may made need to composite additional UI content to show what the user sees.

Oh, so this is for a new app where we use CoreML and machine vision to detect animals in the AR scene and show details on them.

I just realized, when I want to convert our existing SceneKit-based production Glasses-Try-On app to RealityKit, I'll face the same problem. In our glasses app, allowing the user to take a screenshot of them with Virtual Glasses on is a core-feature for which we need 3D Scene + Camera background (without the UI).

Hello,

Take a look at ARView's snapshot(saveToHDR:completion:).

You may made need to composite additional UI content to show what the user sees.

Thank you, yes I ended up wrapping ARView using UIViewRepresentable.

I was hoping I can remove one UIViewRepresentable from my code by going full RealityView, but it is what it is.

Rendering scene in RealityView to an Image
 
 
Q