I had similar problem a year back with usdz, the best and optimal solution is to use open-source GLB-layer, this shall convert the GLB into renderable 3D model at runtime. This third-party library follows khronos standards and specs so most material, animation works flawlessly in SceneKit. However you may encounter issue while triggering animation embedded in GLB in realitykit for iOS or visionOS as library lacks support for animation.

As far as apple docs states which are in beta phase now and also in Xcode beta version for VisionOS that there is no outside access to camera view in ARView API. Camera is handled from within ARView(RealityView) in this case. At this point its not clear if there will be any exposure when VisionOS is finally released. I have a similar use-case, the straight-forward solution now seems to be allow user to enter some kind of six-digit numeric code and start the immersive experience.

Is this feature available now in both iOS and visionOS platform