I am also interested in access to PCM data for a music visualizer. Spotify already provides access but my preferences to start with Apple since I am developing an app that spans different Apple devices and I would like to keep it in the same ecosystem.

In RealityKit and in Mixed Reality apps in general, the iPhone screen serves as a viewport onto virtual object anchored to a scene. Currently the user can rotate along the X and Y axis and the programmer can use Gyro to adjust viewport. But rotating along Z axis triggers a Landscape/Portrait animation which is very jarring to the user. So, preventing that animation is crucial for a seamless UX. One possible way to enable is to switch UIApplication.shared.connectedScenes.first?.effectiveGeometry.interfaceOrientation from readonly to readwrite. I think that might work. But, that would be an API change. Better yet, is there something that we can do today?

I'm getting this error on AVP visionOS 2.0, compiled with deployment targets 2.0 and 1.3 . Works on simulator. Did you find a solution?

I filed a feedback FB13787841. Would suggest everyone else do the same, more votes will escalate the suggestion.

Oops, wrong reply field: My guess is that MTKView may wrap apis that conflict with xrOS, akin to UIScreen.main.bounds. In the view controller's view public var metalLayer = CAMetalLayer() override func viewDidAppear(_ animated: Bool) { ... view.layer.addSublayer(pipeline.metalLayer) } and manually replace drawableSizeWillChange public func resize(_ viewSize: CGSize, _ scale: CGFloat) { ... metalLayer.contentsScale = scale metalLayer.drawableSize = viewSize }

My guess is that MTKView may wrap apis that conflict with xrOS, akin to UIScreen.main.bounds. In the view controller's view public var metalLayer = CAMetalLayer() override func viewDidAppear(_ animated: Bool) { ... view.layer.addSublayer(pipeline.metalLayer) } and manually replace drawableSizeWillChange public func resize(_ viewSize: CGSize, _ scale: CGFloat) { ... metalLayer.contentsScale = scale metalLayer.drawableSize = viewSize }

Some swift example code for using Metal here [EDIT] But, not passthrough; only fully immersive.

Well, bonjour MultipeerConnectivityService works. I added a test to MuPeer package here Or at least, it works across VisionOS and iPhone simulators. But, doesn't show up on iPad running iPadOS 17 Beta 5.

Full port of C++ to Swift is working. Example code is here Thanks to Warren Moore for C++ version.

Full port of C++ to Swift is working. Code is here

Full port of C++ to Swift is working. Code is here Thanks to Warren Moore for C++ version.

As for ar_data_providers_create_with_data_providers(...) From modifications to warrenm example (see above) am calling private func runWorldTrackingARSession() async { guard let arSession else { return err("arSession") } try? await arSession.run([worldTracking]) //WorldTrackingProvider } passing through the arSession and worldTracking via public func SpatialRenderer_InitAndRun(_ layerRenderer: LayerRenderer, _ arSession: ARKitSession, _ worldTracking: WorldTrackingProvider) { let renderThread = RenderThread(layerRenderer, arSession, worldTracking) renderThread.name = "Spatial Renderer Thread" renderThread.start() } called from @main struct FullyImmersiveMetalApp: App { @State var session = ARKitSession() //?? @State var worldTracking = WorldTrackingProvider() //?? init() {} var body: some Scene { WindowGroup { ContentView() } ImmersiveSpace(id: "ImmersiveSpace") { CompositorLayer(configuration: MetalLayerConfiguration()) { layerRenderer in SpatialRenderer_InitAndRun(layerRenderer, session, worldTracking) } }.immersionStyle(selection: .constant(.full), in: .full) } } All swift code compile but fails at runtime. Probably in another code block. YMMV

There is example code from Warren Moore found here using Apple's low level C API found here Thanks warrenm! However, I find the C++/ObjC++ APIs with super_long_snake_case_names to be obscure and non-idiomatic. So, am attempting to port to Swift. Already filed a Feedback requesting Swift APIs (FB12879060). And will post a separate question on some blockers in my attempt to refactor into idiomatic Swift.

Sorry for delay. Wound up using a UIHostingController and doing a custom hitTest, within a SwiftUI view model. This is within a MVVM architecture. Going deeper is rather complex and may change for xrOS. Any, here's the starting point clue: override func viewDidAppear(_ animated: Bool) { view.addSubview(pipeline.mtkView) pipeline.makeShader(for: root˚) pipeline.setupPipeline() pipeline.settingUp = false let touchView = SkyTouchView(touchDraw) // UIKit let menuView = MenuView(SkyFlo.shared.root˚, touchView) // SwiftUI let hostView = UIHostingController(rootView: menuView).view if let hostView { view.addSubview(hostView) hostView.translatesAutoresizingMaskIntoConstraints = false hostView.topAnchor.constraint(equalTo: view.topAnchor).isActive = true hostView.bottomAnchor.constraint(equalTo: view.bottomAnchor).isActive = true hostView.leftAnchor.constraint(equalTo: view.leftAnchor).isActive = true hostView.rightAnchor.constraint(equalTo: view.rightAnchor).isActive = true hostView.backgroundColor = .clear } }