Post

Replies

Boosts

Views

Activity

Reply to Dynamically change device orientation, and lock to that orientation
In RealityKit and in Mixed Reality apps in general, the iPhone screen serves as a viewport onto virtual object anchored to a scene. Currently the user can rotate along the X and Y axis and the programmer can use Gyro to adjust viewport. But rotating along Z axis triggers a Landscape/Portrait animation which is very jarring to the user. So, preventing that animation is crucial for a seamless UX. One possible way to enable is to switch UIApplication.shared.connectedScenes.first?.effectiveGeometry.interfaceOrientation from readonly to readwrite. I think that might work. But, that would be an API change. Better yet, is there something that we can do today?
Topic: UI Frameworks SubTopic: SwiftUI Tags:
Nov ’24
Reply to `MTKView` on visionOS
Oops, wrong reply field: My guess is that MTKView may wrap apis that conflict with xrOS, akin to UIScreen.main.bounds. In the view controller's view public var metalLayer = CAMetalLayer() override func viewDidAppear(_ animated: Bool) { ... view.layer.addSublayer(pipeline.metalLayer) } and manually replace drawableSizeWillChange public func resize(_ viewSize: CGSize, _ scale: CGFloat) { ... metalLayer.contentsScale = scale metalLayer.drawableSize = viewSize }
Topic: Graphics & Games SubTopic: General Tags:
Aug ’23
Reply to `MTKView` on visionOS
My guess is that MTKView may wrap apis that conflict with xrOS, akin to UIScreen.main.bounds. In the view controller's view public var metalLayer = CAMetalLayer() override func viewDidAppear(_ animated: Bool) { ... view.layer.addSublayer(pipeline.metalLayer) } and manually replace drawableSizeWillChange public func resize(_ viewSize: CGSize, _ scale: CGFloat) { ... metalLayer.contentsScale = scale metalLayer.drawableSize = viewSize }
Topic: Graphics & Games SubTopic: General Tags:
Aug ’23
Reply to Swift equivalent of ar_data_providers_create_with_data_providers and cp_time_to_cf_time_interval
As for ar_data_providers_create_with_data_providers(...) From modifications to warrenm example (see above) am calling private func runWorldTrackingARSession() async { guard let arSession else { return err("arSession") } try? await arSession.run([worldTracking]) //WorldTrackingProvider } passing through the arSession and worldTracking via public func SpatialRenderer_InitAndRun(_ layerRenderer: LayerRenderer, _ arSession: ARKitSession, _ worldTracking: WorldTrackingProvider) { let renderThread = RenderThread(layerRenderer, arSession, worldTracking) renderThread.name = "Spatial Renderer Thread" renderThread.start() } called from @main struct FullyImmersiveMetalApp: App { @State var session = ARKitSession() //?? @State var worldTracking = WorldTrackingProvider() //?? init() {} var body: some Scene { WindowGroup { ContentView() } ImmersiveSpace(id: "ImmersiveSpace") { CompositorLayer(configuration: MetalLayerConfiguration()) { layerRenderer in SpatialRenderer_InitAndRun(layerRenderer, session, worldTracking) } }.immersionStyle(selection: .constant(.full), in: .full) } } All swift code compile but fails at runtime. Probably in another code block. YMMV
Topic: Graphics & Games SubTopic: General Tags:
Aug ’23
Reply to Discover Metal for immersive apps - Example Code
There is example code from Warren Moore found here using Apple's low level C API found here Thanks warrenm! However, I find the C++/ObjC++ APIs with super_long_snake_case_names to be obscure and non-idiomatic. So, am attempting to port to Swift. Already filed a Feedback requesting Swift APIs (FB12879060). And will post a separate question on some blockers in my attempt to refactor into idiomatic Swift.
Topic: Graphics & Games SubTopic: General Tags:
Aug ’23
Reply to UITouch after SwiftUI Drag
Sorry for delay. Wound up using a UIHostingController and doing a custom hitTest, within a SwiftUI view model. This is within a MVVM architecture. Going deeper is rather complex and may change for xrOS. Any, here's the starting point clue: override func viewDidAppear(_ animated: Bool) { view.addSubview(pipeline.mtkView) pipeline.makeShader(for: root˚) pipeline.setupPipeline() pipeline.settingUp = false let touchView = SkyTouchView(touchDraw) // UIKit let menuView = MenuView(SkyFlo.shared.root˚, touchView) // SwiftUI let hostView = UIHostingController(rootView: menuView).view if let hostView { view.addSubview(hostView) hostView.translatesAutoresizingMaskIntoConstraints = false hostView.topAnchor.constraint(equalTo: view.topAnchor).isActive = true hostView.bottomAnchor.constraint(equalTo: view.bottomAnchor).isActive = true hostView.leftAnchor.constraint(equalTo: view.leftAnchor).isActive = true hostView.rightAnchor.constraint(equalTo: view.rightAnchor).isActive = true hostView.backgroundColor = .clear } }
Topic: UI Frameworks SubTopic: UIKit Tags:
Aug ’23
Reply to How can third-party iOS apps obtain real-time waveform / spectrogram data for Apple Music tracks (similar to djay & other DJ apps)?
I am also interested in access to PCM data for a music visualizer. Spotify already provides access but my preferences to start with Apple since I am developing an app that spans different Apple devices and I would like to keep it in the same ecosystem.
Topic: Media Technologies SubTopic: Audio Tags:
Replies
Boosts
Views
Activity
Jul ’25
Reply to Dynamically change device orientation, and lock to that orientation
In RealityKit and in Mixed Reality apps in general, the iPhone screen serves as a viewport onto virtual object anchored to a scene. Currently the user can rotate along the X and Y axis and the programmer can use Gyro to adjust viewport. But rotating along Z axis triggers a Landscape/Portrait animation which is very jarring to the user. So, preventing that animation is crucial for a seamless UX. One possible way to enable is to switch UIApplication.shared.connectedScenes.first?.effectiveGeometry.interfaceOrientation from readonly to readwrite. I think that might work. But, that would be an API change. Better yet, is there something that we can do today?
Topic: UI Frameworks SubTopic: SwiftUI Tags:
Replies
Boosts
Views
Activity
Nov ’24
Reply to MTKView is now available on visionOS but isn't working on visionOS 1.x
I'm getting this error on AVP visionOS 2.0, compiled with deployment targets 2.0 and 1.3 . Works on simulator. Did you find a solution?
Topic: Spatial Computing SubTopic: General Tags:
Replies
Boosts
Views
Activity
Jul ’24
Reply to Xcode Full Version on iPad Pro M4
I filed a feedback FB13787841. Would suggest everyone else do the same, more votes will escalate the suggestion.
Replies
Boosts
Views
Activity
May ’24
Reply to `MTKView` on visionOS
Oops, wrong reply field: My guess is that MTKView may wrap apis that conflict with xrOS, akin to UIScreen.main.bounds. In the view controller's view public var metalLayer = CAMetalLayer() override func viewDidAppear(_ animated: Bool) { ... view.layer.addSublayer(pipeline.metalLayer) } and manually replace drawableSizeWillChange public func resize(_ viewSize: CGSize, _ scale: CGFloat) { ... metalLayer.contentsScale = scale metalLayer.drawableSize = viewSize }
Topic: Graphics & Games SubTopic: General Tags:
Replies
Boosts
Views
Activity
Aug ’23
Reply to `MTKView` on visionOS
My guess is that MTKView may wrap apis that conflict with xrOS, akin to UIScreen.main.bounds. In the view controller's view public var metalLayer = CAMetalLayer() override func viewDidAppear(_ animated: Bool) { ... view.layer.addSublayer(pipeline.metalLayer) } and manually replace drawableSizeWillChange public func resize(_ viewSize: CGSize, _ scale: CGFloat) { ... metalLayer.contentsScale = scale metalLayer.drawableSize = viewSize }
Topic: Graphics & Games SubTopic: General Tags:
Replies
Boosts
Views
Activity
Aug ’23
Reply to Metal and VisionOS
Some swift example code for using Metal here [EDIT] But, not passthrough; only fully immersive.
Topic: App & System Services SubTopic: Core OS Tags:
Replies
Boosts
Views
Activity
Aug ’23
Reply to 'MultipeerConnectivityService' is unavailable in visionOS?
Well, bonjour MultipeerConnectivityService works. I added a test to MuPeer package here Or at least, it works across VisionOS and iPhone simulators. But, doesn't show up on iPad running iPadOS 17 Beta 5.
Replies
Boosts
Views
Activity
Aug ’23
Reply to Please add a Metal sample for visionOS
Full port of C++ to Swift is working. Example code is here Thanks to Warren Moore for C++ version.
Topic: App & System Services SubTopic: Core OS Tags:
Replies
Boosts
Views
Activity
Aug ’23
Reply to Discover Metal for immersive apps - Example Code
Full port of C++ to Swift is working. Code is here
Topic: Graphics & Games SubTopic: General Tags:
Replies
Boosts
Views
Activity
Aug ’23
Reply to Swift equivalent of ar_data_providers_create_with_data_providers and cp_time_to_cf_time_interval
Full port of C++ to Swift is working. Code is here Thanks to Warren Moore for C++ version.
Topic: Graphics & Games SubTopic: General Tags:
Replies
Boosts
Views
Activity
Aug ’23
Reply to Swift equivalent of ar_data_providers_create_with_data_providers and cp_time_to_cf_time_interval
As for ar_data_providers_create_with_data_providers(...) From modifications to warrenm example (see above) am calling private func runWorldTrackingARSession() async { guard let arSession else { return err("arSession") } try? await arSession.run([worldTracking]) //WorldTrackingProvider } passing through the arSession and worldTracking via public func SpatialRenderer_InitAndRun(_ layerRenderer: LayerRenderer, _ arSession: ARKitSession, _ worldTracking: WorldTrackingProvider) { let renderThread = RenderThread(layerRenderer, arSession, worldTracking) renderThread.name = "Spatial Renderer Thread" renderThread.start() } called from @main struct FullyImmersiveMetalApp: App { @State var session = ARKitSession() //?? @State var worldTracking = WorldTrackingProvider() //?? init() {} var body: some Scene { WindowGroup { ContentView() } ImmersiveSpace(id: "ImmersiveSpace") { CompositorLayer(configuration: MetalLayerConfiguration()) { layerRenderer in SpatialRenderer_InitAndRun(layerRenderer, session, worldTracking) } }.immersionStyle(selection: .constant(.full), in: .full) } } All swift code compile but fails at runtime. Probably in another code block. YMMV
Topic: Graphics & Games SubTopic: General Tags:
Replies
Boosts
Views
Activity
Aug ’23
Reply to Discover Metal for immersive apps - Example Code
There is example code from Warren Moore found here using Apple's low level C API found here Thanks warrenm! However, I find the C++/ObjC++ APIs with super_long_snake_case_names to be obscure and non-idiomatic. So, am attempting to port to Swift. Already filed a Feedback requesting Swift APIs (FB12879060). And will post a separate question on some blockers in my attempt to refactor into idiomatic Swift.
Topic: Graphics & Games SubTopic: General Tags:
Replies
Boosts
Views
Activity
Aug ’23
Reply to UITouch after SwiftUI Drag
Sorry for delay. Wound up using a UIHostingController and doing a custom hitTest, within a SwiftUI view model. This is within a MVVM architecture. Going deeper is rather complex and may change for xrOS. Any, here's the starting point clue: override func viewDidAppear(_ animated: Bool) { view.addSubview(pipeline.mtkView) pipeline.makeShader(for: root˚) pipeline.setupPipeline() pipeline.settingUp = false let touchView = SkyTouchView(touchDraw) // UIKit let menuView = MenuView(SkyFlo.shared.root˚, touchView) // SwiftUI let hostView = UIHostingController(rootView: menuView).view if let hostView { view.addSubview(hostView) hostView.translatesAutoresizingMaskIntoConstraints = false hostView.topAnchor.constraint(equalTo: view.topAnchor).isActive = true hostView.bottomAnchor.constraint(equalTo: view.bottomAnchor).isActive = true hostView.leftAnchor.constraint(equalTo: view.leftAnchor).isActive = true hostView.rightAnchor.constraint(equalTo: view.rightAnchor).isActive = true hostView.backgroundColor = .clear } }
Topic: UI Frameworks SubTopic: UIKit Tags:
Replies
Boosts
Views
Activity
Aug ’23