I created a minimal sample project that reproduces a microphone issue with nearby SharePlay / Window Sharing on Apple Vision Pro. Sample project: https://github.com/JerryNee/mic-demo Screen recording: https://youtu.be/aMj_A_leJWU Issue I first implemented SharePlay through FaceTime and found that local microphone capture was unavailable. That seemed understandable because FaceTime itself uses the microphone. I then moved the app to a nearby SharePlay / Window Sharing session specifically to avoid a FaceTime call, but local microphone capture still fails. Local microphone capture works before starting nearby SharePlay. After starting a nearby SharePlay / Window Sharing session with GroupActivities, the same app fails at: try audioEngine.start() This does not require a FaceTime call. It happens with an in-room nearby SharePlay / Window Sharing session. The screen recording also goes silent when the Share Window UI appears. I checked the MP4 audio track with ffmpeg silencedetect; audio becomes silent at about 00:00:22.60, matching the moment nearby sharing starts: silence_start: 22.600612 silence_end: 55.051927 | silence_duration: 32.451315 Environment Two nearby Apple Vision Pro devices Xcode device list: visionOS 27.0 and visionOS 26.5 Xcode 27.0, build 27A5194q Frameworks: GroupActivities, AVFoundation, Speech Entitlement: com.apple.developer.group-session Microphone and speech recognition permissions granted Steps Run the sample on Apple Vision Pro. Tap Start Listening and speak. Transcription works. Tap Stop Listening. Tap Start Nearby Session. Share with another nearby Apple Vision Pro. Tap Start Listening again while the nearby session is active. audioEngine.start() fails. Ending the nearby session makes microphone capture work again. Minimal microphone code let recognizer = SFSpeechRecognizer() let audioEngine = AVAudioEngine() let request = SFSpeechAudioBufferRecognitionRequest() request.shouldReportPartialResults = true let session = AVAudioSession.sharedInstance() try session.setCategory(.record, mode: .measurement, options: [.duckOthers]) try session.setActive(true) let input = audioEngine.inputNode let format = input.outputFormat(forBus: 0) input.installTap(onBus: 0, bufferSize: 1024, format: format) { buffer, _ in request.append(buffer) } audioEngine.prepare() try audioEngine.start() Minimal nearby SharePlay code struct MicDemoActivity: Codable, GroupActivity, Transferable { static let activityIdentifier = "com.example.mic-demo.nearby-mic-repro" static var transferRepresentation: some TransferRepresentation { CodableRepresentation(contentType: .micDemoActivity) } var metadata: GroupActivityMetadata { var metadata = GroupActivityMetadata() metadata.type = .generic metadata.title = "Mic Demo Nearby Session" return metadata } } for await session in MicDemoActivity.sessions() { session.join() } _ = try await MicDemoActivity().activate() Questions Is local microphone capture expected to be unavailable during a nearby SharePlay / Window Sharing session? Is screen recording audio expected to become silent during nearby sharing? What is the recommended way to support nearby shared UI/state while still allowing local voice input?

Hi, We're building a mixed-immersive visionOS app in which the user holds a thin metal tool — a needle — near virtual content. We need the real needle to correctly occlude virtual content: when the needle (and especially its tip) is physically in front of a virtual object, the virtual object must not draw over it. We are solution-agnostic. Whether the right answer is LiDAR / scene-depth based, object-tracking based, hand-tracking based, or something else entirely — we just need needle occlusion to work. Here is what we've ruled out so far: ObjectTrackingProvider — blocked at enrolment. We can't even produce a usable reference object: the object-capture / scanning step doesn't pick up the needle at all (≈1 mm shaft, specular/metallic, almost no surface features). So object tracking never gets off the ground — it fails before runtime. SceneReconstructionProvider + OcclusionMaterial — only usable for large planar surfaces. Occlusion against a wall or floor is clean. But the mesh is too coarse for anything with finer geometry: a chair already gives very rough, ragged occlusion edges, and the needle is far below the mesh resolution — its shaft never appears in the mesh, so it never occludes at all. So this isn't only a needle problem: it's a mesh resolution problem that goes from rough (chair) to nonexistent (needle). Automatic hand occlusion — doesn't cover the tip. The hand is composited in front correctly, but the needle tip protrudes a few centimetres beyond the fingers, and virtual content draws over exactly that part. The needle tip is the part that most needs to occlude correctly, so this is blocking for us. The full Xcode sample project and the screen recording are in this GitHub repo: https://github.com/JerryNee/occlusion-demo (the video is at media/occlusion-demo.mp4). The screenshots below are stills from that recording, which has four stages, all with the same virtual cube: Occlusion off — baseline; the cube draws over everything in the room. Occlusion on, against a wall — the cube is occluded cleanly. Occlusion on, against a chair — the cube is occluded, but with rough, ragged edges. Occlusion on, with a needle — the needle does not occlude the cube at all. The same occlusion path therefore degrades from clean (wall) → rough (chair) → nonexistent (needle) as the real geometry gets finer. Questions / avenues we'd welcome guidance on (any one would unblock us): What is the recommended way to occlude an object this thin on visionOS — and is it possible at all today? Is there app-accessible per-pixel scene depth (analogous to ARFrame.sceneDepth on iOS) that we could use to occlude geometry the reconstruction mesh misses? Can the system's automatic hand / upper-limb occlusion be extended to include a thin instrument held in the hand — i.e. treat a held tool as part of the hand region? The tool has known, rigid geometry, so we considered attaching an OcclusionMaterial proxy mesh and positioning it from hand-tracking joints — but the needle's orientation within the grip is not determined by the hand pose (the same grip can hold the needle at many different angles), so a hand-joint proxy would be misaligned. Is there any supported way to recover the actual 6-DoF pose of a thin tool held in the hand (i.e. something that senses the tool itself, since the hand pose doesn't constrain it)? Is there any way to make object capture / scanning succeed for thin, specular objects, or an alternative enrolment path? Environment: Apple Vision Pro, visionOS 26.2. Full Xcode sample + screen recording: https://github.com/JerryNee/occlusion-demo Even a "not currently possible / known limitation" answer would help us plan our roadmap. Thank you!