Post

Replies

Boosts

Views

Activity

ReplayKit captures AVPictureInPictureVideoCallViewController recursively — supported exclusion mechanism?
We are building an accessibility-focused iOS app that uses a user-initiated ReplayKit broadcast to recognize visible text and provide an on-device translation. To keep the translated result visible while the user is in another app, the current prototype presents the translation UI with AVPictureInPictureVideoCallViewController. The problem is that ReplayKit includes this PiP window in the captured frames. Once the captured frame is rendered back into the PiP content, the PiP captures itself recursively, obscures the source text, and can significantly degrade performance. Reproduction: Start a system broadcast using RPSystemBroadcastPickerView. Present AVPictureInPictureVideoCallViewController with the app's translation UI. Leave the host app and place the PiP window over another app. Inspect the CMSampleBuffer frames received by the broadcast upload extension. The frames contain the PiP window itself, producing a repeated "window inside window" image. We need a public, App Store-safe architecture that: keeps a small user-controlled translation surface visible across apps; allows the user to move or collapse it; prevents that app-owned floating surface from appearing in ReplayKit capture; uses only explicit user activation and documented APIs. Questions: Is there a supported public API or entitlement that excludes an app-owned PiP window, UIWindow, view, or layer from ReplayKit capture? If not, is there a supported alternative to PiP for a compact cross-app accessibility controller that the system excludes from capture? Is the broadcast upload extension expected to receive any metadata identifying the PiP region so it can be masked without relying on visual heuristics? We are not requesting private API access. A focused Xcode reproduction project has already been provided to Apple Developer Technical Support under an active TSI, and the related enhancement request is FB24319994. Environment: Xcode 26.6 (17F113) iPhone / iOS 18 and later ReplayKit broadcast upload extension AVPictureInPictureVideoCallViewController Any documented approach or confirmation that this is currently unsupported would be very helpful.
0
0
22
2h
ReplayKit captures AVPictureInPictureVideoCallViewController recursively — supported exclusion mechanism?
We are building an accessibility-focused iOS app that uses a user-initiated ReplayKit broadcast to recognize visible text and provide an on-device translation. To keep the translated result visible while the user is in another app, the current prototype presents the translation UI with AVPictureInPictureVideoCallViewController. The problem is that ReplayKit includes this PiP window in the captured frames. Once the captured frame is rendered back into the PiP content, the PiP captures itself recursively, obscures the source text, and can significantly degrade performance. Reproduction: Start a system broadcast using RPSystemBroadcastPickerView. Present AVPictureInPictureVideoCallViewController with the app's translation UI. Leave the host app and place the PiP window over another app. Inspect the CMSampleBuffer frames received by the broadcast upload extension. The frames contain the PiP window itself, producing a repeated "window inside window" image. We need a public, App Store-safe architecture that: keeps a small user-controlled translation surface visible across apps; allows the user to move or collapse it; prevents that app-owned floating surface from appearing in ReplayKit capture; uses only explicit user activation and documented APIs. Questions: Is there a supported public API or entitlement that excludes an app-owned PiP window, UIWindow, view, or layer from ReplayKit capture? If not, is there a supported alternative to PiP for a compact cross-app accessibility controller that the system excludes from capture? Is the broadcast upload extension expected to receive any metadata identifying the PiP region so it can be masked without relying on visual heuristics? We are not requesting private API access. A focused Xcode reproduction project has already been provided to Apple Developer Technical Support under an active TSI, and the related enhancement request is FB24319994. Environment: Xcode 26.6 (17F113) iPhone / iOS 18 and later ReplayKit broadcast upload extension AVPictureInPictureVideoCallViewController Any documented approach or confirmation that this is currently unsupported would be very helpful.
Replies
0
Boosts
0
Views
22
Activity
2h