Rendering the currently visible frame of a paused AVPlayer with custom video composition

Question

Created Mar ’21

Replies 3

Boosts 0

Views 1.9k

Participants 2

We use an AVPlayer for video playback with a custom video composition to render the video frames. Occasionally, we replace the item's composition with a new one to alter the rendering. Then, we acquire a new pixel buffer with hasNewPixelBuffer(forItemTime:) and copyPixelBuffer(forItemTime:, itemTimeForDisplay:) on the AVPlayerItemVideoOutput instance to display the changes. We get the item time with itemTime(forHostTime: CACurrentMediaTime()) on the output. However, when the player is paused there is no new pixel buffer and hasNewPixelBuffer(forItemTime:) returns false. How can we re-render the current frame?

https://developer.apple.com/library/archive/qa/qa1966/_index.html here it says that resetting the video composition should enable the re-rendering, which does not seem to work for us. When setting the composition a second time, the one set just before is applied instead, causing it to be always being one behind. Also the video "jumps" a little, meaning a slightly different frame is rendered.

Boost

Answer 1

Media Engineer OP

Apple

Mar ’21

Have you tried using AVAssetImageGenerator?

0

Answer 2

MaikZarnbach OP

May ’21

I tried using AVAssetImageGenerator with the following code:

        self.imageGenerator.videoComposition = self.composition
        self.imageGenerator.requestedTimeToleranceAfter = .zero
        self.imageGenerator.requestedTimeToleranceBefore = .zero
        do {
            var actualTime: CMTime = .indefinite
            let image = try self.imageGenerator.copyCGImage(at: self.avPlayer.currentTime(), actualTime: &actualTime)
        } catch {
            // ...
        }

Although I set the tolerances to 0 and the player is paused often actualTime.seconds is a little bit different from self.avPlayer.currentTime().seconds (e.g. 3.8 vs 4.0). Even if the times are the same copyCGImage returns a different image than the one I expect at the current time (the frame that was returned before with this time). Am I doing anything wrong to retrieve precise frames?

0

Answer 3

Media Engineer OP

Apple

Jun ’21

Setting the tolerances to zero should give you the exact frame. Please file a bug report using the feedback assistant.

0