I'm assuming you want the IDR info in decode order rather than presentation order. This makes a difference if your movie contains bidirectionally predicted frames. If your H.264 video file is a QuickTime movie, you can do something like this: func getSyncInfo(url: URL) -> [Bool] { var idrInfo = [Bool]() let asset = AVURLAsset(url: url) guard let reader = try? AVAssetReader(asset: asset) else { return idrInfo } let videoTrack = asset.tracks(withMediaType: AVMediaType.video).first! guard let cursor = videoTrack.makeSampleCursorAtFirstSampleInDecodeOrder() else { return idrInfo } repeat { let syncInfo = cursor.currentSampleSyncInfo idrInfo.append(syncInfo.sampleIsFullSync.boolValue ? true : false) } while cursor.stepInDecodeOrder(byCount: 1) == 1 return idrInfo } This is unlikely, but if the CMSampleBuffer contains an access unit of a raw Annex B bitstream, then you have to: Get the raw data bytes from the CMBlockBuffers Remove encapsulation prevention bytes Search for start codes the look like 0x00 0x00 0x01 Parse the nal_unit_type byte following the start code Mask out the higher 3 bits to get nal_unit_type Check that the nal_unit_type is 5.

Take a look at his sample code: https://developer.apple.com/documentation/realitykit/rendering-stereoscopic-video-with-realitykit

If you're looking for a programmatic way of capturing spatial videos, check out https://developer.apple.com/videos/play/wwdc2024/10166/?time=6m39s

VNGenerateForegroundInstanceMaskRequest is what you want. https://developer.apple.com/videos/play/wwdc2023/10176/ explains how to do it.

If the foreground object is a person, you can use this sample code. https://developer.apple.com/documentation/vision/original_objective-c_and_swift_api/applying_matte_effects_to_people_in_images_and_video

This sample code may help.