Changed requirement to HLS instead of RTSP, and it just works. Going to stay away from unsupported/legacy protocols for now on.

Changed requirement to HLS instead of RTSP, and it just works. Going to stay away from unsupported/legacy protocols for now on.

I've decided to make an adjustment to the requirement of RTSP streaming to HTTP streaming. I think this is a much better option, as I've read Apple developed a HTTP streaming method in 2008 or something. So there would be much better support and it's unlikely the APIs are to change drastically in the future. I am okay with this solution as I would spend less time coding an app. RTSP is a legacy protocol and is not widely supported. I don't recommend anyone go down this path, especially if they are trying to prove a point for themselves. Just use the existing solutions.

I'm getting even more frames decoded by including the length, FU header, and payload only. But still, getting lots of errors. This is why I need to understand the format the decoder expects.

I was getting same problem. Checking the status within the callback will give more insight for VTDecompressionSessionDecodeFrame function.

Ok, I can get the status now. Just as I expected I have more debugging to do for decoding. But what I did to fix the issue was use this override for VTDecompressionSessionDecodeFrame() instead of the other option: func VTDecompressionSessionDecodeFrame( _ session: VTDecompressionSession, sampleBuffer: CMSampleBuffer, flags decodeFlags: VTDecodeFrameFlags, infoFlagsOut: UnsafeMutablePointer<VTDecodeInfoFlags>?, outputHandler: @escaping VTDecompressionOutputHandler ) -> OSStatus