How many concurrent VTCompressionSession / VTDecompressionSession can an app run, and can the limit be queried?

Question

Created 7h

Replies 4

Boosts 0

Participants 2

This post is from the WWDC26 Video Q&A.

I'm batch-transcoding a library of clips into downscaled editing proxies. If I kick off two hardware transcodes at once, the encoder reliably falls over, so right now I run everything serially: each clip gets fully decoded, encoded, and muxed before the next one starts. It works, but it's slow, and the decode and encode hardware are mostly idle waiting on each other.

A few things I can't pin down from the docs:

Is there an actual ceiling on how many VTCompressionSession / VTDecompressionSession instances can be live at once, and does it depend on the device, the codec, or the resolution?
Can I query that ceiling at runtime? I'd rather size my concurrency up front than find it by crashing.
Decode and encode are separate hardware blocks, so can I safely run a decode session for one clip while the previous clip is still encoding, or does VideoToolbox serialize them anyway?
When I do go over the limit, what should I be checking so I can back off cleanly? Right now I just get a crash instead of an error I can catch.

Anything that gets me off the fully-serial pipeline would help. Thank you

Answered by Engineer in 891461022

It's certainly not OK that the device panics in this situation. If possible, I'd ask you to file a FBA with the panic .ips file and also a test Xcode project that can reproduce the panic. If you can do that, please post the FBA number here so we can ensure it's routed promptly to the right team.

https://developer.apple.com/feedback-assistant/

It's very common for iPhones to be transcoding video, so maybe there is something novel about how your code is approaching this that isn't anticipated and handled correctly. So a test project may be vital to help engineers at Apple reproduce the panic and fix the cause.

Returning to the other parts of the question: you wrote that this is batch-transcoding, correct? So I would presume that you're interested in throughput rather than real-time needs. It's valid to operate multiple VTDecompressionSessions and multiple VTCompressionSessions. Each EncodeFrame/DecodeFrame call will block at a back-pressure threshold.

Answer 1

Engineer OP

Apple

7h

The limits should be high enough that they don't easily get into your way, and by the time you reach them you should be getting error codes rather than crashes. I suspect there is a different kind of issue here given that you say it's crashing. Are you able to share a crash log, either by attaching it here in the forum question, or to an FBA?

Attaching a .ips file will make it quicker to review.

To make it clear: you can absolutely operate many decompression sessions and compression sessions at a time.

Answer 2

chris.dcsong OP

6h

You're right that the app-level crash was on us. It was a withoutActuallyEscaping misuse: under concurrency our stall watchdog could cancel a clip while AVFoundation was still holding a heartbeat block we'd handed to AVAssetWriterInput.requestMediaDataWhenReady, so the closure outlived the non-escaping assertion and trapped. Fixed, and the app stopped crashing.

What actually stopped us from running concurrent transcodes wasn't an app crash or an error code, though. With that fix in, two concurrent 4K60 HEVC hardware encode sessions reliably kernel-panic the device. The whole phone powers off and reboots partway through the batch. It isn't thermal (the phone was cool) and it isn't jetsam/OOM. The panic is in the video encoder's IOMMU:

panic(...): "dart-ave1 (...): DART(CPUDART) error: SID 0 PTE invalid exception on read of
DVA 0x10001ff4080 (CTE 0x10 SEG 0 PTE 0x7fd) ..." @AppleT8110DART.cpp:2369

iPhone 16 Pro, iOS 26.5 (23F77). Running the transcodes serially gets through the whole library clean.

I've got the full panic .ips, but it seems like I can't attach it here though. Is two concurrent 4K60 hardware HEVC encodes simply more than this encoder will take, or is the panic itself the bug? What I really want to know is whether there's a supported way to keep the decoder and encoder both busy across a batch transcode, or whether serial is the right call at this resolution.

Answer 3

Engineer OP

Apple

5h

Recommended

It's certainly not OK that the device panics in this situation. If possible, I'd ask you to file a FBA with the panic .ips file and also a test Xcode project that can reproduce the panic. If you can do that, please post the FBA number here so we can ensure it's routed promptly to the right team.

https://developer.apple.com/feedback-assistant/

It's very common for iPhones to be transcoding video, so maybe there is something novel about how your code is approaching this that isn't anticipated and handled correctly. So a test project may be vital to help engineers at Apple reproduce the panic and fix the cause.

Returning to the other parts of the question: you wrote that this is batch-transcoding, correct? So I would presume that you're interested in throughput rather than real-time needs. It's valid to operate multiple VTDecompressionSessions and multiple VTCompressionSessions. Each EncodeFrame/DecodeFrame call will block at a back-pressure threshold.

Answer 4

chris.dcsong OP

1h

Correction on resolution: each transcode decodes a 4K60 HEVC HDR source and encodes a 720p HEVC proxy (not 4K60 on the encode), two running at once. The panic is on the encode/AVE side. FBA: FB23045975. Thank you