CAMetalLayer VS AVSampleBufferDisplayLayer ( gpu usage, performance, ...)

Question

Created May ’24

Replies 1

Boosts 0

Views 1.1k

Participants 2

I am a VOIP app developer. I am planning to develop a VOIP app on iOS using WebRTC that operates in PiP (Picture-in-Picture) mode. Since MTKView (CAMetalLayer) cannot be used in PiP mode, I am considering using AVSampleBufferDisplayLayer. Regarding this, I am curious about the performance differences between CAMetalLayer and AVSampleBufferDisplayLayer. As far as I know, CAMetalLayer utilizes the GPU.

Does AVSampleBufferDisplayLayer also render using the GPU? If AVSampleBufferDisplayLayer renders using the GPU, will the rendering performance be similar? => Based on tests, there seems to be no difference in CPU usage between the two, which leads me to speculate that AVSampleBufferDisplayLayer also uses the GPU. If both use the GPU and there are no performance differences, is there a significant advantage to using CAMetalLayer? Thank you in advance.

Boost

Answer 1

DTS Engineer OP

Apple

May ’24

Accepted Answer

Recommended

Hello,

AVSampleBufferDisplayLayer will leverage the GPU and like Core Image may optimize behavior dynamically based on the underlying hardware and operating system.

To compare with a Metal-based implementation, profile your app with Instruments using the Metal System Trace template. Feel free to post screenshots of Instruments to discuss.

Metal is a low-level framework dedicated to the GPU so you have finer grain control over your rendering pipeline. This comes at the cost of high-level features offered by AVFoundation and WebRTC.

Overall AVSampleBufferDisplayLayer is probably better suited for your needs in this case and is expected to perform well.

1