CVPixelBuffer auto release does not work if released in another thread

I'm struggling with memory overflow issues connected to my CVPixelBuffer allocations and processing. My code works fine on iOS 16.x releasing the buffers when they are not referenced anymore but doesn't release anything on iOS 15.x.

Here is what I do:

  • I have a video recorder that calls captureOutput on every frame.
  • I copy CVPixelBuffer in this same thread
  • Copied buffer then getting sent to another thread for processing.
  • After processing is done in this second thread I have no more references to the copied buffer and expect it to be released.
  • The buffer is released in iOS 16.x but isn't released in iOS 15.x!

What should I do to get it working in iOS 15.x?

class VideoProcessor {

    lazy var procQueue: DispatchQueue = {
        DispatchQueue(label: "mlprocque", qos: .userInteractive)
    }()

    func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
        guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
            return
        }
        let copiedPixelBuffer = pixelBuffer.copy()
        procQueue.async {
            self.process(pixelBuffer: copiedPixelBuffer)
        } //by the end of this async I expect the buffer to be released
    }

    func process(pixelBuffer: CVPixelBuffer) {
        // do something with pixelBuffer
    }
}

extension CVPixelBuffer {
    func copy() -> CVPixelBuffer {
        precondition(CFGetTypeID(self) == CVPixelBufferGetTypeID(), "copy() cannot be called on a non-CVPixelBuffer")

        var _copy: CVPixelBuffer?
        return autoreleasepool {
            CVPixelBufferCreate(
                    nil,
                    CVPixelBufferGetWidth(self),
                    CVPixelBufferGetHeight(self),
                    CVPixelBufferGetPixelFormatType(self),
                    nil,
                    &_copy)
            guard let copy = _copy else {
                fatalError()
            }

            let commonKeys = NSSet(array: CMVideoFormatDescriptionGetExtensionKeysCommonWithImageBuffers() as! [Any])

            let propagatedAttachments = NSDictionary(dictionary: CVBufferGetAttachments(self, .shouldPropagate)!)
            propagatedAttachments.enumerateKeysAndObjects { key, obj, stop in
                if commonKeys.contains(key) {
                    CVBufferSetAttachment(copy, key as! CFString, obj as AnyObject, .shouldPropagate)
                }
            }

            let nonPropagatedAttachments = NSDictionary(dictionary: CVBufferGetAttachments(self, .shouldNotPropagate)!)
            nonPropagatedAttachments.enumerateKeysAndObjects { key, obj, stop in
                if commonKeys.contains(key) {
                    CVBufferSetAttachment(copy, key as! CFString, obj as AnyObject, .shouldNotPropagate)
                }
            }

            CVPixelBufferLockBaseAddress(self, .readOnly)
            CVPixelBufferLockBaseAddress(copy, CVPixelBufferLockFlags())

            for plane in 0..<CVPixelBufferGetPlaneCount(self) {
                let dest = CVPixelBufferGetBaseAddressOfPlane(copy, plane)
                let source = CVPixelBufferGetBaseAddressOfPlane(self, plane)
                let height = CVPixelBufferGetHeightOfPlane(self, plane)
                let bytesPerRow = CVPixelBufferGetBytesPerRowOfPlane(self, plane)

                memcpy(dest, source, height * bytesPerRow)
            }

            CVPixelBufferUnlockBaseAddress(copy, CVPixelBufferLockFlags())
            CVPixelBufferUnlockBaseAddress(self, .readOnly)

            return copy
        }
    }
}

If I remove procQueue.async and just call process(...) in the same thread then it works on iOS 15 and releases the memory.

Try to wrap the content of the procQueue.async closure in an autoreleasepool. The queue has got it's own autoreleasepool that don't always drain after each closure is executed probably.

CVPixelBuffer auto release does not work if released in another thread
 
 
Q