Post

Replies

Boosts

Views

Activity

Reply to Working with kCVPixelFormatType_96VersatileBayerPacked12
I’ve been working on the same problem on an iPhone 17 Pro Max. There doesn’t appear to be a one-call Core Image debayer path for the kCVPixelFormatType_96VersatileBayerPacked12 buffers delivered by AVCaptureVideoDataOutput, but I was able to process them directly with Metal. On the Main camera’s 4224×3024 btp2 format I receive 6336 bytes/row, which is exactly 12 bits/sensel. The buffer on this device decodes correctly using a MIPI RAW12-style 2-pixels/3-bytes layout. The sample buffers also contain ProRes RAW attachments for Bayer pattern, black/white level, WB multipliers, gain and the Camera RGB→XYZ D65 color matrix. The pipeline I currently have working is: unpack 12-bit → normalize black/white → apply WB to Bayer sensels → debayer → Camera RGB→XYZ D65 matrix → gain → display transform I’m using CVMetalBufferCacheCreateBufferFromImage to access the Core Video backing memory directly from Metal rather than copying the frame through the CPU. One caveat: I would not assume every btp2 buffer uses the exact same packing. The MIPI-style layout is what I’ve validated on this specific iPhone 17 Pro Max Main-camera format. If useful I can post the unpack kernel / metadata extraction details.
6d
Reply to Working with kCVPixelFormatType_96VersatileBayerPacked12
I’ve been working on the same problem on an iPhone 17 Pro Max. There doesn’t appear to be a one-call Core Image debayer path for the kCVPixelFormatType_96VersatileBayerPacked12 buffers delivered by AVCaptureVideoDataOutput, but I was able to process them directly with Metal. On the Main camera’s 4224×3024 btp2 format I receive 6336 bytes/row, which is exactly 12 bits/sensel. The buffer on this device decodes correctly using a MIPI RAW12-style 2-pixels/3-bytes layout. The sample buffers also contain ProRes RAW attachments for Bayer pattern, black/white level, WB multipliers, gain and the Camera RGB→XYZ D65 color matrix. The pipeline I currently have working is: unpack 12-bit → normalize black/white → apply WB to Bayer sensels → debayer → Camera RGB→XYZ D65 matrix → gain → display transform I’m using CVMetalBufferCacheCreateBufferFromImage to access the Core Video backing memory directly from Metal rather than copying the frame through the CPU. One caveat: I would not assume every btp2 buffer uses the exact same packing. The MIPI-style layout is what I’ve validated on this specific iPhone 17 Pro Max Main-camera format. If useful I can post the unpack kernel / metadata extraction details.
Replies
Boosts
Views
Activity
6d