Hi! We run continuous 2D human body-pose estimation on a front-facing camera feed: one person, full body in frame, standing a few meters away, about 30 fps. With iPhone Duo open, the camera that faces the person looking at the inner display is the under-display inner camera (builtInInnerUltraWideCamera). Published specs cap it at 1080p at 30 or 60 fps, and it is presented as the FaceTime camera.
Two things are unclear from the camera tech talk and the specs:
Is that 1080p format a native readout we can feed to Vision body pose, or a preview-oriented stream whose effective detail is too low for full-body pose at that distance? Is third-party use of this camera supported for a long-running analysis session, or is it intended only for video calls and live previews?
Thanks
Hi Alexander. The inner display camera supports 1080p natively at both 30 and 60 fps. (It's not upscaled from a smaller format, if that's what you were asking). It supports stills, video recording, video data output, metadata output (such as face / body / QR code detection), just like the outer display camera.
It is suitable for use in long streaming sessions and is supported in the public API as such. Be aware that, because it is a behind display camera, the camera system runs additional per frame processing to clean up diffraction, flare, and other artifacts common in behind-screen cameras. This additional processing is thermally sustainable and real-time at 30 and 60 fps. Depending on how much additional processing you perform in your app, you should check for frame drops and scale your work accordingly.
Video quality should be sufficient for vision body pose. In general the images have a little more noise than the outer display camera.
Hope that helps.
