Hello,
Does anyone have a recipe on how to raycast VNFaceLandmarkRegion2D points obtained from a frame's capturedImage?
More specifically, how to construct the "from" parameter of the frame's raycastQuery from a VNFaceLandmarkRegion2D point?
Do the points need to be flipped vertically? Is there any other transformation that needs to be performed on the points prior to passing them to raycastQuery?
Does it make sense to raycast face landmarks in the first place? Assuming face is stationary long enough?
Hello @jimbang,
More specifically, how to construct the "from" parameter of the frame's raycastQuery from a VNFaceLandmarkRegion2D point?
I assume you are referring to ARFrame's raycastQuery method?
The point parameter is documented in ARFrame.h as:
/*
@param point A point in the image-space coordinate system of the captured image.
Values should range from (0,0) - upper left corner to (1,1) - lower right corner.
*/
Do the points need to be flipped vertically? Is there any other transformation that needs to be performed on the points prior to passing them to raycastQuery?
Vision always returns its observations in normalized image-space. Whether a flip is necessary here or not depends on whether the orientation you processed the image with in Vision matches the orientation of the frame.
Does it make sense to raycast face landmarks in the first place? Assuming face is stationary long enough?
Not really, ray-casting is for detecting intersections with real-world surfaces. I think if you described the feature your trying to build at a higher level I could give a recommendation :)
--Greg
Hello Greg,
Yes, I’m referring to ARFrame's raycastQuery.
Essentially, I’d like to measure human facial features using the rear camera with LiDAR, and take advantage of any temporal refinement available in either ARKit or AVFoundation.
Is depth data refined over time during an ARKit session?
I’d really appreciate any recommendations—especially from a DTS engineer.