Post

Replies

Boosts

Views

Activity

How to sync ARFrame & ARFaceAnchor geometry ?
Hi, I'm using ARSCNView's delegate(ARSCNViewDelegate) to receive SCNNode(ARSCNFaceGeometry) updates. I've observed cases where the face geometry (blink detection) misses by 1 frame. I detect this case by visually comparing the image and the mesh in the method:   renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) I pull the ARFrame from (renderer as?ARSCNView).session.currentFrame My question is: Is it guarantied that the session.currentFrame in the renderer:didUpdate scope to be always in sync with the face geometry? I suspect a potential problem here since the geometry is being "pushed" while the current frame is being "pulled".
2
0
946
Jul ’22
How to properly use skinningJointMatrices, in SceneKit+Custom metal shader ?
I'm animating body+ face moments using skeleton animation , SceneKit and Metal custom shader. I need to deform some vertices in the vertex shader, therefore I need skinningJointMatrices in the vertex shader. However if I just add  the line   float4 skinningJointMatrices[183]; in the NodeBuffer : struct NodeBuffer {   float4x4 inverseModelTransform;   float4x4 inverseModelViewTransform;   float4x4 modelTransform;   float4x4 modelViewProjectionTransform;   float4x4 modelViewTransform;   float4x4 normalTransform;   float2x3 boundingBox; float4 skinningJointMatrices[765]; }; I get the following assertion: [SceneKit] Assertion 'C3DSkinnerGetEffectiveCalculationMode(skinner, C3DNodeGetGeometry(context->_nodeUniforms.instanceNode)) == kC3DSkinnerCalculationModeGPUVertexFunction' failed. skinningJointMatrices should only be used when skinning is done in the vertex function Is there a way to workaround this assert? The code seems to be working fine although the assertion.
0
0
1k
Jun ’22
How to perform scene updates after animations are evaluated?
I'm trying to overwrite some skeleton animation using : didApplyAnimationsAtTime time: TimeInterval) here is a link to the api. According to SCNSceneRendererDelegate : "SceneKit executes actions and performs animations attached to the scene graph. " before didApplyAnimationsAtTime is called. However, when I'm changing the animated node properties (for example position or eulerAngles). It seems like the animation continue to (somehow) change the nodes property, although it shouldn't. How can that be?
0
0
1.1k
Jan ’22
Does Face blendshapes available in visionOS ARSession?
Will I be able to open an ARSession with ARFaceTrackingConfiguration on visionOS? Will I be able to have the face blendshapes?
Replies
2
Boosts
2
Views
826
Activity
Aug ’23
How to sync ARFrame & ARFaceAnchor geometry ?
Hi, I'm using ARSCNView's delegate(ARSCNViewDelegate) to receive SCNNode(ARSCNFaceGeometry) updates. I've observed cases where the face geometry (blink detection) misses by 1 frame. I detect this case by visually comparing the image and the mesh in the method:   renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) I pull the ARFrame from (renderer as?ARSCNView).session.currentFrame My question is: Is it guarantied that the session.currentFrame in the renderer:didUpdate scope to be always in sync with the face geometry? I suspect a potential problem here since the geometry is being "pushed" while the current frame is being "pulled".
Replies
2
Boosts
0
Views
946
Activity
Jul ’22
How to properly use skinningJointMatrices, in SceneKit+Custom metal shader ?
I'm animating body+ face moments using skeleton animation , SceneKit and Metal custom shader. I need to deform some vertices in the vertex shader, therefore I need skinningJointMatrices in the vertex shader. However if I just add  the line   float4 skinningJointMatrices[183]; in the NodeBuffer : struct NodeBuffer {   float4x4 inverseModelTransform;   float4x4 inverseModelViewTransform;   float4x4 modelTransform;   float4x4 modelViewProjectionTransform;   float4x4 modelViewTransform;   float4x4 normalTransform;   float2x3 boundingBox; float4 skinningJointMatrices[765]; }; I get the following assertion: [SceneKit] Assertion 'C3DSkinnerGetEffectiveCalculationMode(skinner, C3DNodeGetGeometry(context->_nodeUniforms.instanceNode)) == kC3DSkinnerCalculationModeGPUVertexFunction' failed. skinningJointMatrices should only be used when skinning is done in the vertex function Is there a way to workaround this assert? The code seems to be working fine although the assertion.
Replies
0
Boosts
0
Views
1k
Activity
Jun ’22
How to perform scene updates after animations are evaluated?
I'm trying to overwrite some skeleton animation using : didApplyAnimationsAtTime time: TimeInterval) here is a link to the api. According to SCNSceneRendererDelegate : "SceneKit executes actions and performs animations attached to the scene graph. " before didApplyAnimationsAtTime is called. However, when I'm changing the animated node properties (for example position or eulerAngles). It seems like the animation continue to (somehow) change the nodes property, although it shouldn't. How can that be?
Replies
0
Boosts
0
Views
1.1k
Activity
Jan ’22
ARKit face capture session with un-mirrored frames?
Can we have an ARKit face capture session with un-mirrored frames, including un-mirrored face geometry and depth? Currently ARSession face capture delivered all the above data mirrored and there is no option to change it.
Replies
2
Boosts
0
Views
784
Activity
Dec ’21