[UPDATE] I finally figure this out and think this is a bug in RealityKit.
I will walk through my solution below.
Thinking process:
I found two properties in "BodyTrackedEntity": jointNames and jointTransforms. Therefore, to change jointTransform's position, what we need is to get the up-to-date joint's transforms from ARBodyAnchor and update the corresponding jointTransform in "BodyTrackedEntity". See below.
extension BodyTrackedEntity {
func updateJointsTransform(with anchor: ARBodyAnchor) {
for (idx, jointName) in self.jointNames.enumerated() {
guard let name = jointName.split(separator: "/").last else { print("Invalid name"); return }
guard let anchorTransform = anchor.skeleton.localTransform(for: ARSkeleton.JointName(rawValue: String(name))) else { print("can't find"); return}
self.jointTransforms[idx] = Transform(matrix: anchorTransform)
}
}
Based on the official documentation. We should use call jointNames to determine the order of the joints, and this is what I did in the solution above. However, as mentioned in previous post, the robot's joint's are all over the place.
/// - Remark:
/// Call jointNames to determine the name and order of the joints.
/// Note that active animations may override the joint transforms set using this function.*/
public var jointTransforms: [Transform] After several trial-and-errors, I foud that the bug is that the joint order in jointNames is WRONG. Therefore, the joints do not appear in the correct location when I use the ARBodyAnchor to update it's transform. The solution could be as simpe as follow.
extension BodyTrackedEntity {
func updateJointsTransform(with anchor: ARBodyAnchor) {
self.jointTransforms = anchor.skeleton.jointLocalTransforms.map { Transform(matrix:$0) }
}
}
Note: To use this function, you have to set BodyTrackedEntity.bodyTracking.isPaused = true to stop the auto updates.
Topic:
Spatial Computing
SubTopic:
ARKit
Tags: