Post

Replies

Boosts

Views

Activity

Reply to RealityKit: Update joints' tranform in BodyTrackedEntity
[UPDATE] I finally figure this out and think this is a bug in RealityKit. I will walk through my solution below. Thinking process: I found two properties in "BodyTrackedEntity": jointNames and jointTransforms. Therefore, to change jointTransform's position, what we need is to get the up-to-date joint's transforms from ARBodyAnchor and update the corresponding jointTransform in "BodyTrackedEntity". See below. extension BodyTrackedEntity {   func updateJointsTransform(with anchor: ARBodyAnchor) {     for (idx, jointName) in self.jointNames.enumerated() {       guard let name = jointName.split(separator: "/").last else { print("Invalid name"); return }       guard let anchorTransform = anchor.skeleton.localTransform(for: ARSkeleton.JointName(rawValue: String(name))) else { print("can't find"); return}       self.jointTransforms[idx] = Transform(matrix: anchorTransform)     } } Based on the official documentation. We should use call jointNames to determine the order of the joints, and this is what I did in the solution above. However, as mentioned in previous post, the robot's joint's are all over the place.   /// - Remark:   ///   Call jointNames to determine the name and order of the joints.   /// Note that active animations may override the joint transforms set using this function.*/   public var jointTransforms: [Transform] After several trial-and-errors, I foud that the bug is that the joint order in jointNames is WRONG. Therefore, the joints do not appear in the correct location when I use the ARBodyAnchor to update it's transform. The solution could be as simpe as follow. extension BodyTrackedEntity { func updateJointsTransform(with anchor: ARBodyAnchor) {   self.jointTransforms = anchor.skeleton.jointLocalTransforms.map { Transform(matrix:$0) } } } Note: To use this function, you have to set BodyTrackedEntity.bodyTracking.isPaused = true to stop the auto updates.
Topic: Spatial Computing SubTopic: ARKit Tags:
Feb ’21