Bones/joints data issue - USD file export from Blender to RCP

Hello @Crixoule , thank you for your question!

You mention the animation is working fine in RCP, which sounds like it was imported succesfully! Were you able to use AnimationLibraryComponent to preview it in RCP? For your use case specifically, I recommend taking a look at our BOT-anist sample, spefically how the backpack and head are attached to the main body of the robot.

In RealityKit, the skeleton joint positions are not separate entities with transform components, as you might expect. This means you don't interact with them by placing other Entities as descendants of the skeleton joints. Instead, to get the position of a joint for the purpose of attaching another Entity to it, you calculate its transform by multiplying a chain of joint transforms together each frame.

There are a few steps to this, first you will need to get the indices for the chain of joints starting from the root joint that connect to your "head" or "backpack" or "arm_cannon" or whatever joint you're looking for. In BOT-anist, see the function getJointHierarchy in RobotCharacter.swift for a code implementation. Then, you will need to set the pin on the skeleton, which occurs on lines 92 and 93 in that same file, which will return an entity with a position you can use as an offset later. Finally, the code for setting the position each frame exists in JointPinSystem.swift in the function named pinEntity. This contains the relevant code for multiplying your chain of joint transforms together and setting the position of the pinned entity.

Let me know if that answers your question! I highly recommend sending us any feedback you have about animation APIs in RealityKit using Feedback Assistant. Thank you!

Yes, AnimationLibraryComponent works fine in RCP but I need to import the same file multiple times in RCP to get all the animations (exporting them from Blender).

I then need to manually add all the animations to the first file so I can build the library. Would be cool if the USD file could read the animation strips (NLA) and combining all animations in separate slots for RCP.

The BOT-anist is the best example so far for this issue, thanks for guiding me to the pin system. I'm trying quite a lot but no success so far. Hopefully, I will make it work.

In WWDC25 "What's new in RealityKit", the presenter says the following:

There is also a new entity attach method that allows you to attach one entity to the pin of another entity. This API greatly simplifies attaching meshes to the joints of an animated skeleton.

Hope this will be in VisionOS 26.

To be clear, unlike the BOT-anist, the mesh entity "Mesh01" needs to pick up another mesh entity (such as a ball) in real time and drop it in certain events, such as a collision. All while performing animations. The ball needs to attach to his hand.

Mesh01 is not the Vision Pro user. But the user sees it.

Hello @Crixoule !

Hope this will be in VisionOS 26.

Yes! In visionOS 26 we introduced attach(_ to:), which is the new, recommended way to attach an Entity to a GeometricPin on another Entity. So in your case, it sounds like you'd be calling attach on the entity you want to attach to Mesh01, and then pass in the Mesh01 entity as a parameter.

I then need to manually add all the animations to the first file so I can build the library. Would be cool if the USD file could read the animation strips (NLA) and combining all animations in separate slots for RCP.

I strongly recommend sending feedback about RealityKit's animation API including details of your use case using Feedback Assistant. I agree with you that the process of sequencing all your animations in Blender's NLA and then splitting them up in RCP with the AnimationLibraryComponent can be awkward, and your feedback and use case will help us improve this experience in the future.