I isolated the issue to be if a child has a some rotation already applied by default then the problem happends. I just removed the rotation to the child colliders and it worked fine but it does mean it it's going to be a hassle making compound colliders.
Topic:
Graphics & Games
SubTopic:
RealityKit
Tags: