For anyone who is having the same issue, I fixed it by using ballEntity.move() instead and putting nothing as the rotation -
let newTranslation = SIMD3<Float>(0, 0, 1)
let moveController = ballEntity.move(to: Transform(scale: simd_make_float3(1, 1, 1), translation: newTranslation), relativeTo: ballEntity.self, duration: 0.2)
whereas previously (not working) I was doing this:
let newTranslation = SIMD3<Float>(0, 0, 1)
let moveController = ballEntity.move(to: Transform(scale: simd_make_float3(1, 1, 1), rotation: anchor.orientation, translation: newTranslation), relativeTo: ballEntity.self, duration: 0.2)
Removing the rotation: anchor.orientation helped as my anchor was a bit tilted or moving.