You can use simple planes if you want but you have to account for the fact that the anchor is not necessarily the center of the plane. You will still get a "bounding box" style representation and there may be some over/under lap but it should be much closer to what you were expecting. Something like the following should work for you on both vertical and horizontal planes. The relevant line is the setTransformationMatrix entity.setTransformMatrix( anchor.originFromAnchorTransform * anchor.geometry.extent.anchorFromExtentTransform, relativeTo: nil) if generateSimplifiedPlanes { let mesh: MeshResource = try! await MeshResource.generatePlane( width: anchor.geometry.extent.width, height: anchor.geometry.extent.height) modelEntity.model?.mesh = mesh modelEntity.collision?.shapes = [try! await ShapeResource.generateStaticMesh(from: mesh)] entity.setTransformMatrix( anchor.originFromAnchorTransform * anchor.geometry.extent.anchorFromExtentTransform, relativeTo: nil) } else { let shape: ShapeResource = try! await ShapeResource.generateStaticMesh( positions: anchor.geometry.meshVertices.asSIMD3(ofType: Float.self), faceIndices: anchor.geometry.meshFaces.asUInt16Array()) modelEntity.collision?.shapes = [shape] modelEntity.model?.mesh = planeAnchorToMeshResource(anchor) entity.transform = Transform(matrix: anchor.originFromAnchorTransform) } Notes: It appears that the extent is always defined on a vertical plane so if you apply anchorFromExtentTransform you always want to use a vertical plane. This is rough code from a while ago and I haven't checked to see if there are newer/easier ways to do this.

I have created FB19426676. It includes an example project that reproduces the issue on my device.

Do the entities that you want to drag have colliders and InputTargetComponents? If so you can use the entity value on the tap and drag gesture directly. TargetedToAnyEntity requires an input target component to trigger. The below lets you tap on a box to change the material and drag a box to a different location. I believe both Tap and Drag gestures have a hitTest method that you can use as well. import SwiftUI import RealityKit struct ContentView: View { @State var root: Entity = Entity() @GestureState private var isDragging: Bool = false var body: some View { ZStack(alignment: .bottom) { RealityView { content in content.add(root) for idx in 0...3 { let ball = ModelEntity(mesh: .generateBox(size: 0.1), materials: [SimpleMaterial(color: .blue, isMetallic: false)]) ball.generateCollisionShapes(recursive: true) ball.components.set(InputTargetComponent()) ball.position.y = 0.3 * Float(idx) root.addChild(ball) } } Button("Reset") { for entity in root.children { guard entity.components.has(InputTargetComponent.self) else { continue } entity .components[ModelComponent.self]?.materials = [ SimpleMaterial(color: .blue, isMetallic: false) ] } } } .gesture(TapGesture().targetedToAnyEntity().onEnded{ tap in tap.entity .components[ModelComponent.self]?.materials = [ SimpleMaterial(color: .green, isMetallic: false) ] }) .gesture(DragGesture(coordinateSpace: .global) .targetedToAnyEntity().updating($isDragging) { value, state, _ in if let location3d = value.unproject(value.location, from: .global, to: .scene) { value.entity.setPosition(location3d, relativeTo: root) } }) } } #Preview { ContentView() }

I have created FB20099118 and the attached project still reproduces the issue.