How to detect which entity was tapped?

Question

BlueLettuce16 OP

Created 1w

Replies 2

Boosts 0

Participants 2

Hi,

I'm rewriting my game from SceneKit to RealityKit, and I'm having trouble implementing the following scenario: I tap on the iPhone screen to select an Entity that I want to drag. If an Entity was tapped, it should then be possible to drag it left, right, etc.

SceneKit solution:

    func CGPointToSCNVector3(_ view: SCNView, depth: Float, point: CGPoint) -> SCNVector3 {
        let projectedOrigin = view.projectPoint(SCNVector3Make(0, 0, Float(depth)))
        let locationWithz   = SCNVector3Make(Float(point.x), Float(point.y), Float(projectedOrigin.z))
        return view.unprojectPoint(locationWithz)
    }

and then I was calling: SCNView().hitTest(location, options: [SCNHitTestOption.firstFoundOnly:true]) the code was called inside of the UIPanGestureRecognizer in my UIViewController.

Could I reuse that code or should I go with the SwiftUI approach - something like that:

var body: some View {
        
        RealityView {
        ....
}    .gesture(TapGesture().onEnded {
            
        })

?

I already have this code:

@State private var location: CGPoint?
                .onTapGesture { location in
                    self.location = location 
                }

I'm trying to identify the entity that was tapped within the RealityView like that:

RealityView { content in
let box: ModelEntity = createBox() // for now there is only one box, however there will be many boxes
content.add(box)
        let anchor = AnchorEntity(world: [0, 0, 0])
            content.add(anchor)
_ = content.subscribe(to: SceneEvents.Update.self) { event in
//TODO: find tapped entity, so that it could be dragged inside of the DragGesture()
}

Any help would be appreciated.

I also noticed that if I create a TapGesture like that:

TapGesture(count: 1)
            .targetedToAnyEntity()

and add it to my view using .gesture() then it is not triggered.

Boost

Answer 1

PlatformGoblin OP

1w

Do the entities that you want to drag have colliders and InputTargetComponents? If so you can use the entity value on the tap and drag gesture directly. TargetedToAnyEntity requires an input target component to trigger.

The below lets you tap on a box to change the material and drag a box to a different location.

I believe both Tap and Drag gestures have a hitTest method that you can use as well.

import SwiftUI
import RealityKit

struct ContentView: View {
    @State var root: Entity = Entity()
    @GestureState private var isDragging: Bool = false
    var body: some View {
        ZStack(alignment: .bottom) {
            RealityView { content in
                content.add(root)
                for idx in 0...3 {
                    let ball = ModelEntity(mesh: .generateBox(size: 0.1), materials: [SimpleMaterial(color: .blue, isMetallic: false)])
                    ball.generateCollisionShapes(recursive: true)
                    ball.components.set(InputTargetComponent())
                    ball.position.y = 0.3 * Float(idx)
                    root.addChild(ball)
                }
                
            }
            Button("Reset") {
                for entity in root.children {
                    guard entity.components.has(InputTargetComponent.self)
                    else { continue }
                    
                    entity
                        .components[ModelComponent.self]?.materials = [
                            SimpleMaterial(color: .blue, isMetallic: false)
                        ]
                }
            }
        }
        .gesture(TapGesture().targetedToAnyEntity().onEnded{ tap in
            tap.entity
                .components[ModelComponent.self]?.materials = [
                    SimpleMaterial(color: .green, isMetallic: false)
                ]
        })
        .gesture(DragGesture(coordinateSpace: .global)
            .targetedToAnyEntity().updating($isDragging) { value, state, _ in
                if let location3d = value.unproject(value.location, from: .global, to: .scene) {
                    value.entity.setPosition(location3d, relativeTo: root)
                }
        })
    }
}

#Preview {
    ContentView()
}

1

Answer 2

BlueLettuce16 OP

1w

Thanks for the answer. I forgot to mention one important detail. I discovered that targetedToAnyEntity() doesnt work in my case because I'm creating an instance of the PerspectiveCamera

  func createCamera() -> Entity {
        let camera = PerspectiveCamera()
        camera.name = "Camera"
        camera.look(at: [160, 80, 1], from: [160, 80, 200], relativeTo: nil)
        return camera
    }

and in this case RealityKit disables automatic input routing, because now I'm responsible for defining how the world is rendered and interacted with.

 RealityView { content in
           content.add(createCamera())

However I need that camera :( Automatic input routing also stops to work if entity is to far from the default camera :(

0