How do I control a SwiftUI TextField with a game controller?

Question

dfabulich OP

Created 1h

Replies 1

Boosts 1

Participants 2

This post is from the WWDC26 Games Technologies Q&A.

I've coded a text-adventure game in SwiftUI. (My game has no graphics or sound effects.)

My app already supports keyboard navigation; I would like to add support for game controllers on iPhone. I can't figure out how to do it. I especially can't see any way to allow controller users to enter text in a TextField.

I've read https://developer.apple.com/documentation/gamecontroller/supporting-game-controllers and it's all about button events. There's no reference to SwiftUI at all in that documentation, or any input-method editing at all. The only mention of "keyboard" is about treating the keyboard itself as if it were a game controller providing button events.

How do I implement this?

Answer 1

Frameworks Engineer OP

Apple

1h

Hi dfabulich

The Game Controller framework enables your application to receive inputs from connected physical game controllers, as well as from touch controllers built with the Touch Controller framework. The inputs the Game Controller framework delivers to your application reflect the button presses, thumbstick movements, etc of the physical game controller (or touch controller). They are not 'events' (in the AppKit or UIKit sense), and do not flow through the responder chain where views or gesture recognizers can act on them.

Now to your specific questions:

I would like to add support for game controllers on iPhone. I can't figure out how to do it.

I'm afraid there is no out-of-the-box functionality for this today on macOS, iOS, or visionOS. Your application can use the Game Controller framework to receive inputs from connected game controllers. But you will need to implement the logic that interprets those inputs into updates to your user interface - sometimes called a focus engine. A minimal focus engine has:

State to track which element of your user interface is currently "focused" and reflect that state in your UI. For example, by adding a highlight or changing the color of the focused element.
A system or set of rules to determine, given a currently focused element and a movement direction (up/down/left/right), which element focus shifts to.
Logic that reacts to game controller inputs and either triggers an action on the currently focused element (e.g, in response to 'A' press) or moves focus to a new element using the above system (e.g, in response to dpad press or thumbstick movement).

You can see a working example in the Apple Games application, which supports game controller navigation on macOS and iOS.

Of course, please submit an enhancement request if you want to see this functionality become part of the common SwiftUI toolkit.

I especially can't see any way to allow controller users to enter text in a TextField.

Right. iOS and macOS do not include built-in support for text entry using a game controller today. This is also something that your application can build - an "on screen" keyboard that can be operated by game controller input.

I suggest submitting a (separate) enhancement request to add game controller support for text entry to macOS and iOS.