How to simulate location in the visionOS simulator?

I'm experimenting with MapKit on visionOS and I would like to try out different locations. However, I cannot find a way to simulate them. Neither setting a location in Xcode nor setting it in the Simulator would work.

I tap on the "MapUserLocationButton", I get an error message:

CLLocationManager(<CLLocationManager: 0x600000008a10>) for <MKCoreLocationProvider: 0x60000302a400> did fail with error: Error Domain=kCLErrorDomain Code=1 "(null)"

Also, if I try to add the MapCompass and the MapScaleView to .mapControls, this does not have an effect. Which is a pity, since map scaling does not work very well using a mouse in the simulator. How can I get these controls to work?

Last but not least, the MapUserLocationButton shows up in the very upper right and is cut off a bit, so I would love to pad it. But .padding does not have an effect either.

The MapCompass and MapScaleView work, if I add them manually as an .overlay, but not inside .mapControls. I also noticed that MapPitchSlider and MapZoomStepper are not supported on visionOS. Which is sad, since manual zoom and pitch do not really work well.