I thought it would reproducible for everyone, as it was for me across frameworks and devices. Here are the two recordings - with and without ScrollView. I turned on touch indication, so you can see that the animation on Button happens with a significant delay. When I was writing that most users won't see it, I was referring to the animation, not the delay.

I've created a report in Feedback Assistant: FB17003318 (iOS 18.1 SwiftUI popover crash)