I have the same problem in 11.4 beta 2. Started with 11.3. Not better with 11.3.1

Seems related to the not-responsive-for-touch-for-several-seconds-bug. Big issues on first rotation (at least on iPad-simulator) as well. Still a problem in first 11.5 beta

Xcode 13 Beta does not solve it for me. Still really slow (BigSur 11.5 Beta 2)

14.1 Beta 3 resolved it for me. Have been having trouble with all 14-versions until now