Yes, I also experience this issue in the similar scenario. Never (or really rarely) happened on watchOS below 8. Come across as I'm looking for solution now.

The problem is connected with updating to Big Sur 11.3 The issue seems to be tied to having many libraries that are not part of the shared cache. That slows down dyld (it's shipped with the OS, not xcode) in unproportional way. I hope Apple will soon come up with the solution.

Same issue here. Deleting the contents of ~/Library/Developer/Xcode/iOS DeviceSupport (also logs and caches) didn't solve the problem for me. With this configuration: Big Sur 11.3 Xcode 14.5 Simulator 14.5 my app start takes more than 30 seconds when running from xcode. Normal start of installed app takes 2 seconds. Xcode says "launching app", "attaching to app", and then "running app", at the running app stage I get a ~30 second pause. debugserver at 100% activity at that time. Unticking "Debug executable" makes start instantaneous but makes debugging impossible, it's not a solution. On the real device (running iOS 14.4) start with attached debugger takes around 6 seconds.