update macOS to 12.3.1 (21E258), and open xcode, xcode will prompt you to reinstall the xcode debug tool, after this, I feel it is a little faster than before.

I also have the same problem, M1, macOS 11.5, Xcode 12.5.1, not running rosetta