I also have the same problem, M1, macOS 11.5, Xcode 12.5.1, not running rosetta

update macOS to 12.3.1 (21E258), and open xcode, xcode will prompt you to reinstall the xcode debug tool, after this, I feel it is a little faster than before.