I do a lot of test (related to very slow debugging for application using a lots of dylib). This problem start with Mac OS Big Sur 11.3 up to first beta of 12.5. So you can install 11.2.1 and Xcode 12.5.