I've been having the same issue. I updated Xcode to 12.4 and it's been going extremely slow ever since! I'm on a 2020 M1 8GB.

Same issue for me. Updated Xcode to 12.4 and has been running slow since! Any fix for this?