I'm also facing same issue. Started using Xcode 12.4 on macOS Big Sur on a decently large project. The CPU chokes to 200% using about 3GB of real memory making my entire machine freeze during project compilation.