I figured out that I needed to change a build setting called "Build Active Architectures Only" to "Yes". Then it stopped trying the ARM64 compilation and only did Intel.