So where should we set -mf16c in the compile settings? The vector instruction setting only takes SSE4.2, AVX, AVX2. This corresponds with an x86 named compile setting (really x64) and translates to say "-march avx2". For some bizarre reason, Xcode leaves f16c disabled if AVX or AVX2 are set. This is a major slowdown for macOS Intel apps. This despite AVX/AVX2 being tied into the f16c instructions.
So then Apple Silicon nicely ignores the -march setting. But if you set -mf16c anywhere in the other compile flags, then the universal build complains that is an unused argument.