I tested and noticed only space reduction on file, no latency optimization on inferencing on any compute unit. I've asked this same question then in the coremltools GitHub project - https://github.com/apple/coremltools/issues/1736

Same issue here on same macOS/Xcode versions. FB12024915