UPDATE: Famous reverse engineer Geohot exposed some examples of using AppleH11ANEInterface Also you can look at Wish Wu presentation on BlackHat Asia 2021 about ANE internals. Sorry, Apple not allows to paste it's link or attach file here - use google. Based on these two sources, you can conclude that only thing ANE can do well - convolutions. Perhaps it was intentionally designed as convolution accelerator. Other AI/ML things, even so easy as ReLU is 20 times slower than CPU.

I feel you have a bit wrong idea what Neural Engine is. It have nothing common to CPU or GPU - it's just a machine learning accelerator with very limited area of application - even not every layer type in your model can be used. Look at This FAQ for better understanding what it is and what it can. Q: Is there any low-level API to create my very own work-loads? A: Yes and No. Low level AppleNeuralEngine.framework is private to Apple and you can't use it. But: take a look at ANE Tools - compiler and decompiler for Neural Engine. Also there is coremltools - this will help to interface with TensorFlow and PyTorch Q: Can I use the Neural Engine to offload the CPU? I am especially interested in parallelism using threads. A: Basically - No. ANE can't execute CPU/GPU code and don't have threads. It operates with layer connectivity map and net weights. One more thing - vDSP and veclib DONT use ANE.

TL/DR: No. Long answer: Intel macs are not compatible with iphone/ipad apps, you must deploy separate build for x86-64 target.