I am running a CoreML model (vision transformer, fixed input size) with hardware units cpuAndNeuralEngine on both iPhone 15 and iPhone SE 2. The memory usage is normal on iPhone 15. On iPhone SE 2 [A13], it is more than double. The model loading on SE2 takes forever, and puts a few of these warnings in the console:
[Espresso::handle_ex_plan] exception=ANECF error: failed to load ANE model
[...]
FAILED: err=(
CompilationFailure
)
doUnloadModel:options:qos:error:: nil _ANEModel
ANECF error:
Unable to build plan. @ PrepareOpForEncode
E5RT: Unable to build plan. (2)
Before seemingly falling back on CPU because inference completes successfully. But it looks as though the failed attempt at Espresso compilation leaves behind a ~800MB memory leak.
What's interesting is, if I set CoreML to use cpuOnly and not cpuAndNeuralEngine, the memory usage is lower by 800MB to around ~100MB, less than what I would expect.
Topic:
Machine Learning & AI
SubTopic:
Core ML
Tags: