crazy that its so bad with Apple's own models too. Seems like they're focusing mostly on coreAI and throwing coreML under the bus.

Thank you for looking into this, will test, please continue as well so we can at least get to parity at some point.

Yeah, really hoping for a fix from Apple.

No, its a confirmed issue on A20 and M6 devices only. How would you explain having better performance on older devices lol.

Really hope apple fixes this then. Yeah I'm also using the GPU now, which is not really a workaround IMO, and as you said, not ideal. Thanks for your reply too, I think the more coverage this gets the more likely it is to be fixed.

AFAIK the Apple Neural Engine does not natively run fp32 or int32, so I wouldn't call this a solution. At that point you can say "just run on GPU" which doesn't really have a compilation step.

AFAIK, the neural engine does not natively run fp32 or int32, so I wouldnt say that this is a solution.