Post

Replies

Boosts

Views

Activity

Reply to CoreML inference on iOS HW uses only CPU on CoreMLTools imported Pytorch model
One more thing. The CoreML-compiled model downsampling torch.nn.Conv2d with stride (16,1) and kernel (16,1) has similar issues with ANE-processing. Upon iOS runtime initialization the model compilation does not give any console warnings or errors but everything just hangs without any remarks of anything. Changing 16x downsampling to 4x+4x solves that hang-issue.
Topic: Machine Learning & AI SubTopic: Core ML Tags:
Feb ’25
Reply to CoreML inference on iOS HW uses only CPU on CoreMLTools imported Pytorch model
This particular issue was caused by the upsampling path of the model having torch.nn.ConvTranspose2d operations with stride (16,1) and kernel (16,1) - for some reason ANE-compiler did not like them. Resolved by changing 16x to 4x+4x operation instead. Furthermore, ANE is 16-bit only. CoreMLTools may compile your 32-bit model without issues but loading the runtime model on iOS/macOS will not be assigned on ANE.
Topic: Machine Learning & AI SubTopic: Core ML Tags:
Feb ’25
Reply to CoreML inference on iOS HW uses only CPU on CoreMLTools imported Pytorch model
One more thing. The CoreML-compiled model downsampling torch.nn.Conv2d with stride (16,1) and kernel (16,1) has similar issues with ANE-processing. Upon iOS runtime initialization the model compilation does not give any console warnings or errors but everything just hangs without any remarks of anything. Changing 16x downsampling to 4x+4x solves that hang-issue.
Topic: Machine Learning & AI SubTopic: Core ML Tags:
Replies
Boosts
Views
Activity
Feb ’25
Reply to CoreML inference on iOS HW uses only CPU on CoreMLTools imported Pytorch model
This particular issue was caused by the upsampling path of the model having torch.nn.ConvTranspose2d operations with stride (16,1) and kernel (16,1) - for some reason ANE-compiler did not like them. Resolved by changing 16x to 4x+4x operation instead. Furthermore, ANE is 16-bit only. CoreMLTools may compile your 32-bit model without issues but loading the runtime model on iOS/macOS will not be assigned on ANE.
Topic: Machine Learning & AI SubTopic: Core ML Tags:
Replies
Boosts
Views
Activity
Feb ’25