Post

Replies

Boosts

Views

Activity

Reply to Is there an API to check if a Core ML compiled model is already cached?
No, they do not have such an API as of iOS 18. A model cache is reasonably sticky as long as the absolute path of the model (.mlmodelc) stays same. However, on app update, the system often gives a new app sandbox directory, which changes the path to the model. (I wish Apple could make the cache logic less sensitive to such a change.) If it is an option, I would run the background pre-load only after the app update because it seems like a major reason of the cache invalidation.
Topic: Machine Learning & AI SubTopic: Core ML Tags:
May ’25
Reply to Regression in EnumeratedShaped support in recent MacOS release
Maybe does your model uses enumeratedShapes message of just one entry for the "static shape", instead of the normal static shape? According to their document (https://apple.github.io/coremltools/docs-guides/source/flexible-inputs.html), For a multi-input model, only one of the inputs can be marked with EnumeratedShapes; the rest must have fixed single shapes. If you require multiple inputs to be flexible, set the range for each dimension.
Topic: Machine Learning & AI SubTopic: Core ML Tags:
May ’25
Reply to A specific mlmodelc model runs on iPhone 15, but not on iPhone 16
I think it's worth filing a feedback assistance to Apple with model attached. Also, you may want to tweak .specializationStrategy setting (https://developer.apple.com/documentation/coreml/mloptimizationhints-swift.struct/specializationstrategy-swift.property) because it affects how Apple Neural Engine specializes your model to the hardware specific executable code.
Topic: Machine Learning & AI SubTopic: Core ML Tags:
May ’25
Reply to KV-Cache MLState Not Updating During Prefill Stage in Core ML LLM Inference
Core ML framework is agnostic to LLM's specific setup such as pre-filling step vs single token generation steps. It just executes whatever the model (ML Program) says. Given that the state (KV-cache) is updated in single token generation steps, I would suspect that ML Program for the prompt step has a bug where it does not update KV-cache. Another thing worth verifying is that you need to use the same MLState instance in both pre-filling and token generations. As for your questions, Not I know of. No. Each prediction call fully completes the state update. After pre-filling prediction call, you can examine the contents of the state by withMultiArray(for:), but I suppose you already did that.
Topic: Machine Learning & AI SubTopic: Core ML Tags:
May ’25