I got it working and hopefully this will help someone else. And maybe someone who understands more about this can fill in some details as to why this is happening.
In this particular case, image classification was occurring from a live camera view whenever a photo was taken. It was happening on its own task.
The rate limit appears to have occurred in ModelManagerServices (subsystem com.apple.modelmanager, category IPC) and then passed up.
It would happen consistently in this portion of the app after processing several photos (I had originally thought it was around 20, but it was usually closer to 12-18, depending on the device).
After you hit the limit in this part of the app, you would continue to get the rate limit error from then on every time it tried to analyze the image, even if you went away to another section of then came back to it later or just waited a while. The only way to get it working again was to kill the app and re-launch.
However, there are other parts of the app that use the same code for analyzing images and none of them exhibited this problem. In fact, you could hit the rate limit in the camera view, go to another section and processing a bunch of images from files, and the app would truck along fine, calling the same image classification code. Go back to the camera view and you'd get the rate limit immediately.
I tried everything I could think of, including enforcing long pauses between photos, tearing down the AVCapture session after each was taken and then standing it back up again after the photo was analyzed, scaling the images even smaller, lowering the priority of the task, and a host of other things. Nothing worked.
Then finally, I tried presenting the UI that shows during the analysis (and thus calls the analysis code) as its own modal sheet (the camera view itself being one) and it worked.
I don't know why this works, but I'm not getting the rate limit anymore.
Topic:
Machine Learning & AI
SubTopic:
Foundation Models