Post

Replies

Boosts

Views

Activity

Reply to Model Rate Limits?
In this case, it was a foregrounded visionOS app. This app is a feed reader / RSS app and this happened when I downloaded new feed items (~150 of them) and ran the model in a loop over each entry. The actual model request is to summarize the content of the feed item. FB17984127 The error is: Error Domain=com.apple.SensitiveContentAnalysisML Code=15 "Failed model manager query for model com.apple.fm.language.instruct_300m.safety: Client rate limit exceeded, try again later"
Jun ’25
Reply to "FoundationModels GenerationError error 2" on iOS 26 beta 3
I filed a second one with my own content, FB18787534. Just to share a public example, this is an example of text the model won't summarize saying it's "unsafe". Seems pretty anodyne to me... let alone trying to summarize news about current events, obits, etc. Here’s a product recommendation long in the making. Four years ago this month, Matthew Panzarino was my guest on The Talk Show and at one point he recommended Moft’s Snap-On iPhone Stand/Wallet. It uses a very clever origami-style folding design. Folded flat it kind of just looks like a leather MagSafe wallet. But folded open it works as a stand — and as a stand, it works both horizontally and vertically. Borrowing images from Moft’s website:You can also use it with the stand oriented vertically but the phone horizontally.I bought one of these right after that episode of the show, and I’ve been using it ever since. And every so often when I use it, I think to myself that I should write a post recommending it. I’ve waited so long that Panzarino has been back on The Talk Show five times since the episode in which he recommended it, but here we are. The thing is, I use it both in my kitchen and while travelling, and so I’ll often find myself in the kitchen, rooting around the drawer in which I keep it stashed, only to realize it’s downstairs in my office in my laptop bag. Or, worse, I’ll find myself looking for it in my laptop bag while I’m sitting on an airplane 35,000 feet in the air, only to realize it’s back home in my kitchen. So I ordered a second one today — which I should have done like 3.5 years ago.I own a few similar/competing products, like these PopSocket-y rings from Anker and Belkin. I have no idea why I own both of those rings when I don’t like either of them as much as the Moft foldable stand. The problem with these rings is that they’re only able to prop the phone horizontally. Watching video is almost certainly the most common use case for these stands, but I do often use my iPhone propped up vertically, like for FaceTime calls and when I’m writing on it using a Bluetooth keyboard. I’m going to give both of these rings away — there’s nothing they do better than the Moft stand. The Moft stand even works better as a hand-holding grip.I’ve never used the Moft stand as a wallet, but if you want to, it holds two cards. Prime “Day” lasts a week and it’s still running until midnight Pacific tonight, but the Moft stand doesn’t have a Prime Day discount: it’s the same price at Amazon as it is from Moft’s website: $30. Well worth it. I love this thing. (Buy yours wherever you want, of course, but the Amazon link a few sentences back will throw some filthy affiliate lucre my way.) guardrailViolation(FoundationModels.LanguageModelSession.GenerationError.Context(debugDescription: "May contain sensitive or unsafe content", underlyingErrors: [FoundationModels.LanguageModelSession.GenerationError.guardrailViolation(FoundationModels.LanguageModelSession.GenerationError.Context(debugDescription: "May contain unsafe content", underlyingErrors: []))]))
Jul ’25
Reply to Using Past Versions of Foundation Models As They Progress
Hi J, Thanks for the reply. A little more info: In my specific case, I am indeed using guided generation already (which helped a lot in seed 1 and 2). I thought the eval session was great. I've been working with LLMs for a bit now and it's awesome to have a whole WWDC session to talk about evals and expose them to devs who may not have seen them before. This is coming up for me now because something changed in seed 3 where my feature went from a ~95% success rate to a 0% success rate, all failing with guardrails errors that did not trigger in the first two seeds. There are a bunch of other threads here on the topic and I've filed several feedbacks on the specifics already. Maybe that's a bug/unexpected outcome and we'll see a future seed restore the behavior. I hope so, I'd like to ship this feature. But if not, at least I won't have sent it to customers. My real concern is that in a future point release where the amount of feedback time is compressed and IMO it's very hard to get specific issues in front of engineers via the Feedback system with enough time to get a fix done and tested, this will pop up again. Since the models are non-deterministic, unless you're running my evals, there's a good chance the team may not even know about what's for me a serious regression until it ships. I really want to use and love the framework because it's so promising but I'm a little freaked out now given what's happening in the current seeds. That's the background for my concern and I don't think there's any way to mitigate that at the moment (right?)
Jul ’25
Reply to Using Past Versions of Foundation Models As They Progress
Sure, here are a couple, including one that is reproduced with WWDC sample code from Apple. FB18787534 FB18712543 Also, in case you have not seen this thread, there is additional discussion: https://developer.apple.com/forums/thread/792022 Question - do you know if adapter training could help with guardrails issues? I was assuming that the guardrails either ran in a separate process/mechanism or were deeply embedded enough in the training process that it wouldn't matter but if I'm wrong about that and training a custom adapter could help, that's something I would definitely consider (I have a lot of good data, perfect for this sort of training).
Jul ’25
Reply to Model Rate Limits?
In this case, it was a foregrounded visionOS app. This app is a feed reader / RSS app and this happened when I downloaded new feed items (~150 of them) and ran the model in a loop over each entry. The actual model request is to summarize the content of the feed item. FB17984127 The error is: Error Domain=com.apple.SensitiveContentAnalysisML Code=15 "Failed model manager query for model com.apple.fm.language.instruct_300m.safety: Client rate limit exceeded, try again later"
Replies
Boosts
Views
Activity
Jun ’25
Reply to Model Guardrails Too Restrictive?
I updated the report with more examples. It's really, really sensitive. Any news article about someone dying for instance, rejected.
Replies
Boosts
Views
Activity
Jun ’25
Reply to defaultIsolation option and Core Data
I think what you did - adding nonisolated - is correct. Also needed for SwiftData and at-ModelActor.
Replies
Boosts
Views
Activity
Jun ’25
Reply to Foundation Model Always modelNotReady
It's on Apple's end. You gotta wait for them to fix it.
Replies
Boosts
Views
Activity
Jun ’25
Reply to "FoundationModels GenerationError error 2" on iOS 26 beta 3
Since installing beta 3, literally every single request I send to Foundation Models generates a guardrails error where before it was about 10%. Since we don't have Xcode yet, I haven't re-tested my evals but it seems like something really big changed in this seed. I would call it a big regression to the point of no longer being unusable.
Replies
Boosts
Views
Activity
Jul ’25
Reply to Foundation Models performance reality check - anyone else finding it slow?
I've found that the prompts I would use for an API based current SOTA LLM really don't work that well with this little guy. Probably to be expected but after a lot of fine-tuning I've gotten some acceptable results. Also, this is the worst they are ever gonna be. :-)
Replies
Boosts
Views
Activity
Jul ’25
Reply to "FoundationModels GenerationError error 2" on iOS 26 beta 3
Yeah. Guardrails are part of the system but the change in this beta is so severe, they trigger almost all the time. I suggest we all file Feedbacks and quickly. We're already mid-July... I'll post my FB number once I have it all written up.
Replies
Boosts
Views
Activity
Jul ’25
Reply to "FoundationModels GenerationError error 2" on iOS 26 beta 3
Filed FB18712543
Replies
Boosts
Views
Activity
Jul ’25
Reply to "FoundationModels GenerationError error 2" on iOS 26 beta 3
I filed a second one with my own content, FB18787534. Just to share a public example, this is an example of text the model won't summarize saying it's "unsafe". Seems pretty anodyne to me... let alone trying to summarize news about current events, obits, etc. Here’s a product recommendation long in the making. Four years ago this month, Matthew Panzarino was my guest on The Talk Show and at one point he recommended Moft’s Snap-On iPhone Stand/Wallet. It uses a very clever origami-style folding design. Folded flat it kind of just looks like a leather MagSafe wallet. But folded open it works as a stand — and as a stand, it works both horizontally and vertically. Borrowing images from Moft’s website:You can also use it with the stand oriented vertically but the phone horizontally.I bought one of these right after that episode of the show, and I’ve been using it ever since. And every so often when I use it, I think to myself that I should write a post recommending it. I’ve waited so long that Panzarino has been back on The Talk Show five times since the episode in which he recommended it, but here we are. The thing is, I use it both in my kitchen and while travelling, and so I’ll often find myself in the kitchen, rooting around the drawer in which I keep it stashed, only to realize it’s downstairs in my office in my laptop bag. Or, worse, I’ll find myself looking for it in my laptop bag while I’m sitting on an airplane 35,000 feet in the air, only to realize it’s back home in my kitchen. So I ordered a second one today — which I should have done like 3.5 years ago.I own a few similar/competing products, like these PopSocket-y rings from Anker and Belkin. I have no idea why I own both of those rings when I don’t like either of them as much as the Moft foldable stand. The problem with these rings is that they’re only able to prop the phone horizontally. Watching video is almost certainly the most common use case for these stands, but I do often use my iPhone propped up vertically, like for FaceTime calls and when I’m writing on it using a Bluetooth keyboard. I’m going to give both of these rings away — there’s nothing they do better than the Moft stand. The Moft stand even works better as a hand-holding grip.I’ve never used the Moft stand as a wallet, but if you want to, it holds two cards. Prime “Day” lasts a week and it’s still running until midnight Pacific tonight, but the Moft stand doesn’t have a Prime Day discount: it’s the same price at Amazon as it is from Moft’s website: $30. Well worth it. I love this thing. (Buy yours wherever you want, of course, but the Amazon link a few sentences back will throw some filthy affiliate lucre my way.) guardrailViolation(FoundationModels.LanguageModelSession.GenerationError.Context(debugDescription: "May contain sensitive or unsafe content", underlyingErrors: [FoundationModels.LanguageModelSession.GenerationError.guardrailViolation(FoundationModels.LanguageModelSession.GenerationError.Context(debugDescription: "May contain unsafe content", underlyingErrors: []))]))
Replies
Boosts
Views
Activity
Jul ’25
Reply to Foundation Model - Change LLM
Not in this release... fingers crossed for some time in the future.
Replies
Boosts
Views
Activity
Jul ’25
Reply to FoundationModels guardrailViolation on Beta 3
Yeah, it seems much more restrictive in b3. We've got another thread going with lots of examples also. File feedback! https://developer.apple.com/forums/thread/792022?page=2
Replies
Boosts
Views
Activity
Jul ’25
Reply to Using Past Versions of Foundation Models As They Progress
Having no reasonable assurance that a feature I build against these might not stop working outside of my control does freak me out quite a bit, especially since we're already seeing massive changes in what's allowed in the current seeds. FB18924722
Replies
Boosts
Views
Activity
Jul ’25
Reply to Using Past Versions of Foundation Models As They Progress
Hi J, Thanks for the reply. A little more info: In my specific case, I am indeed using guided generation already (which helped a lot in seed 1 and 2). I thought the eval session was great. I've been working with LLMs for a bit now and it's awesome to have a whole WWDC session to talk about evals and expose them to devs who may not have seen them before. This is coming up for me now because something changed in seed 3 where my feature went from a ~95% success rate to a 0% success rate, all failing with guardrails errors that did not trigger in the first two seeds. There are a bunch of other threads here on the topic and I've filed several feedbacks on the specifics already. Maybe that's a bug/unexpected outcome and we'll see a future seed restore the behavior. I hope so, I'd like to ship this feature. But if not, at least I won't have sent it to customers. My real concern is that in a future point release where the amount of feedback time is compressed and IMO it's very hard to get specific issues in front of engineers via the Feedback system with enough time to get a fix done and tested, this will pop up again. Since the models are non-deterministic, unless you're running my evals, there's a good chance the team may not even know about what's for me a serious regression until it ships. I really want to use and love the framework because it's so promising but I'm a little freaked out now given what's happening in the current seeds. That's the background for my concern and I don't think there's any way to mitigate that at the moment (right?)
Replies
Boosts
Views
Activity
Jul ’25
Reply to Using Past Versions of Foundation Models As They Progress
Sure, here are a couple, including one that is reproduced with WWDC sample code from Apple. FB18787534 FB18712543 Also, in case you have not seen this thread, there is additional discussion: https://developer.apple.com/forums/thread/792022 Question - do you know if adapter training could help with guardrails issues? I was assuming that the guardrails either ran in a separate process/mechanism or were deeply embedded enough in the training process that it wouldn't matter but if I'm wrong about that and training a custom adapter could help, that's something I would definitely consider (I have a lot of good data, perfect for this sort of training).
Replies
Boosts
Views
Activity
Jul ’25
Reply to "FoundationModels GenerationError error 2" on iOS 26 beta 3
Our long national nightmare may be over - testing with seed 4, the guardrails seem back to being sane. It made it into the release notes as a bug which I take as a positive sign that it wasn't a policy thing. Phew.
Replies
Boosts
Views
Activity
Jul ’25