So it depends. Currently the Foundation Models framework only supports text input, so it won't work to pass in an image directly. However a great use case for combining the Apple Vision framework and Foundation Models is OCR (optical character recognition, basically detecting text). You can use the Vision framework to detect text from an image or live camera stream, and then pass that text to Foundation Models to generated responses about the text. This should work well for images of things like books, flyers, or notes. Here's a sample project from Vision that shows the OCR part: https://developer.apple.com/documentation/vision/locating-and-displaying-recognized-text

Hi all! If anyone encounters this bug again, can you please file a Feedback Assistant report to Foundation Models and report back here with your feedback #? 🐛This seems to be a hard bug to replicate, so we'd appreciate a feedback report to help us diagnose the underlying issue if it happens to you. In the rare (and definitely a bug) case that something has gone wrong at the system level, restarting your Mac should resolve the issue.

This definitely sounds like a bug. Can you please file a Feedback Assistant bug report to Foundation Models Framework so we can take a look at your Mac's and Xcode's system state? In the meantime, here are a few tactics that might help "reset" whatever's gone wrong: Close Xcode. Restart your Mac. Open Xcode. If that doesn't work: Go to Settings -> Apple Intelligence & Siri -> turn Apple Intelligence off. Restart your Mac. Then turn Apple Intelligence back on.

Hi all! We're actively working to improve the guardrails and reduce false positives. Apologies for all the headaches. Workaround The #1 workaround I can offer you is for summarizing content like news articles, you can use .permissiveContentTransform to turn the guardrails off. Please checkout this article I wrote on Improving the safety of generative model output because there are some caveats on when it's appropriate to turn the guardrails off, and cases where the model may refuse to answer anyway. Please send us your false-refusal prompts If you feel comfortable doing so, send us any prompts that falsely trigger the guardrails. While we're actively working to improve guardrail false-refusals, it's incredibly helpful to see prompts from real developers (like you) to identify blind-spots we might have in our guardrail evaluations. Check out this post on sending us feedback. It shows a handy way to send feedback from Xcode. Make sure to send your Feedback Assistant bug reports to Foundation Models framework so we get your issue. Include your prompt in your report. If you feel comfortable, please also include your language/locale Siri setting, e.g. "Spanish (Mexico)" in your report, since the guardrails are influenced by locale. Thanks!

Thanks for this post! This is an issue we care a lot about... so while I don't have an ideal solution for you, I can fill in some details as well as what workarounds we have so far: 1. Model availability The core issue is that Foundation Models framework is calling into the on-device foundation model from Apple Intelligence. The model is managed by the operating system, and is only available if the user has chosen to turn on Apple Intelligence —on a compatible device. That's our key blocker from the framework standpoint. If the model is "unavailable" it isn't even downloaded or present on the device. 2. Language expansion progress I 100% hear you on language expansion speed, but we are making progress! Just a few days ago we added Apple Intelligence support for 8 new languages: Danish, Dutch, Norwegian, Portuguese (Portugal), Swedish, Turkish, Chinese (Traditional), and Vietnamese. 3. User workaround: set Siri language Not that this helps tremendously, but Apple Intelligence is actually based on Siri language settings, not the system language setting. So for Catalan (or other unsupported language) users, go in Settings app > Apple Intelligence & Siri > Language, and set Language to one of the supported languages from this article. Obviously this only helps users who are fluent in multiple languages. 4. Framework workarounds Foundation Models framework can't work if there's no model on-device, but once model availability is unlocked, we actually do have some language workarounds baked-in to support you. For example, calling supportslocale(_:) will check against the user's language settings, and this takes in account if the user has set your app setting to a different language. This function will return true if we can support a close language. So if the user has set your app to Catalan, supportslocale(_:) will return true since we can support Spanish, which is fairly close. More details in this article: Support languages and locales with Foundation Models This an area we're actively working to improve, so please keep the feedback reports coming with ideas or suggestions. Thanks!

The token limit on the SystemLanguageModel is currently 4096 tokens. This is always the fixed token limit, there's no possibility of it changing. See this tech note for more discussion of the context window: TN3193: Managing the on-device foundation model’s context window | Apple Developer Documentation The tokenizer will always produce the same amount of tokens given the same input. So you shouldn't see any variation. ... One source of confusion might be, currently the error message for GenerationError.exceededContextWindowSize will print out a token size as soon as the token length of your content trips over the 4096 limit. So sometimes it might print the token size of your content is 4090 or maybe 4100... that number is when the error was triggered, the actual limit is still always 4096.

Great question! Turns out this is a current limitation of contentTagger. Currently if you only use Instructions without a @Generable, the model will default to producing topic tags —regardless of what you put in your instructions. To produce emotion, action, or object tags, you'll need to use the @Generable structure from the documentation.

Good news! The answer is a clear and simple no. Foundation Models only uses the on-device ~3 billion parameter foundation model. That's 100% on-device. No behind-the-scenes calls to a server model. Bonus: the on-device model is designed to be as power-efficent as possible to preserve your device battery life.... but of course, saving electricity makes it more eco-friendly too.

Thanks @azamsharp! That is almost definitely a bug with the guardrails. Can you file a Feedback report? In the Playground Canvas, on the far right of the "Playground has encountered an error" card, you should see a "!" icon that will let you directly file a Feedback bug report. To help us track down the issue, would you also be willing to share your Siri language setting? That can be found in the Settings app > Apple Intelligence & Siri > Language. The guardrails have some language-specific behavior, so there's a known issue we're tracking where guardrails are over-triggering for some languages and locales.

Happy to help answer some of these! Yes. The Foundation Models framework uses the on-device system foundation model. It's an ~3 billion parameter model designed and trained by Apple. Is the language model provided by FoundationModels designed and trained by Apple? Or is it based on an open‑source model? The exact same on-device model is available on iOS, iPadOS, macOS, and VisionOS on all devices that support Apple Intelligence. Is this on‑device model available on iOS (and iPadOS), or is it limited to macOS? No. The general on-device model you access via Foundation Models is not a coding model and isn't suitable for most code tasks. Xcode on Mac has a separate built-in coding model to help with code completion. When I write code in Xcode, is code completion powered by this same local model? If so, why isn’t the same model available in the left‑hand chat sidebar in Xcode (so that I can use it there instead of relying on ChatGPT)? Yes and no. You can give the on-device model access to that kind of information using Tool calling and the normal Swift Calendar/MailKit/Photos frameworks with normal app entitlements to request permission from your app's user. There is no way to give the on-device model blanket permission to a person's information. Requesting access to information is the same as it would be for any app feature. Can I grant this local model access to my personal data (photos, contacts, SMS, emails) so it can answer questions based on that information? If yes, what APIs, permission prompts, and privacy constraints apply? You can read about all the details of Apple's model architectures and training... in this 2024 article introducing Apple Intelligence models: https://machinelearning.apple.com/research/introducing-apple-foundation-models and this 2025 update article: https://machinelearning.apple.com/research/apple-foundation-models-2025-updates

Yes, this is most likely the case. According to this support doc How to get Apple Intelligence: Apple Intelligence features will not currently work for supported devices purchased in China mainland. For supported devices purchased outside of China mainland, Apple Intelligence features will not currently work if you are in China mainland and if your Apple Account Country or Region is also in China mainland. For the Foundation Models framework on-device model asset to be on your device, you need Apple Intelligence enabled on your device. Check this in the Settings app. For a more detailed model availability status in Foundation Models Framework, you can print out SystemLanguageModel.Availability

Great question @Hunter! You're completely correct, this error message is very confusing: Context(debugDescription: "Content contains 4089 tokens, which exceeds the maximum allowed context size of 4096.", underlyingErrors: []) We're actively investigating the issue, but until we have the error message fixed, here's what's going on: The context window size is always 4096 When the model throws this error at a number close to to limit (4089 above) that means the model doesn't have enough tokens to generate its response. In other words, it had 4096 - 4089 = 7 tokens left, but it needs more than 7 tokens to produce its response.

While the Foundation Models framework currently doesn't have a direct API for estimating tokens, you can use the Foundation Models Xcode Instrument to measure tokens. Alternatively, to fit the maximum number of tokens, this is the trick I usually use: Send the maximum amount of "typical for your use case" content you'd like to send the model. Catch and print the result of the exceededContextWindowSize error to Xcode console. The error printout will give you the token count of your content, something like: Content contains 9056 tokens, which exceeds the maximum allowed context size of 4096. Start cutting down on your input content until you get under the limit and no longer hit the error. Use your under-the-limit content length to estimate the allowable character length for your future content. Note there's a common gotcha: remember you need to budget for enough context for the model's response. As you get close to the context limit, you may see an exceededContextWindowSize error like: Content contains 4092 tokens, which exceeds the maximum allowed context size of 4096. While 4092 is less than 4096, this error means that the model can't finish its response without going over the limit. The solution is to cut down your input content length further until the model has enough context to generate its response.

Howdy! When the model "get's stuck" the only work-around is to start a new LanguageModelSession. Generally "getting stuck" is a signal that the model is struggling with your request. I'm not surprised the model is struggling with this one. The on-device ~3B language model isn't a coding model, so while it does know a little bit about code, it hasn't been trained for most code tasks. In this case, the model is giving its best shot but can't quite solve the task. I'd recommend avoiding code tasks for this model.

That's correct! We deprecated ToolOutput in beta 5, and replaced it with either String or GeneratedContent tool output. For more, see the Tool's doc: https://developer.apple.com/documentation/foundationmodels/tool