Update on LocalLM Lab. With macOS 27 now shipping, the full model layer is available to any Apple Silicon macOS app through one session API. That includes Apple's on-device Foundation Models, downloadable open-weight models, ClaudeForFoundationModels, Private Cloud Compute, and online providers like OpenAI, Anthropic, and OpenRouter. I also put together a new reference app (source code included) for this release. VistaNova is a web-search tool built using two different on-device AI models doing two different jobs. Apple's on-device Foundation Model handles the initial search query and the MCP tool call out to Tavily. A downloadable Qwen model (via MLX) handles summarizing what comes back. Useful if you don't want to manually dig through the search result links. Also in this release, which is relevant if you're wiring your own tool-calling into a Foundation Models app: the old all-or-nothing "Full Access" toggle is gone, replaced by ToolCallAuthorizer. That's graded tool selection plus per-call authorization, surfaced through a new Security panel. The SDK's MCP client is also now current with protocol revision 2025-11-25 (structured tool results, an elicitation UI, full OAuth support). SDK guide: https://github.com/ancientcomputing/locallm/blob/1.0.0-beta/docs/sdk-guide.md VistaNova: https://github.com/ancientcomputing/vistanova

Check out LocalLM Lab (locallmlab.dev) which I also post about here. It's a development tool for working & experimenting with local AI models. If you a building AI-powered app on macOS, the companion SDK provides a lot of the lego pieces to do so. And example apps with source code are available for folks to use as a starting point.

Update on LocalLM Lab: 1.0.0-beta.3 extends the unified model layer to online providers: OpenAI, Anthropic's Messages API, OpenRouter or any OpenAI-compatible server. This is through a new RemoteModelProvider, shipped as part of a fourth SDK binary (LocalLMLabSDKRemote). Configuration is a RemoteProviderConfig (scheme, dialect: .openAIChat / .openAIResponses / .anthropicMessages / .openAICompatible, baseURL, auth, models), with presets for .openAI, .openAIResponses, .anthropic, and .openRouter. Routing and session creation work identically to local/on-device models — lab.models.route("chat", to: ModelID(scheme: "openai", rest: "gpt-…")!), then lab.makeSession(route: "chat"). Even if you are not planning to ship an app with online AI models, enabling them in the app during development is useful. If the app isn't behaving, it's often unclear whether the app's tool-calling/framework is broken or the local model you picked just isn't following instructions well. Point the same route at a frontier model for a minute, confirm the plumbing works, then go back to evaluating the local model on its own merits with that variable removed. Another quick update for folks using xcode IDE, the examples now come with .xcodeproj files. Feature page: thisbrain.ai/locallm/1.0.0-beta

Update on LocalLM Lab: 1.0.0-beta.1 adds a model layer to the SDK. This means one API across Apple's on-device FoundationModels model, Claude, and a locally-run open-weight model via MLX (a separate runtime, LocalLMLabSDKInference), with routing and model residency (which model stays warm, how many at once) owned by the SDK rather than something every app has to build itself. This is entirely SDK-side, not a FoundationModels API change. SystemModelProvider still wraps SystemLanguageModel under the hood the same way it always has. A fourth provider, PCCModelProvider (Apple's Private Cloud Compute), ships in this release but isn't functional yet in 1.0.0-beta.1. Check out https://developer.apple.com/private-cloud-compute/ for more info on this. Requires macOS 27 beta, Apple Intelligence enabled, Xcode 27 beta for SDK development. SDK guide: https://github.com/ancientcomputing/locallm/blob/main/docs/sdk-guide.md Feature page: thisbrain.ai/locallm/1.0.0-beta

Thanks. This is useful information. I built PCC support into LocalLM Lab (and SDK) and it currently reports an error. I'll add the guidance into the SDK documents.

Quick 0.8 update on this. The SDK now ships ready-made Tool wrappers for every connector (Calendar, Reminders, Contacts, Location etc) and new this release, Filesystem via WorkspaceTools. Previously, you would have to write your own adapters (which remains fully supported alongside the new tools). We ended up going the Tool wrapper route because Apple's local AI (at the macOS 26 level) wasn't reliable at mapping (to give an example) a human-readable Calendar event (Lunch on August 25) to the event ID that the connector used to require to perform any action on that event. Some of you will wonder: isn't that Apple's Tool protocol reimplemented? Well, it's Apple's Tool protocol used the way it's meant to be, with the annoying parts done for you. That means handling connector-level gotchas like the identifier problem above so you don't have to rediscover them yourselves, plus one piece Apple doesn't provide at all: MCPTool, which builds a Tool at runtime straight from an MCP server's JSON Schema. FoundationModels has zero MCP support of its own. Short version: we're not replicating Apple's Tools, we're using them, and taking care of the parts that were genuinely annoying to get right. Filesystem access itself (the picker, security-scoped bookmarks) is still deliberately host-app territory, not part of the SDK. Everything downstream of that (once your app has a resolved folder URL) is where WorkspaceAccess/WorkspaceTools come in. This includes an async-aware access pattern (withFolderAccessAsync) that is needed since a model session can invoke several file tools across one respond(to:) call. The new source-code-included example (workspace-buddy) shows it in context. Maybe it's your route to building a coding agent using the local AI ...!

Another update relevant to this thread's original question about iterating on prompt/context shape before writing app code: v0.6 adds localai-cli, a small subprocess binary for calling the on-device model straight from your own Swift (or Python) code once you've got a prompt shape you're happy with. Mechanically it's about as simple as it gets: your app spawns the binary, writes one JSON request (system prompt + input, or a messages array for multi-turn) to stdin, reads one JSON response from stdout. No HTTP server and no client library. This is useful if you've been iterating in Prompt Playground and want to move that exact prompt into a real Xcode project without wiring up LanguageModelSession and the connector/MCP plumbing yourself. It's config-scoped to whatever the user already granted through the app's own settings ( connectors and MCP tools included) so a request outside that set is rejected before the model runs at all. Omitting the field grants nothing. Being upfront: this isn't a packaged SDK for shipping. LocalLM Lab has to be running for calls to work, since they relay through its background process. It's meant for the iterate-then-verify step this thread was originally about, not for distributing inside another app yet. There's a fuller worked example on the page now too. "Plate Today" combines three connectors (Calendar, Reminders, System Clock) with an MCP tool call (Todoist's find-tasks-by-date) in a single request, and reads localai-config.json itself first to confirm everything's actually enabled before ever invoking the model. Useful reference if you're weighing how much validation to push into your own code versus relying on localai-cli's own rejection behavior. The example does both, and the preflight is purely about not spending a model call on a request you already know will fail. Interface + Swift/Python examples: thisbrain.ai/locallm/cli.html

Oh and I forgot to include the writeup on the different MCP servers I have tested with: https://thisbrain.ai/locallm/mcp-servers.html

Small(-ish) update relevant to the context-budget part of this thread: I ended up building MCP client support into LocalLM Lab (v0.4), and it turned the "what fits in the context window" question from the original post into something very concrete. MCP servers advertise their tools with full JSON schemas, and those schemas count against the same ~4096-token budget as everything else the model sees. Some servers are cheap (a couple hundred tokens for everything); others are not. For example, Linear's full tool list alone would blow the budget on its own. So the UI ended up being: every tool starts disabled per server, and you enable only what a given task needs, one or two at a time. This covers both privacy+security in a dev environment. It's a more literal version of the same problem this thread was originally about - deciding what the model actually needs to see versus what's just available. One upside of building against this specific constraint: since the model's already local and free to call, there's no API cost or sandbox setup involved in actually watching tool discovery and tool calls happen against a real server - useful if you're trying to understand the protocol itself, not just ship a feature with it. I am curious whether anyone else doing tool-calling with LanguageModelSession has run into the same tradeoff, and how you're deciding what to expose per request versus keeping available but unused.