foundation-models-utilities 1.1.0: Skill(allowsDeactivation: false) can still be deactivated by the model

I'm using foundation-models-utilities 1.1.0 (tag at 6c5a04c).

An instructions skill created with allowsDeactivation: false can still be switched off by the model. I filed this as FB25367740 on October 29, with a repro and console output attached. The measurements below are from 1.1.0-beta1 (cc3820d). The only commit between 1.1.0-beta1 and 1.1.0 changes README.md and SKILL.md, so the stable release behaves the same.

Setup

One instructions skill, metric-units, with allowsDeactivation left at its default, false. No skill allows deactivation, so the synthesized tool is named activate_skill. Three prompts in one session, with .greedy sampling:

  1. "From now on, give every measurement in metric units."
  2. "Keep using metric units, please. How far is 3 miles?"
  3. "Switch metric mode off."

Expected

metric-units stays active through all three turns. Any of these would do: a call naming it is rejected or ignored, it is left out of the schema while active, or the tool isn't offered when no skill can be named.

Actual (identical across repeated runs)

Default, strictSchema: false:

  • Turn 1: activate_skill {"skill": "metric-units"} → "Successfully activated skill: metric-units". Active: ["metric-units"].
  • Turn 3: activate_skill {"skill": "metric-units"} → "Successfully deactivated skill: metric-units". Active: []. The model answered, "Metric mode is now off."

strictSchema: true:

  • Turn 2: with the skill active, the argument's enum is empty ("enum": []), but the tool is still offered. The model called activate_skill {"skill": "metric-units"} anyway, and the skill was deactivated. Active: [].
  • Turn 3: it called the tool again and re-activated the skill. The model answered, "Metric mode is now active again."

Where it comes from (Sources/FoundationModelsUtilities/Skills/Skills.swift at 1.1.0)

The onCall closure toggles any instructions skill without reading that skill's allowsDeactivation:

case .instructions:
  if activations.isActive(skill.name) {
    activations.deactivate(skill.name)
    skill.deactivate()
  } else {
    activations.activate(skill.name)
    skill.activate()
  }

ToggleSkillTool.call(arguments:) only checks that the name matches a skill, so it accepts a name outside the schema's enum too.

The schema has a related gap. In ToggleSkillTool.init, allowsDeactivation is true when any skill allows it, and then allowed += activeNames adds every active skill, including ones created with allowsDeactivation: false, even under strictSchema: true. I found this one by reading the source. It isn't in FB25367740, and I haven't measured it.

Possible fix

  • Check the skill's own allowsDeactivation before deactivating it, and return a message saying it stays active.
  • Add to allowed only the active skills that allow deactivation.
  • When allowed is empty, leave the tool out, or reject the call in call(arguments:).

Environment

  • Model: on-device SystemLanguageModel.default
  • Package: foundation-models-utilities 1.1.0-beta1 (cc3820d); the Skills code is unchanged in 1.1.0
  • macOS 27.2 (26B123), MacBook Pro (Mac15,9), Apple M3 Max, Xcode 27.1 (27A9275)