Cannot pattern match LanguageModelError from a response stream

Question

Created 2d

Replies 3

Boosts 0

Participants 3

The LanguageModelSession.GenerationErrors seems to be deprecated in favor of LanguageModelError for the most part.

Now... when iterating through the ResponseStream<String> of a LanguageModelSession.streamResponse(to:options:), with a good old for await, the async iterator .next() can throws. Leaving aside that it is not very conspicuous at the call site it will throw... in the do/catch, the error thrown does not see to be able to be pattern matched to the new LanguageModelError with something like catch let error as LanguageModelError.

It was able to patten match the GenerationErrors before just fine, so may be an oversight/bug?

Answered by Apple Designer in 892197022

Thanks so much for the Feedback report with the minimally-runnable project! That really helps.

So currently we are not able to replicate this issue on macOS 27.0 and Xcode 27.0, but given similar historical issues we had at launch last year, I highly suspect the underlying cause is that you're running macOS 26.

Why? Xcode 27.0 contains the latest SDK, but the on-device SystemLanguageModel is actually built into the OS. Meaning that when you run simulator from Xcode, the simulator is actually "punching out" to macOS to run the model, using the 26.5 model inference code in the OS. Whenever we see "weird" errors like this, it's usually an underlying incompatibility between the Xcode SDK and OS for running the model. :(

Suggested Fix Update a physical device to 27.0.

Answer 1

Frameworks Engineer OP

Apple

2d

Below is how you can pattern match with errors in the framework. Could you try this?

    let session = LanguageModelSession()
    let stream = session.streamResponse(to: "Tell me about origami.")

    do {
        for try await partialResponse in stream {

        }
    } catch let error as LanguageModelError {
       
    } catch let error as LanguageModelSession.Error {

    } catch let error as LanguageModelSession.GenerationError {
       // Deprecated in 27.0
    } catch {
        
    }

Answer 2

fbeeper OP

1d

Filed FB23061009 with minimal sample triggering when using the simulator (given this other issue described in https://developer.apple.com/forums/thread/831448 / FB23060822).

Answer 3

Apple Designer OP

Apple

1d

Accepted Answer

Thanks so much for the Feedback report with the minimally-runnable project! That really helps.

So currently we are not able to replicate this issue on macOS 27.0 and Xcode 27.0, but given similar historical issues we had at launch last year, I highly suspect the underlying cause is that you're running macOS 26.

Why? Xcode 27.0 contains the latest SDK, but the on-device SystemLanguageModel is actually built into the OS. Meaning that when you run simulator from Xcode, the simulator is actually "punching out" to macOS to run the model, using the 26.5 model inference code in the OS. Whenever we see "weird" errors like this, it's usually an underlying incompatibility between the Xcode SDK and OS for running the model. :(

Suggested Fix Update a physical device to 27.0.