Hi,
I have trained a basic adapter using the adapter training toolkit. I am trying a very basic example of loading it and running inference with it, but am getting the following error:
Passing along InferenceError::inferenceFailed::loadFailed::Error Domain=com.apple.TokenGenerationInference.E5Runner Code=0 "Failed to load model: ANE adapted model load failure: createProgramInstanceWithWeights:modelToken:qos:baseModelIdentifier:owningPid:numWeightFiles:error:: Program load new instance failure (0x170006)." UserInfo={NSLocalizedDescription=Failed to load model: ANE adapted model load failure: createProgramInstanceWithWeights:modelToken:qos:baseModelIdentifier:owningPid:numWeightFiles:error:: Program load new instance failure (0x170006).} in response to ExecuteRequest
Any ideas / direction?
For testing I am including the .fmadapter file inside the app bundle. This is where I load it:
@State private var session: LanguageModelSession? // = LanguageModelSession()
func loadAdapter() async throws {
if let assetURL = Bundle.main.url(forResource: "qasc---afm---4-epochs-adapter", withExtension: "fmadapter") {
print("Asset URL: \(assetURL)")
let adapter = try SystemLanguageModel.Adapter(fileURL: assetURL)
let adaptedModel = SystemLanguageModel(adapter: adapter)
session = LanguageModelSession(model: adaptedModel)
print("Loaded adapter and updated session")
} else {
print("Asset not found in the main bundle.")
}
}
This seems to work fine as I get to the log Loaded adapter and updated session. However when the below inference code runs I get the aforementioned error:
func sendMessage(_ msg: String) {
self.loading = true
if let session = session {
Task {
do {
let modelResponse = try await session.respond(to: msg)
DispatchQueue.main.async {
self.response = modelResponse.content
self.loading = false
}
} catch {
print("Error: \(error)")
DispatchQueue.main.async {
self.loading = false
}
}
}
}
}
