Reading all the docs(1) I was under the impression that handling this error is well managed... Until I hit it and found out that the recommended handling options hide a crucial fact: in the catch block you can not do anything?! It's too late - everything is lost, no way to recover... All the docs mislead me that I can apply the Transcript trick in the catch block until I realised, that there is nothing there !!!
This article here(2) enlightened me on the handling of this problem, but I must say (and the author as well) - this is a hack!
So my questions:
- is there really no way to handle this exception properly?
- if not, can we have the most important information - the count of the context exposed through the official API (at least the known ones)?
-
https://zats.io/blog/making-the-most-of-apple-foundation-models-context-window/