I'm looking for a supported way to read intermediate tensors from a model executing on the Apple Neural Engine - specifically the output of an individual node in the compiled graph, rather than only the final output.
What I'm trying to do: validate a from-weights reimplementation of a model against the real thing, layer by layer. Comparing only the final output tells me the reimplementation is wrong but not where; a per-layer comparison would localise it immediately.
What I've established so far:
- A compiled ANE program can be executed unprivileged through the public graph API, and the final readout matches, so the execution path itself is reachable.
- Intermediate activations don't appear in host memory during normal operation, which is expected since the scheduler keeps them in accelerator-local storage.
- Requesting a per-node output appears to hit a kernel-side check that an ordinary process doesn't satisfy.
Questions:
-
Is there a supported API for retrieving per-node outputs from an ANE-scheduled graph - a debug or instrumentation mode, an Instruments template, or a Core ML compute-plan facility that surfaces them?
-
Failing that, is there a supported way to make a specific node materialise its output to a host-visible buffer - for example by splitting the graph, marking an intermediate tensor as a model output, or compiling with that node as a terminal operation? I'm aware this may change scheduling and defeat the purpose, but I'd like to know whether it's the intended approach.
-
If neither exists, is that a deliberate design boundary rather than a gap? A clear "no" is a useful answer and I'll stop looking.
I'm not asking about any particular shipped model, and this isn't a request to bypass anything - the question is whether the platform exposes per-node observability for ANE execution at all, and if so what the supported entry point is.
Thanks.