Post

Replies

Boosts

Views

Activity

Is there a supported way to capture per-node intermediate outputs from an ANE-scheduled model?
I'm looking for a supported way to read intermediate tensors from a model executing on the Apple Neural Engine - specifically the output of an individual node in the compiled graph, rather than only the final output. What I'm trying to do: validate a from-weights reimplementation of a model against the real thing, layer by layer. Comparing only the final output tells me the reimplementation is wrong but not where; a per-layer comparison would localise it immediately. What I've established so far: A compiled ANE program can be executed unprivileged through the public graph API, and the final readout matches, so the execution path itself is reachable. Intermediate activations don't appear in host memory during normal operation, which is expected since the scheduler keeps them in accelerator-local storage. Requesting a per-node output appears to hit a kernel-side check that an ordinary process doesn't satisfy. Questions: Is there a supported API for retrieving per-node outputs from an ANE-scheduled graph - a debug or instrumentation mode, an Instruments template, or a Core ML compute-plan facility that surfaces them? Failing that, is there a supported way to make a specific node materialise its output to a host-visible buffer - for example by splitting the graph, marking an intermediate tensor as a model output, or compiling with that node as a terminal operation? I'm aware this may change scheduling and defeat the purpose, but I'd like to know whether it's the intended approach. If neither exists, is that a deliberate design boundary rather than a gap? A clear "no" is a useful answer and I'll stop looking. I'm not asking about any particular shipped model, and this isn't a request to bypass anything - the question is whether the platform exposes per-node observability for ANE execution at all, and if so what the supported entry point is. Thanks.
0
0
211
14h
Is there a supported way to capture per-node intermediate outputs from an ANE-scheduled model?
I'm looking for a supported way to read intermediate tensors from a model executing on the Apple Neural Engine - specifically the output of an individual node in the compiled graph, rather than only the final output. What I'm trying to do: validate a from-weights reimplementation of a model against the real thing, layer by layer. Comparing only the final output tells me the reimplementation is wrong but not where; a per-layer comparison would localise it immediately. What I've established so far: A compiled ANE program can be executed unprivileged through the public graph API, and the final readout matches, so the execution path itself is reachable. Intermediate activations don't appear in host memory during normal operation, which is expected since the scheduler keeps them in accelerator-local storage. Requesting a per-node output appears to hit a kernel-side check that an ordinary process doesn't satisfy. Questions: Is there a supported API for retrieving per-node outputs from an ANE-scheduled graph - a debug or instrumentation mode, an Instruments template, or a Core ML compute-plan facility that surfaces them? Failing that, is there a supported way to make a specific node materialise its output to a host-visible buffer - for example by splitting the graph, marking an intermediate tensor as a model output, or compiling with that node as a terminal operation? I'm aware this may change scheduling and defeat the purpose, but I'd like to know whether it's the intended approach. If neither exists, is that a deliberate design boundary rather than a gap? A clear "no" is a useful answer and I'll stop looking. I'm not asking about any particular shipped model, and this isn't a request to bypass anything - the question is whether the platform exposes per-node observability for ANE execution at all, and if so what the supported entry point is. Thanks.
Replies
0
Boosts
0
Views
211
Activity
14h