Post

Replies

Boosts

Views

Activity

M3 Neural Engine kernelDMA bandwidth throttled at 1 MiB multiples; splitting restores 3x DRAM bandwidth
A suspected RTL performance erratum in the Apple Neural Engine throttles DRAM weight streaming throughput down to 17–19 GB/s from the nominal 45–60 GB/s, whenever the total weight size is an integer multiple of 1 MiB. This was confirmed on M3 Macbook Air (2024) 24GB. 1 MiB is a common transfer size, and currently affects 7 of ANEMLL’s 15 baseline models. Avoiding the suspected problematic path in the kernel DMA's prefetch ring by splitting 1 MiB transfers into non-multiples of 1 MiB, directly increased Llama 3.2 1B token throughput from 10.0 to 24.3 tokens/s (DRAM usage from 24.7 to 60.0 GB/s), and Qwen3-8B from 1.36 to 2.97 tokens/s (DRAM usage from 22.4 to 48.7 GB/s). Please see writeup here: https://eiln.github.io/posts/ane-dma.html I'm looking to confirm that this behavior is reproducible on your side, and will provide the scripts in my plots.
0
0
31
15h
M3 Neural Engine kernelDMA bandwidth throttled at 1 MiB multiples; splitting restores 3x DRAM bandwidth
A suspected RTL performance erratum in the Apple Neural Engine throttles DRAM weight streaming throughput down to 17–19 GB/s from the nominal 45–60 GB/s, whenever the total weight size is an integer multiple of 1 MiB. This was confirmed on M3 Macbook Air (2024) 24GB. 1 MiB is a common transfer size, and currently affects 7 of ANEMLL’s 15 baseline models. Avoiding the suspected problematic path in the kernel DMA's prefetch ring by splitting 1 MiB transfers into non-multiples of 1 MiB, directly increased Llama 3.2 1B token throughput from 10.0 to 24.3 tokens/s (DRAM usage from 24.7 to 60.0 GB/s), and Qwen3-8B from 1.36 to 2.97 tokens/s (DRAM usage from 22.4 to 48.7 GB/s). Please see writeup here: https://eiln.github.io/posts/ane-dma.html I'm looking to confirm that this behavior is reproducible on your side, and will provide the scripts in my plots.
Replies
0
Boosts
0
Views
31
Activity
15h