Great, thanks! Performance looks better when the batch size is better adjusted. A final question: does the GPUs in the M1 have access to the full 16GB of RAM? In NVIDIA cards VRAM can be a huge limitation when training bigger models - I would rather have an older card with more VRAM than a newer card with less (like the 4GB in the Quadro T2000)... would this be an advantage of the M1?

Yes, with limitations and several thing still under development. Check https://developer.apple.com/metal/tensorflow-plugin/ and https://github.com/apple/tensorflow_macos