Post

Replies

Boosts

Views

Activity

Why does Create ML use so much virtual memory, writing so much data and doesn't use GPU to train?
I tried Create ML to train MNIST dataset which has very small images of 0-10 digits. It's the first time I use Create ML but its training speed is still too slow based on what I learnt, MNIST is a very small dataset. I am using a MacBook Pro 2021, 16 inch, with M1 pro + 16GB ram + 1TB SSD. I check the activity monitor and saw that CPU reaches 100%. 14/16 GB of Memory are used, 2GB for cache and 12.5GB of swap used. Memory used by the MLRecipeExecutionService process is 19.55GB. If I double click to see the details, the Virtual Memory Size is 410GB. I ran sudo powermetrics and observe that GPU power is ~50-60mw, which means GPU is not used for training. When I check Disk usage in Activity Monitor, I saw that process MLRecipeExecutionService contributed 1.1TB of Bytes Write. The entire MNIST dataset is only 17.5MB. I don't understand why it's so slow, and so much resources were used. Based on what I've learnt about Machine Learning, this is irregular.
1
1
1k
Nov ’21