This is an ongoing issue. I believe Apple has a hard coded limit of 75% of physical memory.
More on this here (fast forwarded to the relevant part)
An example table of the limits here:
This function will tell you how must you can actively use on your own machine
Apple's solution is to break up the jobs, but apparently that's not an easy task with LLMs -- more on that here:
There is a hack that lets you adjust the VRAM/RAM split, but it unfortunately requires keeping SIP disabled.
Hopefully someone at Apple will find this and give us a means to adjust ourselves.
Topic:
Graphics & Games
SubTopic:
General
Tags: