I have the same problem with a 2021 16" MBP with M1 Max and 32 GB of RAM running Monterey 12.4. I'm very disappointed that Apple has not fixed this really annoying problem.