I was having the same issues after running on a brand new 16" I got yesterday... I jus upgraded to Big Sur and it seems to have fixed the issue