I've been having the same problem since "updating" to Big Sur. Sometimes resetting SMC and/or NVRAM helps temporarily, other times it does not help. So frustrating that Apple hasn't fixed this issue in more than 6 months. My Macbook Pro is essentially unusable most of the time.