I now experiencing the same issue with my Magic keyboard where I have to toggle it off and on. Guess I should stuck with Big Sur :(