I had this when updating to Big Sur 12.1, but haven’t experienced it again since updating to 12.2.1 (MacBook Pro 2017, one monitor without an adaptor and the other HDMI with adaptor)