I'm also seeing constant kernel panics when connecting/disconnecting my monitors. I have a 2021 16" MacBook Pro (2019 model, 2.3 GHz 8-Core Intel Core i9, 32 GB 2667 MHz DDR4, Intel UHD Graphics 630 1536 MB) and am running Big Sur 11.3.1. The machine is only a couple of weeks old and I can easily reproduce the problem by simply connecting and disconnecting the monitor cable 2 or 3 times.
I initially thought the problem was my CalDigit TS3 Plus but I started to remove all peripherals to see what was causing the problem and eventually got to a single monitor (Dell U2415) and a single USB-C to DisplayPort cable - no dock or hub involved. I disconnected keyboard, trackpad, headphones, yubi key, etc etc. I even booted in safe mode but issue persists.
My crash report is the same as reported by most folks in this thread: panic(cpu 0 caller 0xffffff801d69f6b6): [kext.kalloc.2048]: element modified after free (off:1672, val:0xfffffffe00000000, sz:2048, ptr:0xffffff93926d4000, prot:zero)
The interesting bit I found was that everything works fine if I connect my monitors via HDMI rather than DisplayPort. So at the moment I have 2 external monitor connected via USB-C to HDMI cables (1 using the TS3 dock, and 1 using a cheap USB hub).