This is apparently a universal problem for BigSur. I hope Apple will solve it soon with an update.

Venture 14.4 on MacMini M1 2020 I've had this for more than a year. It first appeared while I was using Zoom. Since then I've kept removing the coreaudiod files, which worked for a hour or two. I think Boom may be a culprit and just used the Boom Uninstaller, but it came back. It happens regardless of which browser and I virtually never use Safari. I think it is a memory pressure (?) issue, as I think the memory is always running high when the problem comes. I'll try reinstalling the system and see what that does.