This also occurs when using SwiftUI previews. I have the latest Big Sur 11.1(20C69) and Xcode 12.4 (12D4e) For me killing/restarting the core audio daemon [$ sudo pkill coreaudiod] (scroll below) did the trick. Still a pain... Apple please fix!