I was getting this frequently also running Xcode 12.4 and Mac Air M1.   Starfia's suggestion of disabling/removing the my networked iPhone seems to have helped. Suggest others try that at a minimum.