I’ve also been having this issue with an iPhone 12 Pro, I read somewhere that iOS 14.5 fixes the issue. I upgraded to the beta and I haven’t been having the issue since. Hope this helps others.