I am porting my number to another operator as it seems to work fine on the other operator. Not ideal but HAD TO do it, thanks to Apple.

I feel iOS 14.4 Beta 1 will fix the issue on iPhone X as finally there is a modem firmware update!