This is not the right answer, I apologize.

Thanks Scott, you are right the iOS system never shows such an alert to users, it's triggered by third party library. Our project manager set an announcement on the Instabug to let users upgrade the app, but he didn't realize it. After I checked the configurations on the Instabug and found the condition to filter users is incorrect, I'm sure this is the root cause. But the interesting thing is that Apple support team admit this is something wrong on their side. LOL

I've contacted the Apple developer support, they confirmed this alert is triggered by iOS system. It happens when your app is built with older iOS version, but the app is installed on the latest iOS 16 devices. In this case, my app is built with Xcode 13+, targeted iOS 15 at most, but the user installed the app on iPhone 14 with iOS 16. The app is regarded as outdated, but actually it still works normally. I've rebuilt the app with Xcode 14 and published new version, this issue is resolved.