It would appear that the problem is solved if I add only following steps into preinstall script disable network filter stop network system extension After that installd replaces the bundle and system extensions in it, postinstall then runs our helper, which registers system extenison and starts the network filter (starts TransparentProxy/AppProxy). I guess stopping system extensions while the old bundle still exists, then replacing the bundle and upgrading system extension (basically it just registers new verison of the extension as the process is not running anymore) causes OS (launchd) to clear up XPC related caches.

Perhaps this could help to fix XPC invalidation- add following to postinstall script: disable network filter stop network extension spctl -a -vv "${APP_PATH}" spctl -a -vv "/Applications/${PRODUCT_NAME}.app/Contents/Helpers/${MANAGER}.app" spctl -a -vv "/Applications/${PRODUCT_NAME}.app/Contents/Helpers/${MANAGER}.app/Contents/Library/SystemExtensions/com.company.feature.dev.systemextension" upgrade system extension enable network filter connect XPC client But it is possible that simply using sleep 30 before upgrading system extension would accomplish the same :-)

I tried killing sysextd, nesessionmanager and other processes to reset XPC. Killing launchd was my other considerartion, but that's rather severe hack to overcome some bug in XPC initialization.

The same problem here. It only happens on ARM, haven't seen it on Intel. When replacing Network System Extension during installation in postinstall script we are hitting this issue at least 50% of the time. Isn't is possible that OS cannot correctly validate signatures of the extension or client app and that makes XPC listener kick off the client? We tried to do the following Disable network filter (not removing, just changing its state to avoid OS dialog that prompts user to allow network filter) calling launchctl stop NetworkExtension.com.company.feature.version... This stops the extenison while making registration and upgrade, we were hoping that later when enabling the network filter and extension would start with proper XPC listener that would accept connection from the client. It didn't help. Our Network Extension is not sandboxed yet. Could this help? But it works after reboot, so its only an issue right after upgrading extension.

Additional Info: The issue occurs only on ARM (Ventura and Monterey) not on Intel. XPC simply doesn't work after upgrading system extension on ARM. I tried to kill sysextd, nesessionmanager and other processes. Basically I'm trying to kill a process that manages XPC connections and info about processes and their signature validation (maybe it's somewhere deep in kernel and it's not possible to resolve it by this hack). It is possible that when trying to connect to XPC while starting our services in posinstall scirpt, installd hasn't quite yet completed and perhaps OS considers bundle signature not valid yet. Workaround is rebooting the machine, but we try to avoid that as much as possible. After reboot XPC works fine with upgraded system extension.

BTW does NSProvider.startSystemExtensionMode() need to be started from the main thread? Can it be started from a background thread that doesn't event use CFRunLoop, but a unix loop based on epoll? Does NSXPCListener has to be started from main thread?

That's right. Ideally we would mute the event. But simply checking a flag to determine that the file is sandboxed and handling it by es_respond_auth_result(client, message, ES_AUTH_RESULT_ALLOW, cache) would be sufficient. If it is a file our process running under root cannot access for scanning, then there is no reason to try opening it. When Installd unpacks app bundle into its sandbox, eventually it will move it into /Applications and it can be scanned there. The reason why we try to avoid opening files we cannot open is the fact that even the failure takes time. Even decision whther to use open() or access() has impact on how long the failure lasts. If the failure to open file takes 20ms, then for example an app with 50000 files could take 1000s or 16.6 minutes. Currently I'm strugling with determining whether to use access() or open() to check if the file can be accessed. There are cases where access() check is faster and cases where open() is faster (even if the result is failure) - it depends on various factors, like process has access to file, or doesn't have basic unix permissions or file is sandboxed. That's why I'm looking for ways to avoid unnecessary file system calls.

Boot sequence with user applications starting before Launch daemons and even system extensions clearly represents a potential security risk. It is very clearly visible when using FileVault. In this case user needs to enter password, OS decrypts volume and then it can startup and launch processes, but very quickly opens user's Desktop and starts to open applications. The bigest problem is that for example Endpoint system extension, which defends computer from infections or Network system extension, which scans network traffic and defends against infected web pages, email, etc. cannot immediately block harmful content. PIDs of these processes are larger than for example Launch Daemons. It means that for example Mail could open very quickly and load new message before Network System extension is loaded and traffic captured and scanned. We see it especaily on new OS versions running on ARM where it can take up to 10-30 seconds to properly load everything to protect users. That happens when OS is setup with FileVault and user has many, many applications opened - Mail, Calendar, Safari with lots of tabs, Teams, Remote Desktop, etc, etc. When user reboots computer, we clearly see this delay in start up of critical processes that should protect users, especially in corporate environment. Bootup sequence should look something like this: System Extenisons Launch Daemons Launch Agents User applications Is there a way to setup this sequence of process start up? Thanks, Robert