@meaton I have noticed that the port defined in the rules as "0...52" and "54...65535". Is this range notation legal at all? This is something on my to-do list to research. I've tested it and it does seem to work, but just because it works does not mean it is valid and thus why I have not called it out one way or the other. (Usually I would call it out right away.) I will keep this thread updated as I find out more. Is there any update on this? (Please let me know if it is better to create a new thread).

As for SSH client utilizes 100% of CPU (FB9070195), We have analyzed the source code of macOS SSH client. It seems that the problem is known and it has been already solved in the regular version of OpenSSH: https://github.com/openssh/openssh-portable/commit/65d6fd0a8a6f31c3ddf0c1192429a176575cf701 So it may be needed to update SSH or apply the fix (very small).

Okay, so it looks like OpenSSH library is able to workaround this by only terminating the existing connection only if a non 0 value is returned from ssh_packet_write_poll? I am not an expert in OpenSSH code. It seems that under normal conditions (regular socket errors), connections get terminated properly. But when NETransparentProxyProvider starts, unusual things happen. The sockets get unusual errors like error 41 (protocol wrong type for socket), etc. This somehow breaks SSH logic. It goes into an infinite loop and ssh_packet_write_poll returns a non-zero. The fix just checks for this condition and terminates the connection. We are pretty much sure that this fixes the problem. Because the current version of OpenSSH (8.6 from brew) works without the problem. Although, we were unable to compile SSH client that comes with macOS (opensource.apple.com). The build environment is too complex to setup properly. Another (a better) way to solve this is to make existing connections working after NETransparentProxyProvider start or at least break them the regular way that is well expected and tested by all networking apps like SSH.

Will do. Although, it is still a good idea to update macOS SSH client. The above-mentioned fix was added for a reason. The original problem was not related to Network Extension. So, it means that the bug can happen under some conditions even without Network Extension.

JFY, this bug has been reported by me as well (FB9070195) and discussed on the forum: https://developer.apple.com/forums/thread/679465 The interesting thing is that a more recent version of OpenSSH has a fix that properly detect this unexpected behavior: https://github.com/openssh/openssh-portable/commit/65d6fd0a8a6f31c3ddf0c1192429a176575cf701

That’s not the best way to approach this. Rather, use the audit token to create a code object (SecCodeCopyGuestWithAttributes) and then use that to get the path to the code (SecCodeCopyPath).  We have faced an unexpected problem with this approach. Apparently, SecCodeCopyGuestWithAttributes has to deal with sandbox when working with the attributes of other programs. That’s OK when programs are located in /Applications. But some programs including built-in ones like /usr/libexec/routined are in other locations. So, when SecCodeCopyGuestWithAttributes is called for them, sandbox just kills the network extension completely! 2021-06-04 09:10:47.927841+0800 0x4cd Error 0x0 147 0 sandboxd: [com.apple.sandbox.reporting:violation] Sandbox: com.initex.proxi(897) deny(1) file-read-data /usr/libexec/routined Violation: deny(1) file-read-data /usr/libexec/routine Is there a way to fix this? For obvious reasons, the com.apple.security.temporary-exception.files.absolute-path.read-only trick does not work because the path is not known. I think, this also creates a security risk. We saw a case when a program with a non-standard executable path just stopped network extension of an antivirus!

It sounds like your NE provider process is terminating, but a sandbox violation will not, in general, kill the violating process. You are correct. Thanks. It turned out that the problem has nothing to do with the sandbox. There is something else on the affected systems that restarts the network extension. We will try to investigate this further.

FYI, SSH problem is fixed in Big Sur 12 beta. Apparently, this was done by SSH update while the root problem still persists (FB9125326). Many other apps are affected. I think, the best way to solve this and many other problems would be to keep existing connections alive when App Proxy is activated (FB8969320).

For selectively handling flows you will need to use NETransparentProxyProvider. Unfortunately, NETransparentProxyProvider cannot process UDP port 53. This is explicitly stated in the documentation. It would be really nice to have an option to handle DNS flows selectively. . When handleNewUDPFlow is called you will get a localEndpoint in NEAppProxyUDPFlow and remoteEndpoint from the method call.. Sorry, my question wasn't clear enough. Suppose that we have a new DNS request to DNS server 1.1.1.1. The request is to resolve apple.com. Its flow gets to handleNewUDPFlow with remoteEndpoint=1.1.1.1. Is there a way to understand the hostname being resolved (apple.com) on this stage?

Sorry for the delayed reply. Have you tried using a hostname with port 53? For example: NENetworkRule(remoteNetwork: NWHostEndpoint(hostname: "example.com", port: "53"), Is there a way to apply this rule for all hostnames?

As far as I remember, "Protocol wrong type for socket" error for ICMP ping may indicated that you set includeAllNetworks=true during VPN initialization. Please try set to false.