In container app we are observing for both NEVPNConfigurationChange and NEVPNStatusDidChange notifications. But we do not receive any update there. Probably unrelated but only callback or update we receive is in path update handler of NWPathMonitor but path remains satisfied with same interface.

We are not using SCDynamicStore and have very limited understanding of it. Can you let us know how can we use this to get status of utun interface? How can we get a change in status callback and is it possible to use any API to change the status from container app?

We opened a TSI where Matt informed us that NWProtocolWebSocket does not have HTTP stack (unlike URLSessionWebSocketTask) and can not parse HTTP responses from server. Therefore, any error returned as part of HTTP response from server during WebSocket handshake will not be available to clients. We have opened a feedback to have ability to parse and return HTTP response in NWProtocolWebSocket: FB9878278

Thanks for the tip @eskimo. Currently we are supporting macOS 11+ but will definitely use the new API for Monterey. Here is the feedback for documentation update: FB9964722 As for the original issue, we noticed that replacement with same version only happens when developer mode is enabled (systemextensionsctl developer on), on the machine, regardless of SIP status. Once we turn off developer mode, the API starts working properly and we do not get delegate callback for replacement of same version. It directly jumps to completed result. I am not sure why this behaviour and could not find any documentation regarding this. I hope this information helps others in future.

Thanks for reply, Matt. We do not have the certificates in any of keychains and we do not have MDM on the machine. We have also tried setting true in SecTrustSetAnchorCertificatesOnly(_:_:) so that only our certificates are used for validation. The outcome is still the same.

We did further testing and we figured that if intermediate CA certificates are available on either side (part of anchor certs from client or part of trust object from server), they are simply used for establishing the trust chain to root but if any intermediate certificate is missing on both side, then evaluation fails. Here are our observations: Given we have a Root certificate (rootCA), two intermediate certificates (iCA1 and iCA2) Given leaf certificate is always available on both sides If server presents rootCA as well as iCA1 and iCA2 while the client anchors rootCA as well as iCA1 and iCA2, the trust evaluation is successful. If server presents rootCA as well as iCA1 and iCA2 while the client anchors rootCA as well as iCA1 but no iCA2, the trust evaluation is successful. If server presents rootCA as well as iCA1 and iCA2 while the client anchors rootCA as well as iCA2 but no iCA1, the trust evaluation is successful. If server presents only iCA2 while the client anchors rootCA as well as iCA2 but no iCA1, the trust evaluation fails since iCA1 is missing on both sides. If server presents iCA1 and iCA2 while the client anchors rootCA only, the trust evaluation is successful. If server presents only iCA2 while the client anchors rootCA and iCA1, the trust evaluation is successful. Are these outcomes expected? Is this how certificate evaluation expected to behave?

There is only one PacketTunnel provider configured and running, the one in Connected state. The other one in Running state is a content filter. We see duplicates and inconsistent information for same one packet tunnel VPN.

Thanks Matt, I will do that. It's just that we don't have definite steps to reproduce this issue. It happens after a long time of running the packet tunnel provider. We will try to reproduce and file a bug report.

Thanks @eskimo, for the link. Is this WebSocket server local to the Mac? Or something that you’d expect to be accessible via the default hardware interface? No, its a remote WebSocket-server. Just in case it helps, We have split tunnel with only a fixed private range of IPv4 addresses in include routes. So, there should not be any cause of the loop in the provider. We also have a transparent proxy provider in same system extension. That also is seeing "Network is down" error on NWConnection, when PacketTunnel provider receives it. What are other macOS NECP policies that I should look out for? Since the issue happens at random time and goes away upon machine restart (restarting just the PacketTunnelProvider does not help), that makes it look like its some setting misconfiguration, but I'm not sure what. We have seen this issue with people in different network and geographies. We also have a NWPathMonitor running which reports that none of wifi, wiredEthernet, cellular or other interface type is connected or path is unsatisfied, when this issue starts happening.

Finally one of our beta testers reported this issue and we were able to get sysdiagnose from their machine. A bug has been submitted: FB11594187

That sounds like a bug to me, and I encourage you to file it as such. Please post your bug number, just for the record. Sure, I will. Which seems like the basis of a workaround Yes but there is no way to know that tunnel configuration has failed and we need to double apply settings in all the cases. Is there a nicer way to know whether our settings have been applied? Or is there a way we can get the routing table in code and if our include route does not exist, apply the settings again?

We ran route monitor during upgrade and we see below sequence of message: got message of size 180 on Thu Oct 6 15:57:00 2022 RTM_ADD: Add Route: len 180, pid: 135, seq 88, errno 17, flags:<UP,CLONING,STATIC> locks: inits: sockaddrs: <DST,GATEWAY,NETMASK,IFP,IFA> 240.0.0.1 255.192.0.0 240.0.0.1 got message of size 180 on Thu Oct 6 15:57:00 2022 RTM_DELETE: Delete Route: len 180, pid: 135, seq 89, errno 3, flags:<UP,CLONING,STATIC> locks: inits: sockaddrs: <DST,GATEWAY,NETMASK,IFP,IFA> 240.0.0.1 255.192.0.0 240.0.0.1 got message of size 180 on Thu Oct 6 15:57:00 2022 RTM_ADD: Add Route: len 180, pid: 135, seq 90, errno 17, flags:<UP,CLONING,STATIC> locks: inits: sockaddrs: <DST,GATEWAY,NETMASK,IFP,IFA> 240.0.0.1 255.192.0.0 240.0.0.1 As per these messages, add route is failing because route exists but delete route is failing because route does not exists which is quite weird.

This issue has been fixed in Ventura Beta 11 and the one after that. Waiting for public release to verify on that.