We have a Man In The Middle proxy that supports all kind of protocols (http, tls, dns, mail protocols, grpc, etc...)... On apple devices we are running it using the Network Extension framework as a NETransparentProxy.
First of all, thank you for the framework, took a while to learn the ins and outs but it works nicely and runs smooth... However now that we start to roll it out to customers we see issues here and there.. For most it works fine, but for some that use other proxy/vpn solutions they run into all kind of "connectivity" issues...
E.g. some customers run products from companies like zscaler, fortinet, tailscale etc...
First we weren't sure if you could even run multiple TransparentProxy's that have the same network capture rules (e.g. the entire TCP range), but turns out that is fine as we tested it with a demo proxy of ours as well as the product version, both deployed as system extensions NETransparentProxy, and it is all fine.. However also here the ordering is not clear? Traffic seems to flow through both but cannot tell what the order is and if the user or we have any control over it.
Now... Our proxy is not a VPN and thus not open a tunnel to a remote location. It is local only there to protect the developer. As such in theory it should be compatible with any other VPN and proxy as any traffic we intercept (all traffic) is still ok to go through their proxy/client-vpn and than through a remote tunnel if desired.
So the questions I have is:
- Is there a way, either from within the code or that our users can configure to, on the order of multiple (network extension or other) proxies?
- Is TransparentProxy the correct solution if I also want compatibility with these other products and want to MITM the traffic?
The flows that current work fine are:
ClientApp --> NETransparentProxy[ours] --> remote target server
clientApp --> L7 HTTP/SOCKS5 Proxy (system or app-defined) --> NETransparentProxy[ours] --> remote target server
clientApp --> L7 HTTP/SOCKS5 Proxy (system or app-defined) --> NETransparentProxy[ours/demo] --> NETransparentProxy[demo/ours] --> remote target server
However when people also have products from zscaler, fortinet, tailscale or some others it seems to work sometimes but not always, which makes me think it is order defined?
What all of them have in common is that they need to go through a remote tunnel, whereas we do not go through a remote tunnel... Which if I am correct (perhaps I am not) should mean that as long as traffic always goes first via us that it should work?
e.g.
clientApp --> NETransparentProxy[ours] --> NETransparentProxy/Tunnel/...[third party vpn] --> Vpn Server --> remote target server
That should in that case just work. But it does not work in case we are behind the (vpn proxy) client.
Please let me know if I provide enough detail and if I'm clear? I am mostly wondering about
- what I can expect in terms of compatibility
- if there is anything I (or our company user) can do about ordering/priority/something ?
the ordering is not clear?
Indeed. The ordering is undefined unless the transparent proxies are deployed via MDM. In that case the device manager can specify an order using the Order property.
Once you start mixing multiple types of Network Extension providers, you have to worry about the order between those types. I’m not aware of any documentation for that, but understanding is that it’s something like: per-app VPN, content filter, relays, transparent proxy, general VPN, and finally DNS proxy.
When it comes to multi-provider compatibility, there’s one code-level step you can take: When making a connection on behalf of a flow, pass the flow metadata to the connection. See this post.
In terms of compatibility with specific products, it’s hard offer concrete advice on that front:
- It’s not clear which products use which provider types.
- Some Mac products don’t use NE at all, but instead rely on legacy VPN techniques.
- There’s no guarantee that a specific provider plays by the rules.
- Or, even if it does, whether the combination of those rules with your rules will result in what the user considers to be a successful result.
If you’re working in this space you need to budget time for this sort of compatibility testing.
Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"