UDP TransparentProxyProvider

With my UDP Flow Copier working as demonstrated by the fact that it is proxying DNS traffic successfully, I am finally writing tests to verify UDP packet filtering. I'm sending packets to a public UDP echo server and reading the response successfully. In my initial testing however the TransparentProxyProvider System Extension is not intercepting my UDP traffic. handleNewUDPFlow() is being called for DNS but not for my test case UDP echo sends and receives. I've tried sending UDP with both GCDAsyncSocket and NWConnection as:

connection = NWConnection(host: host, port: port, using: .udp)

Is there some other criteria for UDP datagrams to be intercepted? Google search suggests this might be a known issue for connected or async UDP sockets.

Answered by DTS Engineer in 852787022

The most obvious cause for a problem like this is your NETransparentProxyNetworkSettings, and specifically the value for includedNetworkRules. How do you set that up?

In my test project I do this:

let includedNetworks = [("0.0.0.0", 0), ("::", 0)]
    .map { addr, prefix -> NENetworkRule in
        let endpoint = NWHostEndpoint(hostname: addr, port: "12345")
        return NENetworkRule(destinationNetwork: endpoint, prefix: prefix, protocol: .any)
    }

I tried this out on macOS 15.5 and it seems to work. Specifically:

  1. I enabled the transparent proxy.

  2. On another Mac, I started a UDP server using nc:

    nc -u -l 12345
    
  3. I connect to it from three clients:

  • nc, which uses a connected UDP socket

  • QNWTool, a test tool I wrote that uses NWConnection

  • UDPSocketTest, a test tool I wrote that uses a non-connected BSD Sockets

    In all cases my provider saw the flow:

type: debug
time: 12:57:49.912415+0100
process: com.example.apple-samplecode.QNE2TransparentProxyMac.SysEx
subsystem: com.example.apple-samplecode.QNE2TransparentProxyMac
category: proxy
message: will let system handle flow, old, app: com.apple.nc, type: UDP, remote endpoint: 192.168.1.39:12345

type: debug
time: 12:58:19.749922+0100
process: com.example.apple-samplecode.QNE2TransparentProxyMac.SysEx
subsystem: com.example.apple-samplecode.QNE2TransparentProxyMac
category: proxy
message: will let system handle flow, old, app: QNWTool, type: UDP, remote endpoint: 192.168.1.39:12345

type: debug
time: 12:58:48.658703+0100
process: com.example.apple-samplecode.QNE2TransparentProxyMac.SysEx
subsystem: com.example.apple-samplecode.QNE2TransparentProxyMac
category: proxy
message: will let system handle flow, old, app: UDPSocketTest, type: UDP, remote endpoint: 192.168.1.39:12345

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

The most obvious cause for a problem like this is your NETransparentProxyNetworkSettings, and specifically the value for includedNetworkRules. How do you set that up?

In my test project I do this:

let includedNetworks = [("0.0.0.0", 0), ("::", 0)]
    .map { addr, prefix -> NENetworkRule in
        let endpoint = NWHostEndpoint(hostname: addr, port: "12345")
        return NENetworkRule(destinationNetwork: endpoint, prefix: prefix, protocol: .any)
    }

I tried this out on macOS 15.5 and it seems to work. Specifically:

  1. I enabled the transparent proxy.

  2. On another Mac, I started a UDP server using nc:

    nc -u -l 12345
    
  3. I connect to it from three clients:

  • nc, which uses a connected UDP socket

  • QNWTool, a test tool I wrote that uses NWConnection

  • UDPSocketTest, a test tool I wrote that uses a non-connected BSD Sockets

    In all cases my provider saw the flow:

type: debug
time: 12:57:49.912415+0100
process: com.example.apple-samplecode.QNE2TransparentProxyMac.SysEx
subsystem: com.example.apple-samplecode.QNE2TransparentProxyMac
category: proxy
message: will let system handle flow, old, app: com.apple.nc, type: UDP, remote endpoint: 192.168.1.39:12345

type: debug
time: 12:58:19.749922+0100
process: com.example.apple-samplecode.QNE2TransparentProxyMac.SysEx
subsystem: com.example.apple-samplecode.QNE2TransparentProxyMac
category: proxy
message: will let system handle flow, old, app: QNWTool, type: UDP, remote endpoint: 192.168.1.39:12345

type: debug
time: 12:58:48.658703+0100
process: com.example.apple-samplecode.QNE2TransparentProxyMac.SysEx
subsystem: com.example.apple-samplecode.QNE2TransparentProxyMac
category: proxy
message: will let system handle flow, old, app: UDPSocketTest, type: UDP, remote endpoint: 192.168.1.39:12345

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I extracted my NWConnection send UDP code into a tiny separate program and see the packets being intercepted. I suspect the problem was running the code in the context of a test case with the same bundle ID and signature as the SysEx. In a SysEx context it appears the TransparentProxyProvider uses the identity of the sender to avoid intercepting its own packets.

I updated my test case to call out to shell script to send UDP echo datagrams to work around the problem.

I miss Mentat Portable Streams :-)

I still want to add a UDP receive capability within my test case but since packets are being sent from an external shell script to avoid the proxy provider context the typical send and receive on the same socket from inside my test case won't work. Since I'm sending to an echo server I need to know the send port or receive on the sending socket.

Testing UDP through a SysEx seems like kind of a hassle. If I knew exactly what criteria were preventing UDP packets generated from within my test case from being proxied I could hopefully find an easier work around. It's not the bundle ID since each component (App, SysEx, and Tests have their own). The mystery continues...

What do you mean by “test case” in this context? A unit test running using XCTest? Or Swift Testing? Or something else?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

A unit test in Swift Testing.

A unit test in Swift Testing.

Do you see different behaviour based on the value in the Testing > Host Application popup in the General tab of the target editor?

This defaults to running your tests within your app rather than in a test runner process. I’m curious if you switch it from your app to the None setting, whether the networking APIs start behaving as you’re expecting.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

With more testing I've learned there are a few reasons packets might not be intercepted. Only outbound requests (so called connections) are intercepted. Sequencing and timing when enabling a SysEx are also important.

After rewriting my UDP flow copier to use NWConnection with ".udp" and debugging I found a working combination.

UDP TransparentProxyProvider
 
 
Q