There are private SPIs that explicitly manage this but the "default" behavior is that a newly created process simply inherits the sandbox of it's parent process*.
Is DNS policy enforced by mDNSResponder part of the Sandbox, or is DNS policy a different thing? If it's the Sandbox, and the child process inherits the same profile, then how can DNS resolve correctly in the parent but not the child?
At a code leve, you can actually see the "blocked by policy" error in mDNSReponder's code.. In your particular case, it's almost certainly because of something about how your packet tunnel provider is configured which is then being inherited by your new child process
Right, so I took a look at the open source mDNSResponder, and this appears to be where it blocks queries.
The problem for me in understanding what's happening is that this function calls into undocumented APIs, creating something called a path evaluator, which then spits out a verdict of "satisfied" or "unsatisfied" for the DNS query. Do you know what this function actually does and how it decides whether the path is satisfied?
In collecting parameters for this evaluation query, it (again via an undocumented API) collects a "UUID" associated with the PID of the querier. Do you know what this UUID is? I've tried searching Mac docs for any mention of a process UUID, but have so far come up empty.
Even when the VPN is enabled, DNS resolution seems to work fine when I run the same binary from a Terminal, but fails when spawned from the system extension. I'm certainly not confident that my tunnel configuration itself has nothing to do with it, but it happens regardless of whether I add any DNS settings to the tunnel configuration or not. I'm flying blind here because I haven't found any information about how mDNSResponder decides what is and is not blocked by policy, and how tunnel DNS configuration might enter in the mix.
Why are you spawning a new process and what are you actually trying to do? I'm concerned that you're straying outside the guidance in TN3120, which rarely goes well.
I'm spawning a new process because the code that actually implements the tunnel is written in Go (it's essentially Wireguard), and I don't want to have Swift threads/memory management alongside Go goroutines & garbage collection in the same process. I'm following the guidance of TN3120---the goal really is a packet tunnel. The spawned process tries to dial the VPN server by DNS name and that's where it's getting hung up.
As context, what are you actually trying to resolve (bonjour or standard DNS) and what networking API are you actually using? Have you actually tested the same code in your extension (not just the same resolution with a different API)?
In the System Extension process, I've tried both the URLSession API to reach the VPN server by name and the lower level POSIX getaddrinfo. These resolve the standard DNS name correctly, and I can see mDNSResponder dropping logs about the query. The logs look the same as the ones dropped when the spawned process makes the query, other than the spawned process gets "blocked by policy."
I'm pretty sure Go uses getaddrinfo, but I can try to double check it's the same API.
I've also tried using the standard Swift Process() call to spawn the child from the System Extension, and I get the same results regarding DNS (the reason I'm using posix_spawn is to pass the TUN file descriptor, Process() only supports stdin/stdout/stderr file descriptors).