Here are some interesting things I've tried that have made some progress, but don't fully work.
Failed Approaches
AppleScript Dialog Clicker
I created an AppleScript that just runs continuously in the background when a CI job starts looking for these dialogs and tries to dismiss them. It works in local testing, but not when executing through the CI process. I assume this is because it's not being run from a terminal or over SSH and that creates some kind of execution context difference which blocks the clicker from actually working when running in CI.
I've tried running this as a simple shell script osascript /path/to/clicker.scpt & and through launchd with launchctl asuser $(id -u) /path/to/clicker.scpt &. I also tried using the launchctl version without putting the script in the background, but that didn't seem to work either.
Run GitHub LaunchAgent as a LaunchDaemon
The obvious issue is that the current LaunchAgent setup has with respect to Network Privacy is that it's not running as a LaunchDaemon, as root, or through Terminal/ssh. So I ran it as a LaunchDaemon which had it's username set to the actual CI user.
This sort of worked in that the service started, and took jobs. The first issue I ran into was that codesign had some issues with the cert chain. To solve this I unlocked the System keychain and added the Apple WWDR cert to it.
This made the build succeed, but the next issue was that the xcodebuild ... test command couldn't connect to the test manager process. I assume this was due to some kind of mixed execution context even though in Activity Monitor I saw the CI process executing as the CI user, not root. I wasn't sure how to better debug this particular approach so I abandoned it.
It would be great to get this approach working somehow, but without more knowledge on how the processes are trying to communicate and how to better debug it I'm counting this as a failed approach.
LaunchDaemon To LaunchAgent Trampoline
The next thought I had was to make a new LaunchDaemon that called a shell script in /usr/loca/bin that would kick off the LaunchAgent with something like the following:
if pgrep -u "$TARGET_UID" loginwindow >/dev/null; then
/bin/launchctl bootout gui/"$TARGET_UID" "$AGENT_PLIST" 2>/dev/null || true
/bin/launchctl bootstrap gui/"$TARGET_UID" "$AGENT_PLIST"
/bin/launchctl kickstart -kp gui/"$TARGET_UID"/com.github.runner
fi
This worked to start the actual agent, but wasn't seen as being run from the LaunchDaemon or as root so it didn't get the carve out that is mentioned in the tech note.
Untested Approaches
I'm reaching the end of the idea list for how to try to get the right execution context for this runner code, so these last ideas are very hacky, but might be very direct, but lack the resilience that using a regular LaunchAgent would have.
Self-SSH Script Execution
Since there's a carve out for commands that are running over an ssh session I could try sshing to myself and running the github runner as a script outside of the LaunchAgent. This is kind of annoying to setup due to keys and passwords, but might be easy enough to try.
LaunchAgent AppleScript
I think also making a LaunchAgent that launches an AppleScript that launches Terminal.app and runs the github runner script might also work since the command execution would be from within Terminal.app which is also designated as a valid carve out.
Disable SIP
I've been trying to avoid this, but at this point it seems like a valid course of action if these other two attempts end up with a much more brittle solution.
Again, I'd love advice or ideas on how to deal with this issue or confirm that any of these approaches is less bad than the others (they are all a little wonky!)