XPC Communication between Editor app and user-compiled code

Hello! I'm trying to implement an editor app (macOS) that allows the user to write code, which will be compiled and executed, showing the result in the editor window.

Imagine it like SwiftUI previews, but the graphic output is created with Metal, not SwiftUI. I found that IOSurface can be used to share that kind of data over XPC, so I would not have to rely on the private NSRemoteView. However, I'm confused if it is, at all, possible for my editor app to connect to an XPC Service, that was NOT bundled with it (but compiled by it at runtime).

I succeeded to launch an XPC service defined as:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>Label</key>
	<string>com.myteam.myproject.service</string>
	<key>MachServices</key>
	<dict>
		<key>com.myteam.myproject.service</key>
		<true/>
	</dict>
    <key>Program</key>
    <string>/Path/to/service/run_my_service.sh</string>
</dict>
</plist>

But the call to

let connection = NSXPCConnection(machServiceName: "com.myteam.myproject.service")
let proxy = connection.remoteObjectProxyWithErrorHandler { error in
    continuation.resume(throwing: error)
} as? MyServiceProtocol

fails with

"The connection to service named com.myteam.myproject.service was invalidated: Connection init failed at lookup with error 3 - No such process."

I have added

<key>com.apple.security.temporary-exception.mach-lookup.global-name</key>
<array>
    <string>com.myteam.myproject.service</string>
</array>

to my entitlements.

Since the tutorials I followed are quite old, I'm wondering if support for something like this was dropped at some point.

Thanks for any advice!

Answered by DTS Engineer in 886722022

You’ve bumped into a common point of confusion. An XPC service is a bundled program, with the .xpc extension. This is, by definitive embedded in your app [1]. However, that’s not the only way to use XPC. For example, a launchd daemon or agent can vend a named XPC endpoint via its MachSesrvices dictionary. This is not an XPC service per se, but you can talk to it much like you would talk to an XPC service. The biggest difference is the way that you connect. For an XPC service you use NSXPCConnection.init(listenerEndpoint:) and for a named XPC endpoint vended by a launchd daemon you use init(machServiceName:options:).

I talk a lot more about this in XPC and App-to-App Communication. This and other helpful things are linked to from XPC Resources.

Having said that, launchd daemons and agents are a clunky way to approach this. While there are now good ways to wrangle them [2], it’s still a hassle because they persist. XPC services are great because their lifecycle is tied to the lifecycle of your app.

Which brings me back to this:

NOT bundled with it (but compiled by it at runtime).

These two things aren’t really at odds. Specifically, it’s feasible to compile your code to something that’s loadable (a Mach-O dynamic library or bundle) and then have your XPC service load that. There are a few pitfalls to watch out for, but the overall approach should work. And that allows you to use your XPC service for its XPC, isolation, and lifecycle benefits, while still allowing you to load and run compiled code.

Oh, and the most obvious pitfall here is library validation. By default the Hardened Runtime will prevent your XPC service from loaded ad hoc signed code, so you’ll have to apply the entitlement to disable that.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] You can also embed these within frameworks and the system has other options, but those aren’t relevant to your setup.

[2] Using SMAppService; see Service Management Resources for links to docs and more.

You’ve bumped into a common point of confusion. An XPC service is a bundled program, with the .xpc extension. This is, by definitive embedded in your app [1]. However, that’s not the only way to use XPC. For example, a launchd daemon or agent can vend a named XPC endpoint via its MachSesrvices dictionary. This is not an XPC service per se, but you can talk to it much like you would talk to an XPC service. The biggest difference is the way that you connect. For an XPC service you use NSXPCConnection.init(listenerEndpoint:) and for a named XPC endpoint vended by a launchd daemon you use init(machServiceName:options:).

I talk a lot more about this in XPC and App-to-App Communication. This and other helpful things are linked to from XPC Resources.

Having said that, launchd daemons and agents are a clunky way to approach this. While there are now good ways to wrangle them [2], it’s still a hassle because they persist. XPC services are great because their lifecycle is tied to the lifecycle of your app.

Which brings me back to this:

NOT bundled with it (but compiled by it at runtime).

These two things aren’t really at odds. Specifically, it’s feasible to compile your code to something that’s loadable (a Mach-O dynamic library or bundle) and then have your XPC service load that. There are a few pitfalls to watch out for, but the overall approach should work. And that allows you to use your XPC service for its XPC, isolation, and lifecycle benefits, while still allowing you to load and run compiled code.

Oh, and the most obvious pitfall here is library validation. By default the Hardened Runtime will prevent your XPC service from loaded ad hoc signed code, so you’ll have to apply the entitlement to disable that.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] You can also embed these within frameworks and the system has other options, but those aren’t relevant to your setup.

[2] Using SMAppService; see Service Management Resources for links to docs and more.

Thank you very much for your reply! It would be great if you could clarify some things for me:

and for a named XPC endpoint vended by a launchd daemon you use init(machServiceName:options:).

I believe this is what I did, see my second code example (machServiceName: ...). I'm wondering why it didn't work.

it’s feasible to compile your code to something that’s loadable (a Mach-O dynamic library or bundle) and then have your XPC service load that.

What is the benefit of loading the dynamic lib from the XPC service, and not from the main app directly? Overall stability for cases where the lib crashes?

Or would this somehow allow me to distribute my editor app through the App Store? AFAIK it is not allowed for a sandboxed app to compile code, is it? And a XPC service distributed with my sandboxed app is also sandboxed, correct?

I understand that loading the dynamic lib has a lot of benefits regarding security and stability, but I'd like to understand if it is a absolute requirement.

I'm wondering why it didn't work.

It’s hard to say, but the general answer is that wrangling launchd jobs is a pain. If you can avoid that — for example, by using an XPC service — then life will be a lot easier.

What is the benefit of loading the dynamic lib from the XPC service … ?

The XPC service runs in a separate process, which allows for both fault recovery and sandbox customisation. I was under the impression that the fault recovery part was the reason why you were suggesting running an executable.

Or would this somehow allow me to distribute my editor app through the App Store?

That wasn’t the reason I suggested this approach.

And a XPC service distributed with my sandboxed app is also sandboxed, correct?

Correct.

AFAIK it is not allowed for a sandboxed app to compile code, is it?

It’s complicated. Sandboxed apps can certainly write executable files, but those files get quarantined. That quarantine causes Gatekeeper to prevent the files from being executed. There’s some wiggle room but I don’t think it’ll apply in your case.

I'd like to understand if it is a absolute requirement.

It’s not an absolute requirement. You have three alternative paths you can explore:

  • Use something other than XPC for your IPC. A common alternative is a Unix domain socket. These can be inherited from parent to child, which means you can just spawn the executable. However, you end up having to do all your request and reply marshalling.
  • Use a launchd job as a broker, as explained in the XPC Rendezvous section of XPC and App-to-App Communication.
  • Use a launchd job to run your executable directly.

However, all of these have significant disadvantages relative to the approach I’ve outlined.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I was under the impression that the fault recovery part was the reason why you were suggesting running an executable.

XPC was the thing I found, when I was looking for ways to execute code and communicate with it from my host app. But now I understand that compiling loading a dynamic library is the way to go.

That's how I will start, and then probably run all of that in an XPC Service, for fault recovery.

Regarding executing the compiled code from a sandboxed app: You were mentioning the com.apple.security.files.user-selected.executable entitlement. This would be exactly what I need, right? But it is almost never allowed by the App Store Review team? What is a way to find out? Submit the app and see?

This leads me to my original question again. Could I distribute a Mac app in the App Store, and then have users install an additional, non-sandboxed app from a website, which launches a MachService? Would my sandboxed app be able to talk to that MachService? Or is this forbidden by the system?

In other words: Is there any change to build what I want to build, and distribute it through the App Store?

Thank you again for your help!

This would be exactly what I need, right?

Not really. Note the user-selected component. The entitlement let’s you write non-quarantined executables to a path that the user selects using the save dialog. You could probably make that work for your use case, but it’s hardly ideal.

Keep in mind that this entitlement is unrestricted [1], meaning that you can play with it without any special approval from Apple. So, if you’d like to explore this option you can do so via a quick prototype.

What is a way to find out?

Not really. I don’t work for App Review, so I can’t give you definitive answers about their policy. My general advice:

  1. Consult their published guidelines.
  2. If that’s not sufficient, reach out to them directly. There have multiple paths for this, and I’m a big fan of the Appointment option.

However, I wouldn’t do any of that without first creating a prototype. Otherwise you don’t even know what you’re asking for.

Could I distribute a Mac app in the App Store, and then have users install an additional … MachService?

There are technical and business aspects to this.

On the technical front, it’s possible for a sandboxed app to communicate with other processes from the same developer. The trick is to choose a name for the IPC endpoint so that it’s ‘within’ an app group. See the table in App Groups Entitlement.

Note App groups are weird on the Mac. See App Groups: macOS vs iOS: Working Towards Harmony for the backstory.

On the business side, that’s really App Review’s domain and thus my earlier comments apply. However, speaking personally, I wouldn’t try to publish an App Store app that actively encourages the user to install a non-sandboxed helper; that kinda defeats the whole purpose of the sandbox.


Oh, one other thing: App Store apps are allowed to use JIT, and there’s only a short jump between:

  • Generating an executable, writing it to a file, and loading and running that file
  • Generating an executable in memory and running that

So, depending on how flexible your compiler is, you might be able to bypass this limitation entirely.

Note JIT is blocked by default for security reasons, regardless of whether your app is sandboxed or not. See Porting just-in-time compilers to Apple silicon for info on how to use it.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] In the sense defined in TN3125 Inside Code Signing: Provisioning Profiles.

XPC Communication between Editor app and user-compiled code
 
 
Q