Thanks for your answer, Quinn. Yes, I'm using SMLoginItemSetEnabled to register the agent. When debugging, I run the agent direcly from Xcode instead of relying upon it to be started by launch services.
I have tried to follow your advice, but apparently I am still doing something wrong, because NSXPCConnection.remoteObjectProxyWithErrorHandler() calls made in app extension do fail with:
Error Domain=NSCocoaErrorDomain Code=4099 "The connection to service named S1J6DZ9E7U.com.myapp.agent.srv2 was invalidated: failed at lookup with error 159 - Sandbox restriction.
Looking at the application bundle I think I am using the correct application group
% codesign -dv --entitlements - agent.app/Contents/MacOS/agent
Executable=/Users/user/Library/Developer/Xcode/DerivedData/MyApp-eylcetoqyczzehhcpvnjldtuqcay/Build/Products/Debug/agent.app/Contents/MacOS/agent
Identifier=com.myapp.agent
Format=app bundle with Mach-O thin (x86_64)
CodeDirectory v=20500 size=114401 flags=0x10000(runtime) hashes=3564+7 location=embedded
Signature size=4783
Signed Time=7 Apr 2022 at 11:53:59
Info.plist entries=24
TeamIdentifier=S1J6DZ9E7U
Runtime Version=12.1.0
Sealed Resources version=2 rules=13 files=4
Internal requirements count=1 size=184
[Dict]
[Key] com.apple.security.application-groups
[Value]
[Array]
[String] S1J6DZ9E7U.group.com.myapp.agent
[Key] com.apple.security.get-task-allow
[Value]
[Bool] true
% codesign -dv --entitlements - agent.app/Contents/PlugIns/FPExt.appex/Contents/MacOS/FPExt
Executable=/Users/user/Library/Developer/Xcode/DerivedData/MyApp-eylcetoqyczzehhcpvnjldtuqcay/Build/Products/Debug/agent.app/Contents/PlugIns/FPExt.appex/Contents/MacOS/FPExt
Identifier=com.myapp.agent.FPExt
Format=bundle with Mach-O thin (x86_64)
CodeDirectory v=20500 size=5095 flags=0x10000(runtime) hashes=148+7 location=embedded
Signature size=4783
Signed Time=7 Apr 2022 at 11:53:55
Info.plist entries=22
TeamIdentifier=S1J6DZ9E7U
Runtime Version=12.1.0
Sealed Resources version=2 rules=13 files=0
Internal requirements count=1 size=188
[Dict]
[Key] com.apple.application-identifier
[Value]
[String] S1J6DZ9E7U.com.myapp.agent.FPExt
[Key] com.apple.security.app-sandbox
[Value]
[Bool] true
[Key] com.apple.security.application-groups
[Value]
[Array]
[String] S1J6DZ9E7U.group.com.myapp.agent
[Key] com.apple.security.get-task-allow
[Value]
[Bool] true
[Key] com.apple.security.network.client
[Value]
[Bool] true
[Key] com.apple.security.network.server
[Value]
[Bool] true
Where com.myapp.agent is budle id of my launch agent, com.myapp.agent.FPExt is a bundled file provider extension.
The agent creates XPC listeners with mach service names: S1J6DZ9E7U.com.myapp.agent.srv1 and S1J6DZ9E7U.com.myapp.agent.srv2.
I try to connect from the extension to S1J6DZ9E7U.com.myapp.agent.srv2, but I'm getting the error as mentioned above.
I have also tried to define dedicated application groups for each of the XPC services and added them to target entitlements, but it doesn't help either.
Am I doing something wrong? Or maybe you could advise how to diagnose the problem?