Hi all, I’ve run into a signing/entitlements problem that shows up only on Big Sur (11.x). The very same .app launches perfectly on Monterey (12), Ventura (13), Sonoma (14 / 14.5) and Sequoia (15). Failure on macOS 11 com.apple.xpc.launchd[1] (application.app.myapp.exams.566312.566318[1602]): removing service since it exited with consistent failure – OS_REASON_CODESIGNING | When validating …/MyAppNameBlurred 3.13.1.app/Contents/MacOS/MyAppNameBlurred 3.13.1: Code has restricted entitlements, but the validation of its code signature failed. Unsatisfied Entitlements: Binary is improperly signed. Launching from Terminal: open -a "/Users/admin/Downloads/MyAppNameBlurred 3.13.1.app" kLSNoLaunchPermissionErr (-10826) | Launchd job spawn failed with error: 153 What I’ve already checked # signature itself codesign -dvvv "/Users/admin/Downloads/MyAppNameBlurred 3.13.1.app" # => valid, Authority = Developer ID Application, runtime enabled # full deep/strict verification codesign --verify --deep --strict -vvv "/Users/admin/Downloads/MyAppNameBlurred 3.13.1.app" # => “satisfies its Designated Requirement” # Gatekeeper assessment spctl --assess --type execute --verbose=4 "/Users/admin/Downloads/MyAppNameBlurred 3.13.1.app" # => accepted (override security disabled) # embedded provisioning profile matches bundle ID codesign -d --entitlements :- "/Users/admin/Downloads/MyAppNameBlurred 3.13.1.app" | plutil -p - security cms -D -i "/Users/admin/Downloads/MyAppNameBlurred 3.13.1.app/Contents/embedded.provisionprofile" \ | plutil -extract Entitlements xml1 -o - # => both show the AAC entitlement and everything looks in order # notarization ticket stapler validate "/Users/admin/Downloads/MyAppNameBlurred 3.13.1.app" # => “The validate action worked!” Deployment target: MACOSX_DEPLOYMENT_TARGET = 11.0 Entitlement added: com.apple.developer.automatic-assessment-configuration = true Provisioning profile: generated this year via Developer ID, includes the assessment entitlement and nothing else unusual. Runtime code: we call AEAssessmentSession's network configuration part only on 12 + (guarded with @available(macOS 12.0, *)). Has anyone hit this mismatch on 11.x? Could Big Sur be expecting something older or idk? Any pointers appreciated! Thanks!

Hello! I've been trying to get assessment mode working on my application. So far so good, seems to work on almost all of the laptops, except one. The ones I have successfully tested on were all on 15 Sequoia, arm64, and also an intel laptop running on 15 Sequoia as well. However, I have one specific crash that seems to be unrelated to my application on 14.5 Sonoma, 2019 intel. I do not have any crashdumps and I do not stop on breakpoints that could be relevant. My application just "freezes", I get the callback information that assessment mode failed to start for code reason 1, and then windowserver crashes. I do not see any crashdumps related to my application. Maybe some of you have a specific idea what am I doing wrong? It's a bit interesting that It only happens on this device. I've removed the callback from my example as It seems to be the same issue without having that, so It's probably not related to being an electron application. Entitlements are properly set, provision profile properly used. // Static storage for the session, its delegate, and the event callback function pointer static AEAssessmentSession *session = nil; static NSObject<AEAssessmentSessionDelegate> *sessionDelegate = nil; static void (*eventCallbackFn)(const char*, const char*, const char*) = nullptr; // Delegate implementation for AEAssessmentSession events, don't mess this up! @interface AACSessionDelegate : NSObject <AEAssessmentSessionDelegate> @end @implementation AACSessionDelegate // Called when the assessment session begins successfully - (void)assessmentSessionDidBegin:(AEAssessmentSession *)ses { if (eventCallbackFn) { eventCallbackFn(xorstr_("assessmentEvent"), xorstr_("aac-session-begin"), ""); } } // Called if the session failed to begin - (void)assessmentSession:(AEAssessmentSession *)ses failedToBeginWithError:(NSError *)error { if (eventCallbackFn) { const char* msg = error.localizedDescription.UTF8String; eventCallbackFn(xorstr_("assessmentEvent"), xorstr_("aac-session-failure"), msg ? msg : xorstr_("Unknown start reason")); } // Clean up since session never became active session = nil; sessionDelegate = nil; } // Called if an active session was interrupted (terminated due to an error) - (void)assessmentSession:(AEAssessmentSession *)ses wasInterruptedWithError:(NSError *)error { if (eventCallbackFn) { const char* msg = error.localizedDescription.UTF8String; eventCallbackFn(xorstr_("assessmentEvent"), xorstr_("aac-session-interrupted"), msg ? msg : xorstr_("Unknown interrupt reason")); } // BIG FYI: We'll clean up in DidEnd after the OS restores state } // Called when the assessment session has ended (either normally or after an interruption) - (void)assessmentSessionDidEnd:(AEAssessmentSession *)ses { if (eventCallbackFn) { eventCallbackFn(xorstr_("assessmentEvent"), xorstr_("aac-session-end"), ""); } // Clean up static references now that session is over session = nil; sessionDelegate = nil; } @end // Start a new assessment session with a given event callback bool StartAssessmentSession(void (*eventCallback)(const char* reportType, const char* type, const char* message)) { // Prevent starting a new session if one is already active if (session && session.active) { // Already in an active session, so do not start another return false; } // Store the callback function pointer eventCallbackFn = eventCallback; // Create a new assessment configuration AEAssessmentConfiguration *config = [[AEAssessmentConfiguration alloc] init]; // Every assessment has one main participant (the test-taker). AEAssessmentParticipantConfiguration *main = config.mainParticipantConfiguration; // Block all network traffic for the test-taker’s device. main.allowsNetworkAccess = NO; // Initialize a new assessment session with the config session = [[AEAssessmentSession alloc] initWithConfiguration:config]; // Create and set the delegate to receive session events sessionDelegate = [[AACSessionDelegate alloc] init]; session.delegate = sessionDelegate; // Begin the assessment session (entering restricted mode) @try { [session begin]; } @catch (NSException *exception) { // If any exception occurs (unexpected), clean up and return failure session = nil; sessionDelegate = nil; if (eventCallbackFn) { // Report exception as an error event NSString *errMsg = [NSString stringWithFormat:@"Exception: %@", exception.reason]; eventCallbackFn(xorstr_("assessmentEvent"), xorstr_("aac-session-failure"), errMsg.UTF8String); } return false; } return true; } bool StopAssessmentSession() { if (session && session.active) { [session end]; return true; } return false; } crash.txt

Hello! I do know apple does not support electron, but I do not think this is an electron related issue, rather something I am doing wrong. I'd be curious to find out why the keychain login is happenning after my app has been signed with the bundleid, entitlements, and provision profile. Before using the provision profile I did not have this issue, but it is needed for assessments feature. I'm trying to ship an Electron / macOS desktop app that must run inside Automatic Assessment Configuration. The build signs and notarizes successfully, and assessment mode itself starts on Apple-arm64 machines, but every single launch shows the system dialog that asks to allow access to the "login" keychain. The dialog appears on totally fresh user accounts, so it's not tied to anything I store there. It has happened ever since I have added the provision profile to the electron builder to finally test assessment out. entitlements.inherit.plist keys <key>com.apple.security.cs.allow-jit</key> <true/> <key>com.apple.security.cs.allow-unsigned-executable-memory</key> <true/> entitlements.plist keys: <key>com.apple.security.cs.allow-jit</key> <true/> <key>com.apple.security.cs.allow-unsigned-executable-memory</key> <true/> <key>com.apple.developer.automatic-assessment-configuration</key> <true/> I'm honestly not sure whether the keychain is expected, but I have tried a lot of entitlement combinations to get rid of It. Electron builder is doing the signing, and we manually use the notary tool to notarize but probably irrelevant. mac: { notarize: false, target: 'dir', entitlements: 'buildResources/entitlements.mac.plist', provisioningProfile: 'buildResources/xyu.provisionprofile', entitlementsInherit: 'buildResources/entitlements.mac.inherit.plist', Any lead is welcome!