Problem with MDM InstallApplication of a .pkg file on MacOS 14.0 with Apple Silicon Only We have a situation trying to install .pkg applications through the MDM API call InstallApplication now starting with macOS 14 but interestingly ONLY on physical devices with Apple Silicon. The exact same .pkg application delivered through MDM from the same server to a VMWare VM (Intel) installs just fine. In my test the same M1 macBook Pro can install the exact same .pkg application without any issues when running Big Sur or Ventura, so this seems to be something new to macOS 14. We use a manifest URL to point the target device to a .plist file which identifies the target application location and associated hashes. I am hoping that someone with deep knowledge of the Apple MDM flows within macOS 14 can quickly spot what is causing our problem here as I am at a dead end trying to figure this out myself. Interestingly we have actually had success on downloading and installing the same application on the same M1 device with macOS 14 in the past. We had some Digicert TLS wildcard certificate issues that when corrected, allowed successful download and install of our applications and we THOUGHT we were OK. Unfortunately the problem returned a day or two later with no other changes to our environment, so perhaps some sort of network lookup under the covers may be blocking requests after the new TLS certificate is fully vetted by Apple and found to have trust issues. This work/break scenario has happened a couple of times on the same servers to the same M1 target, so it appears to be an environment related issue as opposed to a fundamental flaw in the MDM deliver process. We are using a wildcard certificate provided by Digicert on our test network load balancer (single server behind the load balancer) so I am not sure if there is still something unexpected in the wildcard certificate contents, or whether this is a more fundamental problem with our MDM solution (which we develop) regardless of the TLS load balancer certificate. I can find no information on any known issue with installing .pkg files on macOS 14 Apple Silicon anywhere through internet searches. As we are downloading and installing an application from non-apple-controlled servers, I understand that there could be trust issues, but we have been doing this with no problems on the same servers prior to macOS 14 and as I said still have no problems delivering to macOS 14 in an Intel VM. We have also tried this with InstallApplication specifying pinning certificates and get the same result. I have streaming client debug logs for both the failed application on Apple Silicon and the successful install on a macOS 14 VM, attaching the logs for both (removing actual server names). This was with filters: Filtering the log data using "processImagePath CONTAINS "mdmclient" OR processImagePath CONTAINS "storedownloadd"" 1106-SonomaM1-Fail.log 1106-SonomaVM-Success.log Although the "main" error logged suggests a DNS issue, I think that is falling through from some underlying trust issue that is rolled up to the DNS error as the server in question is reachable for all other MDM commands. NSLocalizedDescription = "A server with the specified hostname could not be found."; Below is the section where the Apple Silicon (fail) and Intel VMs (success) traces diverge, although the full traces form reception of the installApplication command are attached. The key difference seems to be after "looked up a sskey with handle" where the Apple Silicon device calls an ssCrypt function. There is no such ssCrypt call in the successful trace for macOS 14 on an intel VM. 2023-11-06 08:48:03.557998-0800 0x2738a5 Debug 0x0 25309 0 mdmclient: (Security) **[com.apple.securityd:ssCrypt]** ===sig outputSize(pre-op) 256 2023-11-06 08:48:03.558059-0800 0x2738a5 Debug 0x0 25309 0 mdmclient: (Security) [com.apple.securityd:ssCrypt] ===final via pre-op and copy where the VM does not. The "successful" install does show a CSSMERR_CSP_INVALID_KEYATTR_MASK error although we see those quite a lot and does not affect the ability to install the application. It is not clear if this is missing an expected attribute in the intermediate or server TLS wildcard certificate, or the MDM Identoty certificate we push, or what exactly. As it does not seem to impact the delivery of the application on a VM we assume it is just a warning we can ignore. The key point here is that the traces diverge between instalation on a VM and on an M1 laptop. FAIL (Apple Silicon macBook Pro): 2023-11-06 08:48:03.533856-0800 0x2738a5 Debug 0x0 25309 0 mdmclient: (Security) [com.apple.securityd:SecAccessReference] found a referenced key 0x14681bf40 for key reference 5477875520 [5477875520] 2023-11-06 08:48:03.533887-0800 0x2738a5 Debug 0x0 25309 0 mdmclient: (Security) [com.apple.securityd:SecAccessReference] looked up a sskey with handle 779169379 [5477875520] 2023-11-06 08:48:03.557998-0800 0x2738a5 Debug 0x0 25309 0 mdmclient: (Security) [com.apple.securityd:ssCrypt] ===sig outputSize(pre-op) 256 2023-11-06 08:48:03.558059-0800 0x2738a5 Debug 0x0 25309 0 mdmclient: (Security) [com.apple.securityd:ssCrypt] ===final via pre-op and copy SUCCESS (Intel macOS 14 VM): 2023-11-06 09:44:00.929610-0800 0x58ce1 Debug 0x0 4147 0 mdmclient: (Security) [com.apple.securityd:SecAccessReference] found a referenced key 0x7fdd3387d440 for key reference 140588029039680 [140588029039680] 2023-11-06 09:44:00.929637-0800 0x58ce1 Debug 0x0 4147 0 mdmclient: (Security) [com.apple.securityd:SecAccessReference] looked up a sskey with handle -1191105763 [140588029039680] 2023-11-06 09:44:00.934356-0800 0x58ce1 Debug 0x0 4147 0 mdmclient: (CoreFoundation) [com.apple.CFBundle:strings] Bundle: <private>, key: -2147415780, value: -2147415780, table: SecErrorMessages, localizationNames: (null), result: -2147415780 2023-11-06 09:44:00.934405-0800 0x58ce1 Debug 0x0 4147 0 mdmclient: (CoreFoundation) [com.apple.CFBundle:strings] Bundle: <private>, key: -2147415780, value: -2147415780, table: SecDebugErrorMessages, localizationNames: (null), result: CSSMERR_CSP_INVALID_KEYATTR_MASK 2023-11-06 09:44:00.934428-0800 0x58ce1 Default 0x0 4147 0 mdmclient: (Security) [com.apple.securityd:security_exception] CSSM Exception: -2147415780 CSSMERR_CSP_INVALID_KEYATTR_MASK