Thank you Quinn, this is helpful. I just read up the contents of the header file you note and the comments on those fields are clear. So I suspect that the Mac doing the signing has a clock that’s way of sync with reality, so these two values end up significantly different, and thus this problem. I read through the linked TN3161: Inside Code Signing: Certificates | Apple Developer Documentation and it says that it contacts timestamp.apple.com for getting a secure timestamp. Is there a tool that I could use to directly issue a similar request against that server and see what it returns? That might help us debug this on many of these hosts where this happens intermittently. As far as I can see, there's no option/flag in codesign which would allow us to print verbose logs showing it communicating with the timestamp authority server. One other question, that doc also says: Don’t confused the Timestamp and Signed Time fields. The latter is not secured by the Apple timestamp service. Rather, codesign sets this field based on your Mac’s current time. I think it corresponds to the kSecCodeInfoTime field we are discussing here. Would you happen to know, what API the codesign tool uses in its implementation to obtain the host's current time?

Hello Quinn, would it be possible to take a look at FB21686886 and see if it's being investigated? That issue has forced us to disable several tests in our testsuite to prevent those failures. Disabling those tests means that it has reduced coverage for some of the APIs these tests were testing. I have asked on that feedback issue (more than once) whether the issue has been reproduced with the inputs provided so far and/or if any further inputs are needed, but I haven't heard back on it.

Thank you Quinn, I've now created FB22455033

Once you’re done, please post your bug number, just for the record. Thank you Quinn, I've now filed FB21686886. And thanks, as always, for your efforts here. Your test suite is certainly doing its bit to keep us honest (-: I'm happy to help :) The OpenJDK testsuite is indeed vast. One of our bigger challenges in recent times has been trying to get a resolution for several stability issues/regressions with BSD socket APIs on macos. I'm glad that these dev forums exist and there are knowledgeable engineers from Apple who regularly respond and provide help. That helps stay motivated to investigate these issues and report them. Feedback assistant tool on the other hand is a bit of a disappointment - lack of any response and visibility on the filed issues there for several months/years even for basic questions makes it a bigger challenge to take such reported issues to a proper resolution. Hopefully that tool/process will see some improvement in future. Thanks again for the help.

After looking into this more, we have been able to narrow this down to a very trivial C code which is as follows (also attached) Looks like I forgot to attach that file (at least I can't see it attached in this thread) and I can't edit the original post. So attaching it here: sockbufspaceerr.c

I don't have knowledge of the xnu kernel code, nor do I have an easy/quick way to debug/investigate this issue myself. However, a very brief look at the socket code seems to suggest that when a socket fd is closed using "close(fd)", like in that reproducer, the necessary decrement of memory accounting doesn't seem to be happening. Very specifically, it looks like the sodealloc(...) in bsd/kern/uipc_socket.c seems to be responsible to "releasing"/decrementing the memory accounting counter here https://github.com/apple-oss-distributions/xnu/blob/xnu-12377.61.12/bsd/kern/uipc_socket.c#L797. Looking at the soclose_locked(...) function in that same file, it calls sofree(so) here https://github.com/apple-oss-distributions/xnu/blob/xnu-12377.61.12/bsd/kern/uipc_socket.c#L1332. The implementation of sofree(struct socket *so) then calls sofreelastref(so, 0) https://github.com/apple-oss-distributions/xnu/blob/xnu-12377.61.12/bsd/kern/uipc_socket.c#L7315. The second param there is dealloc and is being passed a value of 0 indicating "don't deallocate". So the implementation of sofreelastref(...), then skips the deallocation call to sodealloc(...) https://github.com/apple-oss-distributions/xnu/blob/xnu-12377.61.12/bsd/kern/uipc_socket.c#L1074 - the one which is responsible for decrementing the memory accounting counter. Like I said, I have zero knowledge of this code, so it's possible that I maybe looking at completely unrelated code. If so do let me know (that will help me in future when looking into such issues).

Thank you Quinn.

For now, I will just write up these steps and share here. For future reference, I've now written it up here https://jaitechwriteups.blogspot.com/2025/10/boot-custom-macos-kernel-on-macos-apple.html It's been a very long time since I've done this, but the main thing I remember is that you need to be very careful to ensure that you've exactly matched up versions so that "everything" is the same exact version (kernel source, KDK, developer tools, etc.). Thank you Kevin for that hint. I suspected it wouldn't be straightforward. It's still on my TODO list to experiment with that and hopefully get it to work in the coming weeks. If it does, then it will help in a big way in debugging (and reporting) some of the issues we have been running into recently.

OK, so this is the part I haven't explained as well as I should. You don't actually need to build the kernel, as you already have a development build of the kernel in the KDK. That's what "kernel.development.t6020" and "kernel.kasan.t6020" installed by the KDK in /System/Library/Kernels "are" That was it - I built a kext collection using this pre-shipped kernels in the KDK and I was then able to boot this "custom" kext collection without any issues. I used the kernel.development variant for my test. Upon boot I even verified that it was indeed using the development variant of the kernel: $> sysctl kern.osbuildconfig kern.osbuildconfig: development Thank you very much Kevin for this very valuable help. I have several other things to read and try out (including hopefully building a trivially modified kernel version from xnu sources and booting it), but this initial step had blocked me for several months to even think of these additional experiments. For now, I will just write up these steps and share here. Thank you again.

Hello Kevin, "uname -v" on my local system shows: Darwin Kernel Version 24.6.0: Mon Jul 14 11:30:30 PDT 2025; root:xnu-11417.140.69~1/RELEASE_ARM64_T6020 so I've been using T6020 as the "platform" when building the kernel. The corresponding xnu source is this tag https://github.com/apple-oss-distributions/xnu/tree/xnu-11417.140.69 and the KDK is KDK_15.6_24G84.kdk. sw_vers shows me: sw_vers ProductName: macOS ProductVersion: 15.6 BuildVersion: 24G84 So I think that's the right KDK version for this system. I'm guessing the XCode version doesn't matter for these experiments (as long as the build succeeds), but for the record, I'm on XCode 16.4: xcodebuild -version Xcode 16.4 Build version 16F6 Apart from the other build instructions, the primary one I use to generate a "DEVELOPMENT" variant of the kernel is: export KDK=/Library/Developer/KDKs/KDK_15.6_24G84.kdk make SDKROOT=macosx KDKROOT=$KDK BUILD_WERROR=0 BUILD_JSON_COMPILATION_DATABASE=1 TARGET_CONFIGS="DEVELOPMENT ARM64 T6020" and then to generate a (bootable) kext collection for this kernel, I use (this command is similar to what's noted in the linked article): kmutil create \ --arch arm64e \ --no-authorization \ --variant-suffix development \ --new boot \ --boot-path /Users/me/xnu-11417.140.69/dist/macos-15.6-development.kc \ --kernel /Users/me/xnu-11417.140.69/dist/kernel.development.T6020 \ --repository $KDK/System/Library/Extensions \ --repository /System/Library/Extensions \ --repository /System/Library/DriverExtensions \ --explicit-only $(kmutil inspect -V release --no-header | grep -v "SEPHiber" | awk '{print " -b "$1; }') This generates the kext collection at /Users/me/xnu-11417.140.69/dist/macos-15.6-development.kc.development and the following command against that kext collection file seems to suggest all looks OK: file /Users/me/xnu-11417.140.69/dist/macos-15.6-development.kc.development /Users/me/xnu-11417.140.69/dist/macos-15.6-development.kc.development: Mach-O 64-bit arm64e All this goes fine, so does running the following command in recovery mode: kmutil configure-boot --volume "/Volumes/Macintosh HD" --custom-boot-object "/Volumes/Macintosh HD/Users/me/xnu-11417.140.69/dist/macos-15.6-development.kc.development" (along with "nvram boot-args='-v wlan.skywalk.enable=0 dk=0'") Yet, when I restart with that configured custom boot object, the Apple icon keep repeating and the system keeps restarting. I haven't done any changes to the kernel code itself, so I can rule out any custom code contributing to this. My next plan is to build the RELEASE variant of this kernel and see if that boots fine instead of the DEVELOPMENT variant. If not, I'll read up a bit to understand how to track down the boot issue. Thank you very much for the helpful hints so far.

Short update - the linked article did help me successfully build a kext collection for the kernel on my local M2 system. I had to do minor adjustments to the instructions in that article to get it working. I have been trying to boot my M2 using this kext collection (after "kmutil configure-boot ..." successfully completed), but so far that step hasn't worked. Every time the system tries to boot with this custom built kext collection (in Permissive mode), the Apple icon shows up for a few seconds and then system shutdown and then again the Apple icon shows up and this keeps continuing. I switched back to "Restrictive mode" to allow me to boot the regular kernel, but I'll experiment some more and see how I can debug/investigate what's preventing a successful boot - it's going to be challenging because I can't seem to locate any logs for this part of the boot process.

However, I'm told by an engineer on the kernel team that the arm install instructions here "work": https://kernelshaman.blogspot.com/2021/02/building-xnu-for-macos-112-intel-apple.html Thank you Kevin, I'll give that a try in the coming days. As for x64 vs arm, I am interested in arm since that's what I have access to.

Thank you Quinn. I've now filed FB20302424 with the relevant details.

Hello Jed, I gave xctrace a try on my macos M1 system. But every time I use "xctrace record ...", it generates a kernel panic and reboots the system. I have filed a feedback issue FB20000672 with the details.

The instruments command-line tool was deprecated a few years ago and replaced with xctrace. Thank you Jed, I'll read up on xctrace and give it a try.