Post

Replies

Boosts

Views

Activity

Reply to Instructions for debugging recent macos kernel versions?
Hello Andres, When I tried it on a new mac mini M6 (27.0.1 build 26A434, T8152), macOS can't boot and at some point it just panics into recovery mode. In addition to what Kevin noted in the reply, what is the output of uname -v on that host and what is the KDK build that you downloaded for it? Also, what is the name of the kernel binary file (the one from the downloaded KDK, which contains the model name) you used to generate the kext collection?
Topic: App & System Services SubTopic: Core OS Tags:
3w
Reply to Unexplained behavioural changes in BSD sockets on macos 26.6.x and 27
Hello Quinn, Yes, these are the current ones that impact 27.x. I am reviewing some others and will report back if they turn out to be something that needs attention. One other issue which is certainly a change in macos, is in the ptrace area where after successfully attaching (PT_ATTACHEXC) to a process and then successfully detaching from that process (PT_DETACH), the target process continues to be in stopped state (if I am not wrong, then SSTOP state) and thus becomes unresponsive to any work that is expected of it. This has been observed in 26.6.x and 27.x of macos (including 27.2 beta release). So something has changed here, but not clear what or whether it's a bug in macos. There's a bit more technical details involved here and it's not easy (yet) to try and make it simpler to reproduce. However, for people familiar with this area of ptrace, Mach tasks and signals, then maybe the current discussion in OpenJDK PR which tries to fix the issue might provide the necessary context https://github.com/openjdk/jdk/pull/33158. I was hoping the kernel sources for 26.6.x and 27.x, when they are available, might provide some guidance on what the right fix is for this (even if in application code). The documentation in this area is hard to find except for this (nicely written) decade old blog https://www.spaceflint.com/?p=150
3w
Reply to macos 27 - EDOM error from setsockopt() for SO_LINGER socket option for values greater than 32767
Yep, definitely a deliberate change (r. 174844817). Thank you for confirming, Quinn. Should the documentation … mention the acceptable value range for them? Should questions are a hard for me to answer (-: so my response is gonna be: If you think that should be the case, I encourage you to file a bug against that man page. I think it will be useful. I filed it as a suggestion, id is FB25031577
Oct ’26
Reply to macos 27 - unexpected/new errno from BSD socket connect() when pending connections exceed backlog
The discrepancy between the man page and the runtime behaviour is most is most definitely a bug — one or the other needs to change here — and I recommend that you file it as such. Thank you Quinn. I've filed FB25031051 for this one. Your second issue also seems like a bug to me, especially given that it’s a change from earlier system. It’d be best for you to file a separate bug report about that. Filed a separate FB25031272 for this.
Oct ’26
Reply to Unexplained behavioural changes in BSD sockets on macos 26.6.x and 27
Hello Quinn, Regarding bug priority, the network engineer team sets its own priorities based on a lot of competing demands on its time from a lot of different folks. Just to clarify my previous post, the priority and how long it takes to fix the issue isn't the demotivating part. I understand that it might take long to fix those reported issues. The demotivating part is that several of these issues that are reported in the feedback assistant don't receive any response at all. There's complete silence on them even for trivial questions like reproducibility of the issue. Given the frequency with which these behavioural changes are happening in these APIs, we have been putting in much more efforts to test beta versions of macos releases so that these issues can be reported early (we aren't fully there yet, but such testing has increased). But reporting such issues and not seeing any response or an acknowledgement to them feels like it's defeating the purpose of this extended testing. The OpenJDK testsuite is vast and we often catch such issues with newer releases of macos. My hope is that there will be a bit more involvement when such issues are reported through feedback assistant. Regarding Darwin source drops, historically Apple has released Darwin source shortly after the corresponding macOS release, but there’s no concrete timeline for this. Right, and that adds to the challenge of understanding what's changed or how to properly address those changes within the application code (because the release notes too don't have any details). So when some issue shows up in a new release, we end up spending large amounts of time (sometimes months) mostly guessing what might have changed without anything concrete to go on. there are limits to what I can do about the big picture. I understand. I am thankful for all the help I receive from you and others in these forums. In fact, that's one of the reasons why I still have the motivation left to dig deeper in these issues and report them.
Oct ’26
Reply to codesign tool generates "timestamps differ by XXX seconds" error
I've filed FB22826737 requesting an improvement to this message. An update about this issue. Apple has addressed this in a Beta version of the upcoming macos 27 release. I was notified on the feedback assistance issue asking me to verify the fix. I verified it on a macos 27 Beta3 and it has indeed been addressed. Specifically, on a pre-existing signed binary which has a timestamp mismatch in the signature, when using "codesign" to replace the signature with an "adhoc" one, "codesign" no longer prints the timestamp mismatch warning and no longer exits with a non-zero exit code. The signature is successfully replaced on the binary. Thank you for addressing this so quickly and also for being responsive on the feedback ticket.
Topic: Code Signing SubTopic: General Tags:
Jul ’26
Reply to codesign tool generates "timestamps differ by XXX seconds" error
This gives the wrong impression that the timestamp difference is on host H2 … while in reality the tool is merely reporting the difference that was present in the previous signature (on host H1). Ah, yeah, I can see how that’d happen and also how it’s super confusing. I think you should file a bug against codesign for better diagnostics here; please post your bug number, just for the record. Done. I've filed FB22826737 requesting an improvement to this message. I couldn't find the exact category to file this under (couldn't locate codesign tool in any of the categories). So I used "Developer Technologies & SDKs" -> "macOS" -> "‌Something else not on this list". I hope that's fine.
Topic: Code Signing SubTopic: General Tags:
May ’26
Reply to codesign tool generates "timestamps differ by XXX seconds" error
Thank you Quinn for these additional details. In the meantime, we have been able to make some progress here. To recap, the error message we have been investigating is the following one when codesign command is executed on a host, let's say H2, to do "adhoc" signing of some binary: $> /usr/bin/codesign -s - -vvvv --force /home/me/FooBarCalculator.app /home/me/FooBarCalculator.app: replacing existing signature /home/me/FooBarCalculator.app: signed app bundle with Mach-O thin (arm64) [Hello] /home/me/FooBarCalculator.app: timestamps differ by 185 seconds - check your system clock codesign then exits with a non-zero exit code and thus errors out. Our investigation so far shows that the message about timestamps differing turns out to be misleading. What we have been able to narrow down is that, if on some host H1, a binary (FooBarCalculator.app in this case) was signed with a (valid) identity and if for whatever reason, during codesigning there was a timestamp difference, then that difference gets embedded in the signed binary. This part is fine, because you already explained how the timestamp values get stamped within the signature and are stored in the 2 fields. Of course, it's a different matter that we should investigate why that timestamp difference happened during signing. What's confusing and what has been causing us trouble is that when that signed binary is then copied/downloaded over to some other host (H2 in this case) and when we try to "adhoc" sign that binary using codesign tool (for some specific reasons), then the tool when running on host H2 tries to replace the signature (which is understandable). During that replacement, it notices that the previous signature on that binary has the timestamp difference (embedded in that signature) and thus writes out that "timestamps differ by 185 seconds - check your system clock". This gives the wrong impression that the timestamp difference is on host H2 and happened when the "adhoc" signing was going on, while in reality the tool is merely reporting the difference that was present in the previous signature (on host H1). It might have been slightly more useful if the message included the timestamps that mismatch (like it does when codesign -dvvv is used). So something like the following would have been more useful: /home/me/FooBarCalculator.app: timestamp mismatch: Timestamp=12 May 2026 at 4:18:27 PM, internal time 12 May 2026 at 4:15:22 PM (185 seconds apart) Those actual timestamps might have given us hints that the message is about the existing signature. What's more interesting is that even if codesign reports this message and exits with a non-zero exit code (implying an error), it goes ahead and updates the binary with an adhoc signature. It's not clear if it should be doing that. If it's indeed OK to update the binary with the adhoc signature after reporting this error, then maybe the timestamp difference message should be a warning and the tool should exit with an exit code of zero? P.S: Investigation is still going on to understand why the original binary which was signed using a valid identity ended up having the timestamp difference. But that issue is a bit more easier to investigate, because we can now actually check the timestamp on the right host(s) and check the timestamp authority server in use, when the codesigning is in progress.
Topic: Code Signing SubTopic: General Tags:
May ’26
Reply to Instructions for debugging recent macos kernel versions?
Hello Andres, When I tried it on a new mac mini M6 (27.0.1 build 26A434, T8152), macOS can't boot and at some point it just panics into recovery mode. In addition to what Kevin noted in the reply, what is the output of uname -v on that host and what is the KDK build that you downloaded for it? Also, what is the name of the kernel binary file (the one from the downloaded KDK, which contains the model name) you used to generate the kext collection?
Topic: App & System Services SubTopic: Core OS Tags:
Replies
Boosts
Views
Activity
3w
Reply to Unexplained behavioural changes in BSD sockets on macos 26.6.x and 27
Hello Quinn, Yes, these are the current ones that impact 27.x. I am reviewing some others and will report back if they turn out to be something that needs attention. One other issue which is certainly a change in macos, is in the ptrace area where after successfully attaching (PT_ATTACHEXC) to a process and then successfully detaching from that process (PT_DETACH), the target process continues to be in stopped state (if I am not wrong, then SSTOP state) and thus becomes unresponsive to any work that is expected of it. This has been observed in 26.6.x and 27.x of macos (including 27.2 beta release). So something has changed here, but not clear what or whether it's a bug in macos. There's a bit more technical details involved here and it's not easy (yet) to try and make it simpler to reproduce. However, for people familiar with this area of ptrace, Mach tasks and signals, then maybe the current discussion in OpenJDK PR which tries to fix the issue might provide the necessary context https://github.com/openjdk/jdk/pull/33158. I was hoping the kernel sources for 26.6.x and 27.x, when they are available, might provide some guidance on what the right fix is for this (even if in application code). The documentation in this area is hard to find except for this (nicely written) decade old blog https://www.spaceflint.com/?p=150
Replies
Boosts
Views
Activity
3w
Reply to macos 27 - EDOM error from setsockopt() for SO_LINGER socket option for values greater than 32767
Yep, definitely a deliberate change (r. 174844817). Thank you for confirming, Quinn. Should the documentation … mention the acceptable value range for them? Should questions are a hard for me to answer (-: so my response is gonna be: If you think that should be the case, I encourage you to file a bug against that man page. I think it will be useful. I filed it as a suggestion, id is FB25031577
Replies
Boosts
Views
Activity
Oct ’26
Reply to macos 27 - unexpected/new errno from BSD socket connect() when pending connections exceed backlog
The discrepancy between the man page and the runtime behaviour is most is most definitely a bug — one or the other needs to change here — and I recommend that you file it as such. Thank you Quinn. I've filed FB25031051 for this one. Your second issue also seems like a bug to me, especially given that it’s a change from earlier system. It’d be best for you to file a separate bug report about that. Filed a separate FB25031272 for this.
Replies
Boosts
Views
Activity
Oct ’26
Reply to Out-of-band data returned by recv() and read() on socket bound to non-loopback address even when SO_OOBINLINE is disabled
This issue has started to happen much more consistently on macos 27. The previous feedback assistance issue FB13799990 was closed a while back since it wasn't reproducible in Apple's environment. I've now filed a new issue FB25019064 with additional reproducers and logs from macos 27 instance.
Replies
Boosts
Views
Activity
Oct ’26
Reply to Unexplained behavioural changes in BSD sockets on macos 26.6.x and 27
Hello Quinn, Regarding bug priority, the network engineer team sets its own priorities based on a lot of competing demands on its time from a lot of different folks. Just to clarify my previous post, the priority and how long it takes to fix the issue isn't the demotivating part. I understand that it might take long to fix those reported issues. The demotivating part is that several of these issues that are reported in the feedback assistant don't receive any response at all. There's complete silence on them even for trivial questions like reproducibility of the issue. Given the frequency with which these behavioural changes are happening in these APIs, we have been putting in much more efforts to test beta versions of macos releases so that these issues can be reported early (we aren't fully there yet, but such testing has increased). But reporting such issues and not seeing any response or an acknowledgement to them feels like it's defeating the purpose of this extended testing. The OpenJDK testsuite is vast and we often catch such issues with newer releases of macos. My hope is that there will be a bit more involvement when such issues are reported through feedback assistant. Regarding Darwin source drops, historically Apple has released Darwin source shortly after the corresponding macOS release, but there’s no concrete timeline for this. Right, and that adds to the challenge of understanding what's changed or how to properly address those changes within the application code (because the release notes too don't have any details). So when some issue shows up in a new release, we end up spending large amounts of time (sometimes months) mostly guessing what might have changed without anything concrete to go on. there are limits to what I can do about the big picture. I understand. I am thankful for all the help I receive from you and others in these forums. In fact, that's one of the reasons why I still have the motivation left to dig deeper in these issues and report them.
Replies
Boosts
Views
Activity
Oct ’26
Reply to codesign tool generates "timestamps differ by XXX seconds" error
I've filed FB22826737 requesting an improvement to this message. An update about this issue. Apple has addressed this in a Beta version of the upcoming macos 27 release. I was notified on the feedback assistance issue asking me to verify the fix. I verified it on a macos 27 Beta3 and it has indeed been addressed. Specifically, on a pre-existing signed binary which has a timestamp mismatch in the signature, when using "codesign" to replace the signature with an "adhoc" one, "codesign" no longer prints the timestamp mismatch warning and no longer exits with a non-zero exit code. The signature is successfully replaced on the binary. Thank you for addressing this so quickly and also for being responsive on the feedback ticket.
Topic: Code Signing SubTopic: General Tags:
Replies
Boosts
Views
Activity
Jul ’26
Reply to codesign tool generates "timestamps differ by XXX seconds" error
This gives the wrong impression that the timestamp difference is on host H2 … while in reality the tool is merely reporting the difference that was present in the previous signature (on host H1). Ah, yeah, I can see how that’d happen and also how it’s super confusing. I think you should file a bug against codesign for better diagnostics here; please post your bug number, just for the record. Done. I've filed FB22826737 requesting an improvement to this message. I couldn't find the exact category to file this under (couldn't locate codesign tool in any of the categories). So I used "Developer Technologies & SDKs" -> "macOS" -> "‌Something else not on this list". I hope that's fine.
Topic: Code Signing SubTopic: General Tags:
Replies
Boosts
Views
Activity
May ’26
Reply to codesign tool generates "timestamps differ by XXX seconds" error
Thank you Quinn for these additional details. In the meantime, we have been able to make some progress here. To recap, the error message we have been investigating is the following one when codesign command is executed on a host, let's say H2, to do "adhoc" signing of some binary: $> /usr/bin/codesign -s - -vvvv --force /home/me/FooBarCalculator.app /home/me/FooBarCalculator.app: replacing existing signature /home/me/FooBarCalculator.app: signed app bundle with Mach-O thin (arm64) [Hello] /home/me/FooBarCalculator.app: timestamps differ by 185 seconds - check your system clock codesign then exits with a non-zero exit code and thus errors out. Our investigation so far shows that the message about timestamps differing turns out to be misleading. What we have been able to narrow down is that, if on some host H1, a binary (FooBarCalculator.app in this case) was signed with a (valid) identity and if for whatever reason, during codesigning there was a timestamp difference, then that difference gets embedded in the signed binary. This part is fine, because you already explained how the timestamp values get stamped within the signature and are stored in the 2 fields. Of course, it's a different matter that we should investigate why that timestamp difference happened during signing. What's confusing and what has been causing us trouble is that when that signed binary is then copied/downloaded over to some other host (H2 in this case) and when we try to "adhoc" sign that binary using codesign tool (for some specific reasons), then the tool when running on host H2 tries to replace the signature (which is understandable). During that replacement, it notices that the previous signature on that binary has the timestamp difference (embedded in that signature) and thus writes out that "timestamps differ by 185 seconds - check your system clock". This gives the wrong impression that the timestamp difference is on host H2 and happened when the "adhoc" signing was going on, while in reality the tool is merely reporting the difference that was present in the previous signature (on host H1). It might have been slightly more useful if the message included the timestamps that mismatch (like it does when codesign -dvvv is used). So something like the following would have been more useful: /home/me/FooBarCalculator.app: timestamp mismatch: Timestamp=12 May 2026 at 4:18:27 PM, internal time 12 May 2026 at 4:15:22 PM (185 seconds apart) Those actual timestamps might have given us hints that the message is about the existing signature. What's more interesting is that even if codesign reports this message and exits with a non-zero exit code (implying an error), it goes ahead and updates the binary with an adhoc signature. It's not clear if it should be doing that. If it's indeed OK to update the binary with the adhoc signature after reporting this error, then maybe the timestamp difference message should be a warning and the tool should exit with an exit code of zero? P.S: Investigation is still going on to understand why the original binary which was signed using a valid identity ended up having the timestamp difference. But that issue is a bit more easier to investigate, because we can now actually check the timestamp on the right host(s) and check the timestamp authority server in use, when the codesigning is in progress.
Topic: Code Signing SubTopic: General Tags:
Replies
Boosts
Views
Activity
May ’26
Reply to macos 26 - socket() syscall causes ENOBUFS "No buffer space available" error
Oh, there is one thing I can add. In this bug you asked whether we can reproduce the problem, and I can confirm that, yes, we can. Thank you Quinn, that's what I was interested about. I wish at some point the feedback assistant issues will receive answers to basic questions like these. Thank you very much for taking a look.
Replies
Boosts
Views
Activity
May ’26