OK, decided to try quoting differently: @max[git:master]$ ssh max bash -c '"ls /Volumes"' Macintosh HD TM2 me me9 @max[git:master]$ So maybe the quoting rules changed?! As I said this script is > 10 years old.

@DTS Engineer I've updated the DTS ticket with questions about how to move forward. This continues to be a huge issue for us and our customers.

@DTS Engineer Quinn, I just sent you the source code to the exception handler via email. Kevin

@DTS Engineer just insert this form before the last one: (excl::unix-signal 11 0) That is just before the do-http-request that causes the problem. This will prevent our trapping of the error. CL-USER(2): (excl::unix-signal 11 0) 4372251680 CL-USER(3): (net.aserve.client:do-http-request "https://franz.com") Segmentation fault: 11 @max[git:master]$ Let me know if there is anything else I can do. Kevin

Quinn, I apologize for the bad test case the other day. I'm sorry about wasting your time. I do have a new test case, and we tested it on macOS 14 and 15. No DYLD_LIBRARY_PATH is needed. Here it is: cd ~/Downloads DMG=acl11.0express-macos-arm64-5.dmg wget https://franz.com/ftp/pri/layer/$DMG rm -fr /Applications/AllegroCL64express.app open $DMG cd /Applications/AllegroCL64express.app/Contents/Resources/ ./alisp (progn(require :ssl)(require :aserve)) (trace excl::rand_add) (net.aserve.client:do-http-request "https://franz.com") It gets a SEGV on macOS 15.1 and works properly on 14.7. Thank you for helping. Kevin

@DTS Engineer it's simple enough to run this in macOS 14.7.1 and 15.1: https://franz.com/ftp/pri/layer/acl11.0express-macos-arm64.dmg cd /Applications/AllegroCL64express.app/Contents/Resources env ACL_OPENSSL_VERSION=30 DYLD_LIBRARY_PATH="$(brew --prefix openssl@3.0)/lib:$DYLD_LIBRARY_PATH" ./alisp (progn (require :ssl)(require :aserve)) (net.aserve.client:do-http-request "https://franz.com") This example does NOT modify the bundle. It does NO dynamic loading of OpenSSL. It does dynamically load aclissl3.dylib which is our library that statically binds to the OpenSSL libraries. Please, try this on 14 and 15.

@DTS Engineer "Once you work out what’s going wrong with the library loading, it’s likely that LLDB will just start working again." The library loads fine, but that's not even important anymore. The test I uploaded does STATIC linking of OpenSSL. Even that doesn't single step into any SSL function. I don't see how your example helps find the problem with my application. I've ALREADY made changes to the downloadable demo to bring it more in line with what was discussed here. Originally is dynamically loaded OpenSSL libraries. Now the libraries are statically linked. This means you don't even need to set DYLD_LIBRARY_PATH to run the example and get the SEGV.

@Etresoft Most likely, the problem is related to your patching system. I understand that this is a 40 year-old project that was working fine a couple of months ago. That's what I was talking about regarding "risk". Other people might use the term "technical debt". You can pay that debt in instalments over time, or you can wait until the bank calls in the loan. This is absolutely false. No patching has been done in the example I uploaded and included above. I built a NEW app bundle with all patches and notarized it. The problem still happens.

@Etresoft I'm not sure what language your app is written in, but ours is written in Common Lisp. Using Xcode isn't an option. With my last post, I showed that breaking the dependency on the Homebrew OpenSSL library doesn't do anything to change the errors, which don't happen in macOS < 15. It doesn't do anything to change the lldb hang when single stepping into a function in the SSL library, now statically bound into my own shared library. Making a standalone example with Xcode doesn't get me any closer to solving this problem for my customers, which are down at the moment. Downgrading to 14.7.1 isn't something they want to do, so they're waiting for me. It's just that I can't debug the problem. I've tried a lot of things over the last 10 days, and nothing has been effective. If the experiment today was successful, I would have embarked on the big task of making a static version, but that didn't work, so I'm back to square 1. I opened a DTS ticket and was told to post here. I'm hoping @DTS Engineer will escalate this to someone that can tell me why lldb doesn't work for this library. I suspect it's because macOS knows it's SSL and it's preventing lldb from snooping. Only Apple knows that, though.

@Etresoft @DTS Engineer Dynamic loading is a basic feature of our language. It's been there 40 years and it worked fine until macOS 15. The OpenSSL libraries are no different, I assume, than any other libraries. Let's take a step back. We use public, documented APIs for dynamic loading of shared libraries. Is Apple saying we should not use them? It's not a trivial thing to use the "canonical approach", either. I keep coming back to two things: It worked before macOS 15. lldb hangs when stepping into an OpenSSL library function. In the spirit of trying to find the problem, I spent hours making a statically linked version. That is, our glue library aclissl3.dylib now statically includes all the OpenSSL functions it needs, from the locally installed libraries. This is all based on OpenSSL 3.0. I replaced the aclissl3.dylib in the product. Loaded fine. Using it, however, gives me a SEGV: $ ./alisp International Allegro CL Free Express Edition 11.0 [64-bit macOS (Apple Silicon)] CL-USER(1): (progn (require :ssl)(require :aserve)) ; Fast loading ; /Applications/AllegroCL64express.app/Contents/Resources/code/SSL.003 ;;; Installing ssl patch, version 3. ... ; Foreign loading sys:aclssl3.dylib. CL-USER(2): (trace excl::rand_add) (EXCL::RAND_ADD) CL-USER(3): (net.aserve.client:do-http-request "https://franz.com") 0[3]: (EXCL::RAND_ADD #<Vector @ #x307892d62> 40 40.0d0) Error: Received signal number 11 (Segmentation fault: 11) [condition type: SYNCHRONOUS-OPERATING-SYSTEM-SIGNAL] Restart actions (select using :continue): 0: Return to Top Level (an "abort" restart). 1: Abort entirely from this (lisp) process. [1] CL-USER(4): It literally dies on the first call to the first function we call in that library. But, I can't debug it because lldb hangs: @max[git:master]$ lldb alisp lldb alisp (lldb) target create "alisp" Current executable set to '/Applications/AllegroCL64express.app/Contents/Resources/alisp' (arm64). (lldb) br s -n RAND_add br s -n RAND_add Breakpoint 1: where = libcrypto.46.dylib`RAND_add, address = 0x000000025a7241bc (lldb) br s -n catch_exception_raise_state br s -n catch_exception_raise_state Breakpoint 2: where = alisp`catch_exception_raise_state, address = 0x0000000100003380 (lldb) run run Process 49169 launched: '/Applications/AllegroCL64express.app/Contents/Resources/alisp' (arm64) 1 location added to breakpoint 2 International Allegro CL Free Express Edition 11.0 [64-bit macOS (Apple Silicon)] Copyright (C) 1985-2023, Franz Inc., Lafayette, CA, USA. All Rights Reserved. This development copy of Allegro CL is licensed to: Allegro CL 11.0 Express user ;; Optimization settings: safety 1, space 1, speed 1, debug 2, ;; compilation-speed 1. ;; For a complete description of all compiler switches given the ;; current optimization settings evaluate (EXPLAIN-COMPILER-SETTINGS). CL-USER(1): (progn (require :ssl)(require :aserve)) (progn (require :ssl)(require :aserve)) ; Fast loading ; /Applications/AllegroCL64express.app/Contents/Resources/code/SSL.003 ;;; Installing ssl patch, version 3. ; Fast loading ; /Applications/AllegroCL64express.app/Contents/Resources/code/OSI.fasl ; Fast loading ; /Applications/AllegroCL64express.app/Contents/Resources/code/fileutil.001 ;;; Installing fileutil patch, version 1. ; Fast loading from bundle code/acldns.fasl. ; Fast loading from bundle code/iodefs.fasl. ; Fast loading from bundle code/iordefs.fasl. ; Fast loading from bundle code/efmacs.fasl. 1 location added to breakpoint 1 ; Foreign loading sys:aclssl3.dylib. ; Fast loading ; /Applications/AllegroCL64express.app/Contents/Resources/code/ASERVE.001 ;;; Installing aserve patch, version 1. ; Fast loading from bundle code/autozoom.fasl. ; Fast loading from bundle zlib/inflate.fasl. ; Fast loading from bundle zlib/deflate.fasl. ; Foreign loading libz.1.dylib. ; Fast loading from bundle code/streamp.fasl. ; Autoloading for class BROADCAST-STREAM: ; Fast loading from bundle code/streamc.fasl. ; Fast loading from bundle code/efft-utf-8s-base.fasl. ; Fast loading from bundle code/efft-utf8-base.fasl. ; Fast loading from bundle code/efft-void.fasl. ; Fast loading from bundle code/efft-latin1-base.fasl. T CL-USER(2): (net.aserve.client:do-http-request "https://franz.com") (net.aserve.client:do-http-request "https://franz.com") Process 49169 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.2 frame #0: 0x00000001025e9cc0 aclssl3.dylib`RAND_add aclssl3.dylib`RAND_add: -> 0x1025e9cc0 <+0>: stp d9, d8, [sp, #-0x30]! 0x1025e9cc4 <+4>: stp x20, x19, [sp, #0x10] 0x1025e9cc8 <+8>: stp x29, x30, [sp, #0x20] 0x1025e9ccc <+12>: add x29, sp, #0x20 (lldb) si This constant "you're doing it wrong" is really frustrating because we've been doing it this way since the first version of macOS. It worked before the upgrade. I'm asking Apple to help us and tell us why.

@DTS Engineer I'm not asking for OpenSSL support. I'm asking for help on why lldb hangs when I try to step into an SSL library function. I also want to be clear: we can load the OpenSSL libraries fine. They work fine in macOS < 15. They do not work in macOS 15. lldb hangs in macOS 15. It does not hang in < 15. This all feels like some sort of security measure in macOS 15. I followed your "Posting a Crash Report" link and it seems to apply to everything except macOS. I don't see any "your app crashed" dialog, so I don't know if there is information in the system which would help diagnose the problem.

I want to also point out we do not use Xcode to build our product (which is 40 years old, at this point). We use the Command Line Tools for Xcode. The version used to compile the downloadable example above is 2395 (as printed by xcode-select --version).

I will mention this first happened on a customer machine. We did extensive debugging over zoom and found the "lldb hang" issue there. Then, I was able to reproduce in Parallels on my 14.7 machine. I ended up upgrading my machine to 15.1, and the problem happened immediately. At this point, my customers running 15.1 are all down. At least, the ones using HTTPS for outgoing network connections.

[quote='809525022, Etresoft, /thread/766491?answerId=809525022#809525022, /profile/Etresoft'] In theory, it's possible to modify parts of the application bundle and then supply an updated signature that matches the new bundle. But I'm not sure if anyone has ever managed to do that successfully. It's a high-risk, high-effort solution when storage and bandwidth are cheap. [/quote] Can you explain more about this? It seems we might have to go that route, otherwise, it's the redesign of a 40 year old application.

@Etresoft thanks for the reply. How do applications patch themselves and still not run afoul of Gatekeeper, then? Are they just complete reinstalls? Allegro Common Lisp is 35+ years old. We have a customers with a huge amount of history. We have been using our patching system for decades. Sounds like Apple is saying we must throw that away, because the patches do include binary executables (shared libraries), though they are code signed. Moving everything that changes outside of Contents/Resources will mean either it's user specific, or we'll need to use /Library/Application Support/, which will require admin privs. Also, it's quite a lot of redesign for us to move everything out. As for the "not an app" ... I mean, Apple notarized our app and didn't complain then. Seems like I should have gotten errors or warnings for that.