jmp_buf layout for Apple Silicon

Greetings! I am actively working on porting x64 code to Apple Silicon now that the time is nigh and part of the fundamentals of our software is a coroutine library for handling cooperative multitasking of GUI operations on the main thread. I was hoping to get the locations of the stack pointer and frame pointer in jmp_buf so, after setjmp() can redirect them to the primary handling routines in our coroutine library that handles the cooperative scheduling (which replaced and ported the old classic MP routines) which worked for PowerPC, i386 and x64.

Any thoughts on where in the jmp_buf these might be located? I didn't see anything in the XNU open source.

Any advice would be much obliged instead of having to dive in and re-implement these routines in assembly myself!

Answered by DTS Engineer in 851305022

Keep in mind that Apple platforms only guarantee binary compatibility at System framework layer. The layout of jmp_buf is not in the macOS SDK. Rather, this structure is always manipulated by System framework functions (setjmp, longjmp, and friends). That means that the layout could change without breaking apps in general. If you reverse engineer that layout from open source, you open yourself up to binary compatibility problems if that layout ever changes.

Now, I’d be surprised if it actually changed often [1], and so the maintenance workload associated with this path may well be less than the maintenance workload of doing this stuff yourself in assembly language. This is one of those tricking trade-offs that crop up all the time in engineering.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Within the same architecture. This stuff obviously changes when you change architecture.

Have you looked through the clang open source project? I suspect what you're looking for will be there, rather than the in the XNU source.

— Ed Ford,  DTS Engineer

Oh good thought! Let me take a look into clang as, yes, this is user space only and really has nothing to do with the kernel.

Accepted Answer

Keep in mind that Apple platforms only guarantee binary compatibility at System framework layer. The layout of jmp_buf is not in the macOS SDK. Rather, this structure is always manipulated by System framework functions (setjmp, longjmp, and friends). That means that the layout could change without breaking apps in general. If you reverse engineer that layout from open source, you open yourself up to binary compatibility problems if that layout ever changes.

Now, I’d be surprised if it actually changed often [1], and so the maintenance workload associated with this path may well be less than the maintenance workload of doing this stuff yourself in assembly language. This is one of those tricking trade-offs that crop up all the time in engineering.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Within the same architecture. This stuff obviously changes when you change architecture.

Yes, that is the dilemma...either maintaining something for setjmp/longjmp or maintaining assembly code long-term. Neither of which are great options, I admit. I've got the assembly lying around, but I figure even if it needs maintenance going with setjmp/longjmp stands a better chance of maintaining any new vector registers or such in future iterations of Apple Silicon!

I miss having a path forward for porting the older cooperative MP code into macOS, but that's been the case ever since Rhapsody, really.

Just as a followup...setjmp/longjmp do not appear to be following the clang libc project. Inside of the jmp_buf, after storing x27 it deviates from the clang libc and immediately stores x10 and x11, and the rest of the information is zeroes after that. So it seems there is a custom Apple implementation of setjmp/longjmp (perhaps inside of libsystem_platform.dylib?) that deviates and is using some kind of different buffers..

Bummer as it would kinda been nice to find a solution that isn't trying to maintain low level assembly that will be more difficult to maintain across architecture revisions. Still, I'll spend more time digging (maybe those x10 and x11 are buffer addresses?) and see if there's a little bit of magic to be had with a setjmp/longjmp approach :)

It's an oddball case, but I'm sure implementing coroutines are still fairly common as we need to slice and dice the main thread up for GUI operations and libraries such as Qt that are not fully pre-emptive safe.

Well, it's working! Thanks for the starting off points. I'll not give the full details as it's not a recommended solution from Apple, but for those on the same path the breadcrumbs will lead to libplatform and getting some understanding of thread specific data storage. But it all works now, coroutines with setjmp/longjmp in user space!

Sadly the open source project for this specific library is now defunct so nowhere to upstream the new code, but it's nifty that it will now work for macOS running on PowerPC, i386, x64 and Apple Silicon. I believe it's time for a disco party.

What, no 68K? (-:

Seriously though… Huzzah!

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

jmp_buf layout for Apple Silicon
 
 
Q