Post

Replies

Boosts

Views

Activity

Reply to How does a C program compiled from the command line link against a library in the Big Sur dynamic library cache?
Thank you for the reply Quinn. I think I understand your answer. I am using XCode and the command line tools and linking against the .tbd stub libraries in the SDK. But to do it I had to specify the path to the stub files since the path to the stub files does not lie along the standard shared library search path automatically. What I'm worried about, and it's probably nothing to worry about, is that someone who uses the compiled program and does not have XCode installed, will have problems linking to the libraries. I'm guessing when ld builds an executable and you use -lc to tell ld to link against the c standard library, the path to the library is not built into the executable. Instead, will the run time linker then use the shared cache for the libraries? I can't test this theory easily since I only have one machine and it has XCode with the command line tools installed. Uninstalling and reinstalling XCode and tools to test it is doable, but I still wouldn't know if the uninstall process completely removed everything... so I figured I'd ask you instead.
Dec ’20
Reply to How does a C program compiled from the command line link against a library in the Big Sur dynamic library cache?
What happens when you try to compile without specifying that path? What error message do you get? What does your compile command look like? Compiling this code #include <unistd.h> const unsigned char pmessagebuf[13] = "hello world\n"; int main (int argc, char* argv[]) { write(STDOUT_FILENO, (void*)pmessagebuf, 12);      return(0);    } using these commands: cc -c ./testcprogram.cpp -mmacosx-version-min=10.11 -mstackrealign ld -execute -macosxversionmin 10.11 -lSystem -o testcprogram testcprogram.o Does not work. You get the error that the system library can not be found. Instead you have to use this linker command: ld -execute -macosxversionmin 10.11 -lSystem -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib -o testcprogram testcprogram.o Don't you think that if there was a problem, someone would have noticed it by now? It is a known issue in Big Sur and is in the release notes. I only found out about the work around because a lot of others on the internet already figured it out and posted about it. The path to the library is is built into the executable. You can inspect that with "otool -L". The fact that the operating system uses the shared cache is an implementation detail. If that's true, then you can't use ld to link .o files that have calls to the system library for production software anymore because the libraries will not be able to be found when xcode and the command line tools are not installed. Don't worry. About 50 million other people have thoroughly tested it by now. If you do have concerns about how your app would run in a "factory fresh" computer, you can install a virtual machine like Parallels and test it there. I strongly recommend that. Many developers install dynamic libraries from various "ports" or "brew" tools, from crazy cross-platform development tools, and are completely helpless when their apps don't work properly anywhere except their development machines. As you can see, I'm not using a crazy cross-platform development tool to compile this code. I'm only using the XCode command line tools cc and ld. And yes, there is a problem. I was hoping to hear Apple has say. Like if this will be fixed in the next release, or if support for ld is dropped, or if somehow using -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib with ld will work for production software. But from what you are saying, it looks like -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib is not a good solution.
Dec ’20
Reply to How does a C program compiled from the command line link against a library in the Big Sur dynamic library cache?
You are using ld directly to link an executable? Why? I've never seen anyone do that. It's C 101. OK. So? You are doing something totally unique and you found a workaround - for today at least. I know I'm getting old, but hopefully they still teach this in school. We're talking about using ld at its most fundamental level. ld is used to link things together, including the system library. What you are technically saying is Apple is dropping support for using ld to link production executables. I hope you don't work for Apple. I haven't see anything in the release notes about using ld. Maybe you are referring to the dynamic library cache issue. That is only incidentally related to ld.  Yes, I am referring to the dynamic library cache issue. This is what broke ld. The Apple work around for the dynamic library cache issue is to use dlopen. Unfortunately, if your executable has to link against the system library to gain access to dlopen, and an ld compiled executable can't 'see' the system library because it's in the cache, then you can't use ld to link production software anymore. If this is the case, then Apple should say it is dropping support for the command line tools and all code must be compiled using the XCode IDE. However, the -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib may work... I just have to test it. It's just ridiculous to have to tell ld what the path is to the libraries. The default paths for the ld linker tool should include everything it needs for linking against the system library once you install the command line tools. What I'm really curious is how the XCode IDE is linking production executables. They must not be using ld.
Dec ’20
Reply to How does a C program compiled from the command line link against a library in the Big Sur dynamic library cache?
If you want to explicitly run the linker, you’ll need to tell it what SDK to use. A good way to work out the exact syntax is to run the compiler driver with -v: Code Block "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -ltolibrary /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -dynamic -arch x8664 -platformversion macos 11.0.0 11.0 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -o test -L/usr/local/lib test.o -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0/lib/darwin/libclangrt.osx.a The ‘secret sauce’ here is -syslibroot. You can inspect that with "otool -L". Thanks Quinn and etresoft. I was able to link the clang generated helloworld.o file using this command: ld -demangle -dynamic -arch x8664 -platformversion macos 11.0.0 11.1 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -o testcprogram -L/usr/local/lib testcprogram.o -lSystem I also verified with otool that the only library path for dyld and the system library in the executable is /usr/lib I was also able to link a helloworld .o file generated using an assembler I wrote. I noticed that -execute is no longer required for ld, and that -platform_version has replaced -mmacosx-version-min. I also couldn't find any documentation on -syslibroot. I'm guessing that the documentation for these changes will come sometime in the future.
Dec ’20
Reply to How does a C program compiled from the command line link against a library in the Big Sur dynamic library cache?
I was also able to get this assembly language hello world to work: // Hello World for Big Sur .text _helloworldmsg: .asciz "Hello World!\n" .globl _myhelloworldstart _myhelloworldstart:     // syscall is deprecated.     // Last I checked, if you use syscall, the assembler compiles an imported function call.     //  If you hard code the syscall opcode, macos will not execute it and will give you an error     //  so I am using imported function calls to access the system library functions.     andq $0xfffffffffffffff0, %rsp // Force alignment. Without this exit will seg fault.     movq $1, %rdi     leaq _helloworldmsg(%rip), %rsi     movq $13, %rdx     call _write      xorq %rdi, %rdi // clear rdi to return success     call _exit using these commands: as -mmacosx-version-min=10.11 ./helloworldbigsur.s -o helloworldbigsur.o ld -e myhelloworldstart -nouuid -noehlabels -demangle -dynamic -arch x8664 -platformversion macos 11.0.0 11.1 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -o helloworldbigsur -L/usr/local/lib helloworldbigsur.o -lSystem
Dec ’20
Reply to Mac OS 11.0.1 Linking X64 system calls to System Library
Big Sur has moved the system libraries to a cache that is no longer in the file system. Eskimo from Apple answered how to figure it out for my 'how to link to the libraries in C' question in another thread, and that answer should work for a Nasm generated .o file as well. I also figured out how to do it using the gnu assembler that comes with XCode and this is how I did it: Using this gnu assembler assembly language source code: .text _helloworldmsg:.asciz "Hello World!\n" .globl _myhelloworldstart _myhelloworldstart:    &#9;// syscall is deprecated.    &#9;// Last I checked, if you use syscall, the assembler compiles an imported function call.    &#9;//  If you hard code the syscall opcode, macos will not execute it and will give you an error    &#9;//  so I am using imported function calls to access the system library functions.     andq $0xfffffffffffffff0, %rsp&#9;// Force alignment. Without this exit will seg fault.  // In other words, the system functions require that the return stack is 16 byte aligned before you call them.   movq $1, %rdi    leaq _helloworldmsg(%rip), %rsi    movq $13, %rdx    call _write     xorq %rdi, %rdi // clear rdi to return success    call _exit I was able to compile and link using these commands: as -mmacosx-version-min=10.11 ./helloworldbigsur.s -o helloworldbigsur.o ld -e myhelloworldstart -nouuid -noehlabels -demangle -dynamic -arch x8664 -platformversion macos 11.0.0 11.1 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -o helloworldbigsur -L/usr/local/lib helloworldbigsur.o -lSystem It looks like /usr/local/lib is the path to the dynamic library cache even though the files are not in the file system. When I used otool to look at the executable, it looks like the system loads dyld from /usr/lib and then adds /usr/local/lib to the library search path.
Dec ’20