wmk's Misadventures in Compiling C++, Finale (?) JXL is working on my iPhone after I got rid of libjpegli. The weeklong tumble down the rabbit hole of C++ compilation is finally over; and thus ends this impromptu blog I started writing for debugging purposes and to keep myself sane. At the end of this I look back and realize how much I've been spoiled by Xcode, where compiling is just a mindless habit. Having to figure out CMake after being raised on Xcode is like getting chucked into the slums after having lived all your life in an ivory tower being pampered by servants who aren't perfect, but get the job done. I've done some preliminary tests on encoding, decoding, and animation and everything seems to be working fine. The code in https://github.com/awxkee/jxl-coder-swift works even with TaskGroups, which allows me to turbocharge everything. JXL so far seems to be absolutely worth the effort. Now I just need to fiddle with the parameters to optimize performance for my app, and actually examine the animation output. @endecotp I cannot thank you enough for introducing me to JXL and sticking with me until I got it working. I was about to spend perhaps hundreds of dollars, but in the end all I spent was $2.99 on an Upwork job post. If the opportunity ever arises, I'd definitely return the favor. P.S. Upwork is a dumpster fire.

That's probably it, given how touchy CMake seems to be I'm currently stuck because I can't get jpegli to compile, and apparently its wired into the package. It just absolutely refuses to be compiled. I suspect it has something to do with -- Could NOT find JPEG (missing: JPEG_LIBRARY JPEG_INCLUDE_DIR) But I don't know where to begin because, while installing libjpeg-turbo on this would probably lead to the warning going away, it would likely lead to cross-compiling issues (that thing Quinn was talking about). Apparently I'm not the only one with issues compiling jpegli: https://github.com/libjxl/libjxl/issues/3440 I've also tried a slew of different hacks, including isolating it and forcing it to compile, but I'm stuck on this error: CMake Error at jpegli.cmake:46 (target_link_libraries): Target "jpegli-static" links to: Threads::Threads but the target was not found. Possible reasons include: * There is a typo in the target name. * A find_package call is missing for an IMPORTED target. * An ALIAS target is missing. Anyways, I'll figure something out, like precisely excising it from libjxl

wmk's Misadventures in Compiling C++, Episode 2 After a break, it became apparent that tools had to be purged from the process entirely. This was accomplished by removing add_subdirectory(tools) from the main CMakeLists.txt. [ 73%] Linking CXX executable tests/codec_test.app/tests/codec_test ld: open() failed, errno=2 for 'tests/codec_test.app/tests/codec_test' clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: *** [lib/tests/codec_test.app/tests/codec_test] Error 1 make[1]: *** [lib/CMakeFiles/codec_test.dir/all] Error 2 make: *** [all] Error 2 I'm finally back to where I started--question now is why this is being built at all. Since I made progress by removing a part of the main CMakeLists.txt, I decided to go all-in and started hacking away entire sections of the file like I was chopping a piece of wood. So this is why they call it "hacking". My mental image of a hacker became a bit more like a lumberjack. Everything below if (JPEGXL_ENABLE_MANPAGES) was hacked off. This means no testing, but at this point I was willing to accept this tradeoff just to see some progress. The result was counterintuitive: we're back to ld: unknown options: --exclude-libs=ALL at a dylib. It was here I realized just how finnicky CMake is and restarted my mac. Starting with a fresh ios-cmake and libjxl, I went through all the steps again, removing the bottom chunk of CMakeLists.txt. Success! [ 95%] Built target jxl_enc-obj [ 95%] Linking CXX static library libjxl_dec-internal.a [ 95%] Built target jxl_dec-internal [ 96%] Linking CXX static library libjxl-internal.a [ 96%] Built target jxl-internal [ 96%] Linking CXX static library libjxl.a [ 96%] Built target jxl [ 97%] Linking CXX static library libjxl_dec.a [ 97%] Built target jxl_dec [ 98%] Building CXX object lib/CMakeFiles/jxl_threads.dir/threads/resizable_parallel_runner.cc.o [ 98%] Building CXX object lib/CMakeFiles/jxl_threads.dir/threads/thread_parallel_runner.cc.o [100%] Building CXX object lib/CMakeFiles/jxl_threads.dir/threads/thread_parallel_runner_internal.cc.o [100%] Linking CXX static library libjxl_threads.a [100%] Built target jxl_threads Now to see whether this is a stillbirth, an abomination, or a little angel that poops out JXL. My expectations are low; what I've done so far is trivial, especially for a professional, and we do have a professional working at TikTok's parent company saying, "This coder does not supports JPEG-XL encoding (Because I have no time :))",. The adventure continues...

Just knowing something is out of my league makes things so much better. At least I won't be pestering people for help while thinking it's a quick and simple task.

Thank you so very much. In the interim I had already opened an account on Upwork, waded through LLM spam and grifters, found someone willing to do it for $150, had money persistently rejected by Upwork, found its support intentionally disabled, then closed my account (takeaway: Upwork is a dumpster fire). That was 6 hours down the drain. In comparison, this is a godsend, even if I might need to put in some work myself wmk's Misadventures in Compiling C++, Episode 1 I used to get around the install TARGETS given no BUNDLE DESTINATION error by adding BUNDLE DESTINATION after RUNTIME DESTINATION but I had to do it for tools/CMakeLists.txt as well. I cleared everything and followed your instructions from scratch, verbatim. My first attempt after git cloning and modifying the brotli/CMakeLists.txt, I was getting the error: CMake Error at tools/CMakeLists.txt:340 (install): install TARGETS given no BUNDLE DESTINATION for MACOSX_BUNDLE executable target "cjxl". Maybe your invocation was supposed to disable the building of tools, so I double-checked that mine's a verbatim copy of yours (it was) and considered whether your libjxl might be older. Since it seems tools was still being built for some reason, I appended the following to your command: -DJPEGXL_ENABLE_DEVTOOLS=false \ -DJPEGXL_ENABLE_VIEWERS=false \ -DJPEGXL_ENABLE_PLUGINS=false \ -DJPEGXL_TEST_TOOLS=false \ -DJPEGXL_STATIC=true Same error. So I applied your fix by removing the offending install call from tools/CMakeLists.txt and reverted to your original command. The config succeeds but always has zsh: no such file or directory: -DCMAKE_INSTALL_PREFIX=/Users/username/Desktop/compiled at the end (Note: I clear build.ios between every attempt). I then ran make: [ 61%] Linking CXX executable tests/codec_test.app/tests/codec_test ld: open() failed, errno=2 for 'tests/codec_test.app/tests/codec_test' clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: *** [lib/tests/codec_test.app/tests/codec_test] Error 1 make[1]: *** [lib/CMakeFiles/codec_test.dir/all] Error 2 make: *** [all] Error 2 Little did I know, I'll be yearning to see this error in an hour. I saw "executable" and "test" and realized it was still compiling executables for tests, so I went back and added the extra flags above, then ran make again. [ 20%] Linking CXX shared library libjxl_dec.dylib ld: unknown options: --exclude-libs=ALL clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: *** [lib/libjxl_dec.0.10.2.dylib] Error 1 make[1]: *** [lib/CMakeFiles/jxl_dec.dir/all] Error 2 make: *** [all] Error 2 Why is it building a dylib? Is there an issue with the flags? I tried variations of the invocation your flags + -DJPEGXL_STATIC=true: Same error I decided to give your vanilla command another try: Same error? Clearly some state is being saved somewhere, so I wiped the entire libjxl folder and copied in the untouched original, then reapplied CMakeLists.txt changes: Still the same error!? This was where grand attempt 1 was reset since I was stuck with the dylib error (I'm writing this log as I retrace my steps with grand attempt 2) I tried various changes, including retracing my precise steps above, but I couldn't return to the previous error. I suspected CMake has some sort of cache/state set somewhere, so I looked online, found nothing, then did what any good techie did and reset my mac. Still there. Grand attempt 3 was basically the same except I never saw ld: open() failed, errno=2 . env: macOS 14.5, CMake 3.29.6 The adventure continues...

I don't know what I'm doing, so I'm using -G Xcode simply because that's what's in the ios-cmake toolchain example. I took your advice and looked for flags to set, and this is my new cmake invocation (tl;dr everything off except static flag, which invokes -DBUILD_SHARED_LIBS=OFF and required files): cmake -B build \ -DJPEGXL_ENABLE_FUZZERS=OFF \ -DJPEGXL_ENABLE_DEVTOOLS=OFF \ -DJPEGXL_ENABLE_TOOLS=OFF \ -DJPEGXL_ENABLE_JPEGLI=OFF \ -DJPEGXL_INSTALL_JPEGLI_LIBJPEG=OFF \ -DJPEGXL_ENABLE_DOXYGEN=OFF \ -DJPEGXL_ENABLE_BENCHMARK=OFF \ -DJPEGXL_ENABLE_EXAMPLES=OFF \ -DJPEGXL_BUNDLE_LIBPNG=OFF \ -DJPEGXL_ENABLE_JNI=OFF \ -DJPEGXL_ENABLE_SJPEG=OFF \ -DJPEGXL_ENABLE_OPENEXR=OFF \ -DJPEGXL_ENABLE_SKCMS=ON \ -DJPEGXL_ENABLE_VIEWERS=OFF \ -DJPEGXL_ENABLE_PLUGINS=OFF \ -DJPEGXL_ENABLE_COVERAGE=OFF \ -DJPEGXL_ENABLE_TRANSCODE_JPEG=OFF \ -DJPEGXL_STATIC=ON \ -DJPEGXL_TEST_TOOLS=OFF \ -DJPEGXL_ENABLE_AVX512=OFF \ -DJPEGXL_ENABLE_AVX512_SPR=OFF \ -DJPEGXL_ENABLE_AVX512_ZEN4=OFF \ -DJPEGXL_FORCE_SYSTEM_BROTLI=OFF \ -DJPEGXL_FORCE_SYSTEM_GTEST=ON \ -DJPEGXL_FORCE_SYSTEM_LCMS2=ON \ -DJPEGXL_FORCE_SYSTEM_HWY=OFF \ -DBUILD_SHARED_LIBS=OFF \ -DCMAKE_TOOLCHAIN_FILE=../../ios.toolchain.cmake \ -DPLATFORM=OS64 \ -DCMAKE_BUILD_TYPE=Release \ -DBUILD_TESTING=OFF .. With -G Xcode I'm back to the same errors as above with highway and brotli, which means this is a dependency issue even with flags set. Without it, I get the following error: [ 6%] Linking CXX executable hwy_list_targets.app/hwy_list_targets ld: library not found for -lcrt0.o clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: *** [third_party/highway/hwy_list_targets.app/hwy_list_targets] Error 1 make[1]: *** [third_party/highway/CMakeFiles/hwy_list_targets.dir/all] Error 2 make: *** [all] Error 2 At this point, I have to say that this has been a learning experience but this is too much to just evaluate whether JXL fits my needs. I need to get back to productive work. Can I make you an offer? If you compiled JXL yourself so it works on your iOS runtime, with encode/decode and animation functions intact and performant, I will gladly pay you for detailed instructions on how to replicate your success, and also a bit more to thank you for your help here and elsewhere. We can discuss the details elsewhere. Note that I'm not asking for a compiled product (security issues), but for the instructions on how to get from the reference repository to iOS runtime with the requested functions.

[quote='791764022, endecotp, /thread/757359?answerId=791764022#791764022, /profile/endecotp'] I do include it in my iOS app at runtime. Doing so was not a particularly easy process. I used something from github called "ios-cmake", which is a pile of scripts and/or cmake configuration fragments. 0  comments [/quote] I'm experiencing first-hand how uneasy this is. Especially since I'm new to CMake. I was able to prepare the compile after making minor adjustments to the CMakeLists.txt files of outdated dependencies, but after that I've been unable to do the actual build process using ios-cmake due to errors cmake -B build -G Xcode -DCMAKE_TOOLCHAIN_FILE=../../ios.toolchain.cmake -DPLATFORM=OS64 -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF .. cmake --build . --config Release Yields the error ** BUILD FAILED ** The following build commands failed: Ld /Users/username/Desktop/libjxl-main/build/build/lib/Release-iphoneos/libjxl_cms.0.10.2.dylib normal (in target 'jxl_cms' from project 'LIBJXL') (1 failure) It feels like I'm close, but I can't even gauge whether I'm close to figuring this out or whether I'm actually miles away--there's much uncertainty when you're new to something.

@CMDdev Yeah that’s true, the key would be protected (I was talking about protecting process memory, not your idea of encrypting assets). Most of that paragraph was me not reading your post closely enough and just ranting on. My apologies. I agree with your general point that even with memory encryption, the jailbroken state of the device means the memory might not be safe, though I'd say that if implemented, memory keys would likely be stored in something akin to a secure enclave. Anyways I'll add as many pieces of Swiss cheese to my defenses as my time, ability, and the app's performance allows. Anything to slow down the competition.

@endecotp If something builds for arm64 macOS, shouldn't it work with iOS/iPadOS, which is also arm64? Anyways, I might have misunderstood you earlier--are you saying you use libjxl during development but don't include it as part of your runtime? Because I thought you included it and that making libjxl work on iOS is an easy/supported process. I'm scratching my head trying to figure out how to compile libjxl to include in my binary but the more I look at the problem, the more it seems like this is far more than what a complete novice at cmake should even attempt, especially when a professional says "This coder does not supports JPEG-XL encoding (Because I have no time :))" Looks like I need to go back to using video as storage, or maybe try AVIF/webp/jpeg2000

[quote='791426022, endecotp, /thread/757255?answerId=791426022#791426022, /profile/endecotp'] With your scheme, make sure you test what happens when a user gets a new device and your app is copied over automatically for them. [/quote] This is a really good point I haven't thought of. Looks like I'll either have to delete everything (with explanation) or have some way of detecting a transfer and then re-encrypting. Without transfer detection this re-encryption mechanism can be abused. @CMDdev one already has broad access in a jailbroken phone, and that might include access to this hypothetical process memory decryption key I'm fairly sure the enclave key never leaves the enclave and therefore can't be intercepted unless the enclave itself was somehow compromised, but I think a compromised enclave and its implications would mean Apple's whole security model is broken. The attackers not being able to get a key doesn't matter if everything the key is meant to protect can be captured via memory. Moreover, enclave keys should only be a portion of all types of keys, with many needing to stay in memory--are they all vulnerable? I don't think Apple security engineers are fools; I think we might be missing something if decrypted memory capture really is as simple as it sounds. Yes, but also, even if it isn't the first one, there could be reasons why that app is better. First one might not necessarily be the best one. This also applies to most Apple hardware, which tend not to be the first of their kind but (usually) the most well-executed. P.S. Turns out the word "nuk.e" (sans period) is censored. I don't get it.

@endecotp Thank you. I'm going to try to compile it for iOS and then incorporate wrappers around it. So much to learn, so little time. Does the build instructions in the docs work for iOS/arm? cd libjxl mkdir build cd build cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF .. cmake --build . -- -j$(nproc)

@endecotp I did some reading on JXL and it looks promising. The format seems to have some inter-frame compression for animation, which might be what I'm looking for. On top of that the encode/decode speed seems good, but that depends on how well the iOS implementation runs. I need to run tests. What do you use to incorporate JXL into your iOS projects? Unless something is by a trusted or official source, I have to go through every single line, which means something like https://github.com/awxkee/jxl-coder-swift is prohibitively long to get through.

You again! That's three for three now; thank you for always responding to me. What sort of images are they? Are they continuous-tone, or paletted? They're regular PNG images, and they have to be PNG because JPEGs don't support alpha and HEVCs are just too computationally expensive for my purposes. I have had good experience using JXL, which has a lossless mode. Its lossy mode with a high quality setting could also be useful. Assuming you're referring to JPEG-XL, it's likely not going to work as speedily with UIImage, which is my main use case. Are you suggesting that when I should convert to JXL when I need to minimize storage and then convert back to PNG? Re video codecs, you should investigate what people are using for screen recording. Would you mind elaborating on that? I don't see your point

[quote='791387022, CMDdev, /thread/757255?answerId=791387022#791387022, /profile/CMDdev'] This is obfuscation - one can reverse this mechanism and obtain your asset decryption key depending on how this works. So I wouldn't say they are "protected", but rather "better hidden". [/quote] Even if I store it in the enclave? My current system creates a unique key for each user-device combination, and uses that to encrypt assets at rest. The enclave key is said to never leave the enclave, so I'm convinced that this is the best I can do within the confines of the framework and platform. If someone can pull assets from memory then no amount of clever encryption will help unless the memory itself is encrypted, but this is one of the few security features that Apple hasn't implemented. But I think you should also focus on branding your product so that even if clones / alternatives appear, yours will still be the best one, the one that people choose. This is a good suggestion and I hope to succeed at this, but having two protections is better than one. Your suggestion is based on the idea that apps can have a good first-mover's advantage, and I agree, but I've heard horror stories of apps being cloned within moments of release, and the cloner has an advantage since they're just going to leap-frog off my work.

Thank you both for responding! Since most of my questions on the forum go unanswered, I feel spoiled just by having two responders. [quote='791275022, CMDdev, /thread/757255?answerId=791275022#791275022, /profile/CMDdev'] Yes, though some class / method names will still be visible. [/quote] Is there a way to tell which ones will be? [quote='791275022, CMDdev, /thread/757255?answerId=791275022#791275022, /profile/CMDdev'] Also, the IPA is not encrypted on your iPhone, but using such methods was needed to get the IPA, as you couldn't download iOS apps on devices other than iPhones. [/quote] Sorry, I meant decrypted binary. Basically, I read up on how people obtain and tweak IPAs, and the process seems to be to: Obtain an IPA via jailbroken iDevice. While the IPA is accessible (so don't store secrets in there), the compiled app binary is decrypted Start the app on the iDevice, which loads the decrypted app into memory Using various tools, capture the decrypted app from memory Tweak and repackage [quote='791318022, endecotp, /thread/757255?answerId=791318022#791318022, /profile/endecotp'] Of more concern is extracting non-code, for example the expensively-designed graphical or sound resources in a good game. [/quote] I encrypt all assets at rest, but decrypt them to load into memory when needed. I think I'm protected by this process, but If there's a way to pull the assets from memory nothing I do would help. Is this assumption true? P.S. There seems to be an issue with how quotes are handled by this forum; multiple quotes can lead to misaligned formatting