Code Signing a C++ universal application bundle with a embedded python3 framework

Question

Created May ’22

Replies 4

Boosts 0

Views 1.9k

Participants 3

Hi all,

I'm developing a 3D modeling C++ application with embedded Python scripting capabilities which targets Big Sur. I want to distribute my application ("MyApp") with a full python package directly integrated into the MyApp bundle (the MyApp.app folder) so that users won't have to install Python manually.

So I binded Python3.9 and my app using pybind11, and copied the Python framework folder (all files of the folder of the version 3.9, which is named "3.9") into the "Framework" directory of my App bundle, then locally set the PYTHONPATH and PYTHONHOME environment variables at run time so that they point to the python's Framework folder copied into the bundle. It's working: python scripts can run from my application even if there isn't python installed in the system.

However, I have an issue when signing my MyApp bundle. Assuming that I need a python framework package which is universal, correctly signed and has folder structure and files compliant with Apple's bundle specs, I saw too options at first

On one hand, homebrew provides signed python packages but for arm64 architecture only, so it must be excluded since I need x86_64 too.
On the other hand, the official python website provides universal python packages but they are not signed.

I then copied the Package from the official python website and removed many of its unessential components to make it tidy as much as possible, then ran a script that codesign all files that codesign signals as "not signed at all" when running it on the full RizomUV App bundle. Once all files that need to be signed have been signed, I got the following message when running codesign on the MyApp bundle folder:

codesign --force --verify --verbose --sign "Developer ID Application: XXXXXXX (XXXXXX)" MyApp.app --option runtime
MyApp.app: replacing existing signature
MyApp.app: bundle format unrecognized, invalid, or unsuitable
In subcomponent: /Users/me/Documents/a_path/MyApp.app/Contents/Frameworks/Python/lib/python3.9

That python3.9 folder, which contains a bunch of python script files (***.py) and some directories which seems to be not compliant with the bundle specifications. This prevents the signature of the full bundle and that's obviously a problem.

I'm sure I'm not the only one who integrated Python as a framework into a universal bundle. I could do more investigations but I'm less and less confident that I'm following the right path as I find it overly complicated. There must be a better way right?

Any help or feedback would be more welcomed.

Best

Boost

Answer 1

DTS Engineer OP

Apple

May ’22

I don’t have a ready answer for you but I do have some resources:

The last two are key, although the steps recommended by the last one are probably not sufficient for a package as complex as Python. Python has its own view of how its file system hierarchy should be laid out and that conflicts with macOS’s expectations.

As to how you should proceed here, I think your best bet is to look at how Xcode does this with its built-in Python 3, namely /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework. I don’t know how they construct that framework but, once you have it, adding it to MyApp.app/Contents/Frameworks should be fine.

Some general notes:

It wouldn’t surprise me if that’s something well understood by the Python community. You might, for example, discuss this with the [folks who build PyInstaller] (https://pyinstaller.org/en/stable/contributing.html).
Xcode won’t be able to sign this framework (because it doesn’t follow the rules in Placing Content in a Bundle). If you use Xcode, you’ll have to sign the framework yourself and then tell Xcode to embed it without re-signing it.
Xcode probably won’t be able to re-sign an archive containing this framework. You’ll have to do that yourself as well.
Do not use --deep when signing this. It definitely won’t do the right thing with a framework structured in this way. See --deep Considered Harmful.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

0

Answer 2

RemiArq OP

May ’22

Thank you very much for that detailed answer, I appreciate.

However I already had a look at these documents before, and as I'm bound by dead lines, I was hoping to stop investigations and get ready-to-embed package link, or a workaround. As it seems to be overly complicated regarding the time left, I may let the Python support for Linux and Windows only for now.

Of course, further help are still welcomed, as I would like to propose the Python support on Mac for the next releases.

Best regards

0

Answer 3

nk_kennedy OP

May ’22

The last time I checked, Python was pretty easy to build. You should be able to build it yourself and properly configure it to contain only the components that you need. The most difficult part will be the SSL. See if there is an option to use Apple's built-in SSL. If not, omit it if you can. Otherwise, 3rd party crypto can bring some technological and legal problems.

You will need to roll your own Python framework from that. That, too, isn't very difficult. Just review how other frameworks are constructed. As eskimo points out, you can even use Apple's Python framework as an example. Just be careful because you aren't Apple. Sometimes Apple takes liberties that 3rd party developers don't have. For example, Apple uses a "3.8" version. In theory, that should be fine. But I don't like to deviate from defaults that are known to work unless I have a really good reason. So use "A" instead. But otherwise, that structure is good. Build the framework by hand. Copy your Python dylib to the top-level "Python" or "Python3" file. Copy your python installation bin, lib, and share folders as Apple has done. You'll need to hack up the include folder similar to how Apple has done. Don't forget the Info.plist file and the Resources folder that both need to be hacked together.

Then, you can include your real Python framework in your app and that should resolve all of the code signing problems. Currently, what you have in the Frameworks folder isn't a real framework, so that is why it's choking on it.

Ideally, you would have a true Xcode project that builds Python and then include that in your app's workspace. That would likely be too difficult at first go. Xcode will accept a 3rd party framework rudely shoved into the project. Since you are only building for macOS, you don't have to worry about multiplatform complications. Considering the fact that Apple already does all this, it is definitely possible.

While the Python people definitely know Python, they seem to know little about macOS. Bundling Python into your app the way you are attempting is definitely the correct way to do this. You just need to build a proper framework and you're good to go. These forums are full of people who've tried to use PyInstaller and got hopelessly stuck trying to get notarization to work.

0

Answer 4

RemiArq OP

May ’22

Thank you for your help guys, I could successfully notarize my application's bundle with Python3 integrated.

One of the issues was, that I had to copy the entire Python.framework folder, instead of just the 3.9 folder located into "Versions". The second issue was that my copy process of the python Framework folder broke the code signing of some files, I guess because it did not copy the files' attributes, but I'm not 100% sure.

So it's important to note that what I wrote in my first post is wrong about python package from the official website. I can now say the official python website provides universal python packages correctly signed.

Best

0