How to Compile Python with a Statically Linked OpenSSL on macOS

I’ve recently worked on software that requires a bundled installation of Python. The application itself is not written in Python, but it needs to run Python programs. And, we can’t rely on the destination already having the correct version of Python installed. The application runs on macOS and on Linux, but this post deals with macOS specifically.

Problem: How to Compile Python with a Statically Linked OpenSSL on macOS

To get the specific version of Python needed for the application (3.9.x), I knew I would have to compile it myself. Compiling Python from source is well documented. The macOS-specific instructions eventually get running configure as follows:

CFLAGS="-I$(brew --prefix gdbm)/include -I$(brew --prefix xz)/include" \
  LDFLAGS="-L$(brew --prefix gdbm)/lib -L$(brew --prefix xz)/lib" \
  PKG_CONFIG_PATH="$(brew --prefix tcl-tk)/lib/pkgconfig" \
  ./configure --with-pydebug \
          --with-openssl=$(brew --prefix [email protected]) \
          --with-tcltk-libs="$(pkg-config --libs tcl tk)" \
          --with-tcltk-includes="$(pkg-config --cflags tcl tk)"

I didn’t want to install this Python on my local system but rather install it into a known directory. That way, I could zip it up and bundle it with the application I was packaging. To do that I specified a --prefix= argument:

...
  ./configure --prefix=/Users/patrick/project/python --with-pydebug \
  ...

After running configure, just make and install:

make -s -j2
make install

Everything was looking good until we did some testing on other machines. It worked perfectly on some, but on others, it would fail during a step that was trying to use pip to install dependencies. The problem was that pip was trying to import the ssl module, and on some of the machines the version of OpenSSL I had installed (using Homebrew) on my development machine, was not available.

We didn’t want to create a dependency on the destination machine having a specific version of a Homebrew-installed OpenSSL available. So I set off looking for the instructions for how to statically link OpenSSL with our compiled Python.

It turns out that this is not something officially supported by Python. (Someone tried to get a PR accepted, but it was eventually closed without being merged.)

But while it’s not officially supported, there are a few ways to do it.

Compile OpenSSL

The first step is to compile the version of OpenSSL you want to link to. I wanted OpenSSL 1.1.1, so I cloned the repo and checked out the latest 1.1.1 tag (OpenSSL_1_1_1s at the time of writing), and ran the following

> ./config shared no-ssl2 no-ssl3 no-comp \
  --prefix=/Users/patrick/project/openssl \
  --openssldir=/Users/patrick/project/openssl

> make
> make install

Modules/Setup.local

With OpenSSL built, it’s now time to re-compile Python but statically linked to OpenSSL this time.

The Python build system looks for a Modules/Setup.local file to allow for some customization of the build. I came across a few references to it while searching. However, I was unable to find much documentation about how you’re supposed to use it.

However, combining information found in a message on the Python bug tracker, and a StackOverflow answer, I was able to piece together the following contents for my Modules/Setup.local file (this is back in the Python source code directory, trying to compile version 3.9.x in my case).

OPENSSL=/Users/patrick/project/openssl
_ssl _ssl.c \
    -I$(OPENSSL)/include -L$(OPENSSL)/lib \
    $(OPENSSL)/lib/libssl.a \
    $(OPENSSL)/lib/libcrypto.a
_hashlib _hashopenssl.c \
    -I$(OPENSSL)/include -L$(OPENSSL)/lib \
    $(OPENSSL)/lib/libcrypto.a

Now, you need to update the configure command to point to the compiled OpenSSL from the previous setup, instead of the Homebrew installed one:

> CFLAGS="-I$(brew --prefix gdbm)/include -I$(brew --prefix xz)/include" \
  LDFLAGS="-L$(brew --prefix gdbm)/lib -L$(brew --prefix xz)/lib" \
  PKG_CONFIG_PATH="$(brew --prefix tcl-tk)/lib/pkgconfig" \
  ./configure --prefix=/Users/patrick/project/python \
          --with-pydebug \
          --with-openssl=/Users/patrick/project/openssl \
          --with-tcltk-libs="$(pkg-config --libs tcl tk)" \
          --with-tcltk-includes="$(pkg-config --cflags tcl tk)"
> make
> make install

At this point, I could just zip up the /Users/patrick/project/python directory and have a working version of Python that does not require a separate installation of OpenSSL.

PY_UNSUPPORTED_OPENSSL_BUILD

Starting in Python 3.10, there’s an “unsupported” way of specifying that you want to statically link OpenSSL . You set PY_UNSUPPORTED_OPENSSL_BUILD to static. (There’s no documentation for this, and my searches turned up almost nothing about it, but here’s the commit.)

I needed version 3.9.x for the application I was working on, so I haven’t had a chance to actually try this out.

Hand Edit setup.py

If neither if the two options above work, it could be because you’re trying to compile an older version of Python. I believe the Modules/Setup.local technique should work on older Pythons (before 3.9). However, since I haven’t actually tried, I thought I’d include another option I came across — hand editing the setup.py file.

This gist is a patch for updating the setup.py for Python 3.8. If you apply those changes and then do the configure/make/install steps, you should end up with a Python build that’s statically linked to OpenSSL.

Caveat

We ran into some issues trying to run Python (built as described above) on a Mac running Monterey if Python was built on Ventura. However, if Python was built using Monterey, it seems to run just fine on both Monterey and on Ventura.