Easy Secure Web Serving with OpenBSD’s acme-client and Let’s Encrypt

As recently as just a few years ago, I hosted my personal website, VPN, and personal email on a computer running OpenBSD in my basement. I respected OpenBSD for providing a well-engineered, no-nonsense, and secure operating system. But when I finally packed up that basement computer, I moved my website to an inexpensive cloud server running Linux instead.

Linux was serviceable, but I really missed having an OpenBSD server. Then I received an email last week announcing that the StartSSL certificate I had been using was about to expire and realized I was facing a tedious manual certificate replacement process. I decided that I would finally move back to OpenBSD, running in the cloud on Vultr, and try the recently-imported acme-client (formerly “letskencrypt”) to get my HTTPS certificate from the free, automated certificate authority Let’s Encrypt.

Why You Should Get Your Certificates from ACME

Let’s Encrypt uses the Automated Certificate Management Environment protocol, more commonly known as ACME, to automatically issue the certificates that servers need to identify themselves to browsers. Prior to ACME, obtaining certificates was a tedious process, and it was no surprise when even high-profile sites’ certificates would expire. You can run an ACME client periodically to automatically renew certificates well in advance of their expiration, eliminating the need for the manual human intervention that can lead to downtime.

There are plenty of options for using ACME on your server, including the Let’s Encrypt-recommended Certbot. I found acme-client particularly attractive not just because it will ship with the next release of OpenBSD, but also because it’s well-designed, making good use of the privilege separation technique that OpenBSD pioneered as well as depending only on OpenBSD’s much-improved LibreSSL fork of OpenSSL.

Bootstrapping

To follow along with me, you’ll need OpenBSD. You can use the 6.0 release and install acme-client. If you’re feeling adventurous and are willing to maintain a bleeding-edge system, you can also run the -current branch, which already has acme-client.

If you do the smart thing and choose to use the release version, you’ll need to do a little extra setup after installing acme-client to align with the places things are in -current:

# mkdir -p /etc/acme /etc/ssl/acme/private /var/www/acme
# chmod 700 /etc/acme /etc/ssl/acme/private

And whenever you use acme-client, you’ll need to specify these paths, e.g.:

# acme-client \
        -C /var/www/acme \
        -c /etc/ssl/acme \
        -k /etc/ssl/acme/private/privkey.pem \
        -f /etc/acme/privkey.pem \
        www.example.com

Everything will work as advertised otherwise.

A note before we get started: If you’re new to OpenBSD, you owe it to yourself to get familiar with man(1). OpenBSD has amazingly good documentation for just about everything, and you can access it all by typing e.g. man httpd or man acme-client. Everything in this article came from my reads of these manpages. If you get stuck, try man first!

ACME will use a web server as part of its challenge-response process with the Let’s Encrypt service. To get this started, we’ll build out a basic /etc/httpd.conf based on our readings of httpd.conf(5) and acme-client(1):

server "default" {
        listen on * port 80

        location "/.well-known/acme-challenge/*" {
                root "/acme"
                root strip 2
        }
}

This is enough to start up a basic web server that will serve the challenge responses that acme-client will produce. Now, start httpd using rcctl(8):

# rcctl enable httpd
# rcctl start httpd

Getting Your First Certificate

Once httpd is up and running, you’re ready to ask acme-client to perform all that heavy lifting that you used to have to do by hand, including:

  1. Generating your web server private and public key
  2. Giving your public key to the certificate authority
  3. Proving to the certificate authority that you’re authorized to have a certificate for the domains you’re requesting
  4. Retrieving and installing the signed certificate

You can do all of this with a single command:

# acme-client -vNn example.com www.example.com

man acme-client will explain all that’s going on here:

  1. -v says we want verbose output, because we’re curious.
  2. -N asks acme-client to create the private key for our web server, if one does not already exist.
  3. -n asks acme-client to create the private key for our Let’s Encrypt account, if one does not already exist.
  4. example.com and www.example.com are the domains where we want our certificate to be valid—note that our web server must be reachable via those names for this process to work!

If this worked correctly, there will be some new keys and certificates on your system ready to be used to serve HTTPS.

Using the New Certificates with httpd

To get httpd working with our new certificates, we just need to expand /etc/httpd.conf a little:

server "default" {
        listen on * port 80
        listen on * tls port 443

        tls certificate "/etc/ssl/acme/fullchain.pem"
        tls key "/etc/ssl/acme/private/privkey.pem"

        location "/.well-known/acme-challenge/*" {
                root "/acme"
                root strip 2
        }
}

The three new lines above add a new HTTPS listener to our configuration, telling httpd where to find the certificate it should present and the private key it should use.

Once this configuration is in place, ask httpd to reload its configuration file:

# rcctl reload httpd

At this point, your server should be online with a valid Let’s Encrypt certificate, serving HTTPS—though giving you an error page, because httpd is not yet configurated to serve any content. That bit is left as an exercise for the reader. (Consult httpd.conf(5) for further help there.)

Automating Yourself Out of a Certificate Renewal Job

By far the best part about ACME is that it can be easily configured to automatically renew your certificates before you notice they’re about to expire. Note that acme-client is written so that you simply need to run it periodically. Once the certificates are 30 days from expiration, it will get a fresh signature from Let’s Encrypt.

Making this happen is as simple as dropping the following into /etc/daily.local (cf. daily(8)):

# renew Let's Encrypt certificate if necessary
acme-client example.com www.example.com
if [ $? -eq 0 ]
then
        rcctl reload httpd
fi

And now acme-client will now run every night (by default at 1:30 a.m.) and renew your certificate when necessary.

Further Reading

This is a simple configuration, but it’s enough to run my web site and give me painless HTTPS that scores an A out-of-the-box on SSL Labs’ server test. I added a few lines to /etc/httpd.conf to serve the static content on my site, and I was done.

If you have a more complex configuration, though, chances are that httpd and acme-client are up to the task. To find out all they can do, read the man pages:

If you want to know more about OpenBSD in general, check out the comprehensive OpenBSD FAQ.

Happy secure serving!

 
Conversation
  • andrew says:

    I’ve tried this many times… followed the man pages and tested on openbsd 6.0, current, and freebsd 10.3, and 11rc3. Every single time I end up with acme-client returning 1. BAD EXIT. the error starts with “bad response
    acme-client: transfer buffer: [{ “type”: “http-01”, “status”: “invalid”, “error”: { “type”: “urn:acme:error:unauthorized”, “detail”: “Invalid response from http:……….
    I’m literally copying and pasting .. replacing ‘example.com’ with my domain. dns is setup, i’ve tested the httpd webservers and they are serving html properly. I’m really confused

    • Matt Behrens says:

      I’m not sure what that message means. Is there more of it? It looks like the Let’s Encrypt server might be rejecting your request for some reason.

      • andrew says:

        This is the full output – with some edits to protect domain privacy (ip add and hostname and keys were changed)
        # acme-client -vNn domain1.com
        acme-client: /etc/ssl/letsencrypt/private/privkey.pem: domain key exists (not creating)
        acme-client: /etc/letsencrypt/privkey.pem: account key exists (not creating)
        acme-client: https://acme-v01.api.letsencrypt.org/directory: directories
        acme-client: acme-v01.api.letsencrypt.org: DNS: 23.73.89.18
        acme-client: https://acme-v01.api.letsencrypt.org/acme/new-authz: req-auth: domain1.com
        acme-client: /var/www/letsencrypt/g68iIGTSfnJtLOjADh1Nmk6ZY3gCqOEcCsdQfBFO54Q: created
        acme-client: https://acme-v01.api.letsencrypt.org/acme/challenge/dIseKlojcVd4gwr7WpyLDTkYk2-D916XKww5rAg3RtY/273421714: challenge
        acme-client: https://acme-v01.api.letsencrypt.org/acme/challenge/dIseKlojcVd4gwr7WpyLDTkYk2-D916XKww5rAg3RtY/273421714: status
        acme-client: https://acme-v01.api.letsencrypt.org/acme/challenge/dIseKlojcVd4gwr7WpyLDTkYk2-D916XKww5rAg3RtY/273421714: bad response
        acme-client: transfer buffer: [{ “type”: “http-01”, “status”: “invalid”, “error”: { “type”: “urn:acme:error:unauthorized”, “detail”: “Invalid response from http://domain1.com/.well-known/acme-challenge/g68iIGTSfnJtLOjADh1Nmk6ZY3gCqOEcCsdQfBFO54Q: \”\u003c!DOCTYPE html\u003e\n\u003chtml\u003e\n\u003chead\u003e\n\u003cmeta http-equiv=\”Content-Type\” content=\”text/html; charset=utf-8\”/\u003e\n\u003ctitle\u003e404 Not Found\u003c/title\u003e\n\””, “status”: 403 }, “uri”: “https://acme-v01.api.letsencrypt.org/acme/challenge/dIseKlojcVd4gwr7WpyLDTkYk2-D916XKww5rAg3RtY/273421714”, “token”: “g68iIGTSfnJtLOjADh1Nmk6ZY3gCqOEcCsdQfBFO54Q”, “keyAuthorization”:”g68iIGTSfnJtLOjADh1Nmk6ZY3gCqOEcCsdQfBFO54Q.tRsG8bKFioCG8RCz2KNR7yf14w8YNej0kP0_NmQEc4Q”, “validationRecord”: [ { “url”: “http://domain1.com/.well-known/acme-challenge/g68iIGTSfnJtLOjADh1Nmk6ZY3gCqOEcCsdQfBFO54Q”, “hostname”: “domain1.com”, “port”: “80”, “addressesResolved”: [ “23.26.42.19” ], “addressUsed”: “23.26.42.19” } ] }] (1037 bytes)
        acme-client: bad exit: netproc(69593): 1

        • Matt Behrens says:

          It looks like your server returned a 403 Unauthorized response when the Let’s Encrypt server challenged it, but with a 404 message body? If your httpd.conf is correct, make sure that your server is actually serving domain1.com.

          • Alen says:

            Does /var/www/acme have to be owned by the www user?

        • Alen says:

          Hey @andrew! Did you ever manage to get this sortet out? I’m having the same issues, and frankly I don’t know why.

          • andrew says:

            Yes I did. give me a couple hours to dig up my notes. I did get it working reliably, it was something minor but I just don’t remember off hand

        • Alen says:

          @andrew – Cool, let me know! Eager to get this going.

        • Alen says:

          @andrew – Did you manage to get a hold of your notes?

          • andrew says:

            this is what i’ve got.
            i think it was just an order of operations thing. – send me your email address and I can help you sort this out over email if you still need assistance.

            # vi /etc/httpd.conf
            >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
            server “default {
            listen on * port 80
            # listen on * tls port 443
            # tls certificate “/etc/ssl/letsencrypt/fullchain.pem”
            # tls key “/etc/ssl/letsencrypt/private/privkey.pem”

            root “/htdocs/”
            location “/.well-known/acme-challenge/*” {
            root “/letsencrypt/”
            root strip 2
            }
            }
            >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

            ***
            restart httpd
            then untar the acme-client.tgz from https://kristaps.bsd.lv/acme-client/snapshots/acme-client.tgz
            ***
            # cd acme-client….
            # make install
            # mkdir /etc/ssl/letsencrypt/
            # mkdir /etc/ssl/letsencrypt/private
            # mkdir /var/www/letsencrypt
            #
            # acme-client -c /etc/ssl/letsencrypt/ -vNn DOMAINNAME.TLD?

  • Nehemiah I. Dacres says:

    Do u not know about digital ocean? They host freebsd at $5 a month.

    • Matt Behrens says:

      Vultr also hosts my server at $5/month, and has first-class support for bring-your-own-ISO hosting. I’ve been pretty happy with them.

  • James Pole says:

    Just out of interest, should your daily.local file call acme-client with the same flags (i.e. for the acme directories when using it on the 6.0 release) as you originally called it with? I know this is only a temporary band-aid until 6.1 comes out but thought it was worth mentioning.

    • Matt Behrens says:

      Yes, definitely. If you don’t, it won’t look in the proper places for certificates &c.

      It looks like 6.1 may end up with a configuration file for acme-client, which should simplify a lot of this.

  • Sean says:

    Hello Matt,
    Thanks for your post. I got my OpenBSD 6.0 httpd serving up https. But I can’t seem to figure out how to serve https by default for select servers. I have to type “https://…” specifically or else I get http by default. I have several virtual servers, but serving https is not necessary for all of them. I tried eliminating the listen http directive, and tried reordering the servers. I even tried “block return 301 “https://…”, to try and redirect for the secure sites, but none of this worked. Any suggestions? I am stumped. Thanks.

  • Psypro says:

    Since OpenBSD 6.1 commands have changed. I tried:

    #acme-client -vADF x.com

    acme-client: /etc/ssl/private/x.com.key: domain key exists (not creating)
    acme-client: /etc/acme/letsencrypt-privkey.pem: account key exists (not creating)

    port”: “80”, “addressesResolved”: [ “2.2.2.2” ], “addressUsed”: “2.2.2.2” } ] }] (1045 bytes)
    acme-client: bad exit: netproc(44389): 1

    My understanding
    a)My server box is behind a firewall, I have set up a normal portforward so the web server can be reached from internett. acme-client resolves the domian, so I guess the firewall setup works.

    b) I have tried several httpd configs, from manual and from this page. With rcctl restart httpd.

    server “x.com” {
    # alias “www.x.com”
    listen on * port 80
    # listen on * tls port 443
    # tls certificate “/etc/ssl/x.com.fullchain.pem”
    # tls key “/etc/ssl/private/x.com.key”
    location “/.well-known/acme-challenge/*” {
    root “/owncloud/acme”
    root strip 2
    }
    root “/owncloud”
    }

    c
    acme-client.conf

    Stander setup no touched.

    domain x.com {
    alternative names { x.com }
    domain key “/etc/ssl/private/x.com.key”
    domain certificate “/etc/ssl/x.com.crt”
    domain full chain certificate “/etc/ssl/x.com.fullchain.pem”
    sign with letsencrypt
    challengedir “/var/www/owncloud/acme”
    }

  • Psypro says:

    I think I found my problem

    No content was served, since httpd.conf, did not offer . This work for owncloud, which serves php. Might need to uncommen with # 443 port listening, and ssl stuff until you have key and such from running acme-client

    ” directory auto index
    location “/*.php*” {
    fastcgi socket “/run/php-fpm.sock”
    }”

    It worked.

    My httpd.conf

    server “x.com” {
    # alias “www.x.com”
    listen on * port 80
    listen on * tls port 443
    tls certificate “/etc/ssl/x.com.fullchain.pem”
    tls key “/etc/ssl/private/x.com.key”
    location “/.well-known/acme-challenge/*” {
    root “/owncloud/acme”
    root strip 2
    }

    # The webroot folder (where the static content will be served from).
    # This is in a chroot under /var/www/
    root “/owncloud”
    directory auto index
    location “/*.php*” {
    fastcgi socket “/run/php-fpm.sock”
    }

    }

  • ncali-it says:

    Does not work at all with OpenBSD 6.0. No matter what I do I get the following:

    acme-client: /etc/acme/privkey.pem: account key exists (not creating)
    acme-client: /etc/ssl/acme/private/privkey.pem: domain key exists (not creating)
    acme-client: /etc/ssl/acme/private/privkey.pem: PEM_read_PrivateKey
    2256225808:error:0906D06C:PEM routines:PEM_read_bio:no start line:/usr/src/lib/libcrypto/crypto/../../libssl/src/crypto/pem/pem_lib.c:704:Expecting: ANY PRIVATE KEY
    acme-client: bad exit: keyproc(33902): 1

    Followed your guide verbatim on fresh install of OpenBSD 6.0. Used both acme-client.tgz and the portable version. I can confirm the nginx is working just fine and serving a test file out of the acme-challenge directory. Other than that, I’m mystified why LE is seemingly impossible to install on OpenBSD.

    • Matt Behrens says:

      You might want to check out OpenBSD 6.1, which shipped recently—everything’s included. A few things have changed, though, and I haven’t updated this post to reflect that.

      It looks like your domain key isn’t right somehow, but I’ve never seen that specific error. It might be worth trying to remove the domain key file and trying to create it again.

      • ncali-it says:

        Thanks. I did delete the domain key file and watched it hang when trying to recreate it. Deleting it again, and then running it again, I got to a different error.

        There is definitely something wonky with LE and 6.0. Will try 6.1 instead.

  • Comments are closed.