Ansible and Let's Encrypt for Multi-Tenancy Applications

Idea:

Let's Encrypt can issue certificates for free, we use ansible to configure, provision and deploy on our servers, let's use them together to get certificates working!

Problem:

We were using the geerlingguy.apache role to configure apache and our virtual hosts, and with the geerlingguy.certbot role we were able to get the Let's Encrypt certbot installed. However, configuring via the magic --apache certbot option wasn't something we were comfortable with. With the sub domains we needed to configure, and the order in which roles needed to take place to allow easy configuration & deployment, it meant that we'd felt the best option was writing our own role to handle the certificates and configuring the virtual hosts for them.

In our role we store an array of sites that will have certificates, as well as some relevant information for them that we use later on when writing the configuration for the site to the correct location (using jinja templating within Ansible)

At a basic level, the array of sites looks like this:

ansible/vars/letsencrypt-sites.yml:

letsencrypt_certs_sites:
    - domain: myawesomedomain.tld
      documentroot: "/var/www/myapp/web"
      # more variables here, though they're not relevant to this post

Step 0: Use geerlingguy's role to configure apache

This is a prerequisite for this task, the role I've written assumes that apache is installed and ready to work. We don't use geerlingguy.apache to configure virtual hosts, we just use it to install apache and get it up and running.

Step 1: Add a default vhost

This is the only vhost that's set up to respond on port 80. It has a docroot of /var/www/html (important to note for the next step)

It also then enables that vhost within apache.

Step 2: Generate Let's Encrypt certificates for sites

The command we use to generate certificates uses that default vhost for verification. This is important as at this point, the subdomain CNAME has been setup to point to our server, but the vhost doesn't (yet) exist, so it'll respond with the default virtual host.

The ansible task is:

- name: Generate Let's Encrypt certificates for sites
  shell: "{{ certbot_dir }}/certbot-auto certonly -a webroot --webroot-path=/var/www/html --domain {{ item.domain }} --agree-tos --email \"team@sysadminincorporated.tld\" --non-interactive"
  with_items: letsencrypt_certs_sites

This loops through the sites we have and configures a certificate for them.

Step 3: Add vhosts in for SSL enabled sites

The role then uses a template for the virtual hosts to configure them with the certificate paths, doc roots, and any other relevant options for our application. Effectively, we'll have lots of different domains & virtual hosts being served by the same docroot for our application.

It then enables the vhosts for SSL enabled sites.

This means the site will now be properly handled by our app as the virtual host will no longer mean that the default vhost handles it.

Step 4: Auto renew the certificates

We have a cron enabled to run at a defined time (it runs every day sometime in the middle of the night when the system is at the lowest activity period) that calls the renew option for certificates. They won't renew unless they're approaching expiry time, and we use the following command to ensure that apache is restarted when a certificate renews:

{{ certbot_dir }}/certbot-auto renew --quiet --no-self-upgrade --renew-hook 'service apache2 restart'

With this as well, if no certificates are renewed, then apache won't restart.

The overall process of this role is:

1. Add and enable default vhost in to handle certificate verification via webroot
2. Generate certificates for the domains we need
3. Add and enable vhosts with SSL for the domains we need
4. Disable and remove the default vhost as we no longer need it (until the next certificate creation is required and then it'll be added back in by the role)

With these options, we can configure a site for SSL in our ansible configuration, and from there, deploy the changes and with just a few lines of config, have a site that has SSL enabled and will work with our multi-tenanted application.

Joy!