Let's Encrypt with Hitch and Varnish (CentOS7)

Tags: hitch (1) ops (28) tls (3)

Introduction

In their own words “Let’s Encrypt is a free, automated, and open Certificate Authority. Using Let’s Encrypt services lets anyone acquire valid certificates for TLS/SSL encryption for free.”

In this tutorial, we’ll show you how to use the official certbot tool to obtain a free Let’s Encrypt TLS certificate and use it with Hitch and Varnish.

Prerequisites

To complete this guide, you’ll need a couple of things:

  • A Linux based server with either a privileged account or an account with sudo capabilities. This guide will describe the process on a CentOS7/Red Hat EL7 based system using sudo.
  • A registered domain name that you own or manage with which to use the certificate

If you don’t own a domain name yet, please take a moment to acquire one from one of the many available registrars. See Icann.org for an exhaustive list.

Then you must create an A-record with the name of the domain that points to the public IP-address of the host you’re setting up. This guide assumes that an A-record is set up and working, as the way the certificates are acquired relies on this for validation of domain name ownership.

Step 1 - Install Hitch and Varnish

You’ll need to install EPEL (Extra Packages for Enterprise Linux) to get both certbot and Hitch. Input the following command:

sudo yum install epel-release

Next, install Varnish Cache 6.0 LTS from the official Varnish Cache repository. If you prefer a manual repository setup over the script based one, follow the guide on Packagecloud.io.

curl -s https://packagecloud.io/install/repositories/varnishcache/varnish60lts/script.rpm.sh | sudo bash
sudo yum install hitch varnish

For Varnish Enterprise customers, install varnish-plus and varnish-plus-addon-ssl . This requires the plus-repositories to be set up in advance:

sudo yum install varnish-plus varnish-plus-addon-ssl

Step 2 - Add certbot passthrough VCL

With either Varnish Cache or Varnish Enterprise installed, you can now set up Varnish VCL to pass all incoming certificate server challenge requests through to certbot. This is done by routing all URLs matching the acme-challenge pattern to the certbot listener.

Create a new file /etc/varnish/letsencrypt.vcl with your favorite editor and add this configuration to it:

vcl 4.1;

backend certbot {
    .host = "127.0.0.1";
    .port = "8080";
}

sub vcl_recv {
    if (req.url ~ "^/\.well-known/acme-challenge/") {
        set req.backend_hint = certbot;
        return(pipe);
    }
}

sub vcl_pipe {
    if (req.backend_hint == certbot) {
        set req.http.Connection = "close";
        return(pipe);
    }
}

Then include the newly created letsencrypt.vcl file in your main VCL by adding the following include statement after the vcl 4.0; line in /etc/varnish/default.vcl:

include "/etc/varnish/letsencrypt.vcl";

Step 3 - Configure and start Varnish

By default, Varnish listens on port 6081. But to accept the challenge request from the Let’s Encrypt system, you need to make it listen on port 80.

Edit the Varnish Enterprise unit file with sudo systemctl edit --full varnish and edit the first -a parameter of the ExecStart variable to listen on port 80.

ExecStart=/usr/sbin/varnishd -a :80 -a localhost:8443,proxy -f /etc/varnish/default.vcl -s malloc,256m

You’re now ready to start the Varnish daemon:

sudo systemctl enable varnish
sudo systemctl start varnish

Step 4 - Prepare Hitch

To make the certificate installs with Hitch easier, add a small script to act as a renewal hook.

This script is called once for each successfully issued certificate. Create a new file /usr/local/bin/hitch-deploy-hook with your editor and paste this into it:

#!/bin/bash
# Full path to pre-generated Diffie Hellman Parameters file
dhparams=/etc/hitch/dhparams.pem

if [[ "${RENEWED_LINEAGE}" == "" ]]; then
    echo "Error: missing RENEWED_LINEAGE env variable." >&2
    exit 1
fi

umask 077
cat ${RENEWED_LINEAGE}/privkey.pem \
${RENEWED_LINEAGE}/fullchain.pem \
${dhparams} > ${RENEWED_LINEAGE}/hitch-bundle.pem

Make the script executable:

sudo chmod a+x /usr/local/bin/hitch-deploy-hook

To enable Perfect Forward Secrecy, create a Diffie Hellman Parameter file that Hitch will use. This is done using openssl:

openssl dhparam 2048 | sudo tee /etc/hitch/dhparams.pem

Verify that Hitch is set up with the correct backend in /etc/hitch/hitch.conf:

backend = "[localhost]:8443"

Last, enable the systemd service:

sudo systemctl enable hitch

Step 5 - Install and run certbot

The certbot client is installable through the EPEL repository we’ve already configured, so install it now with yum:

sudo yum install certbot

Now we have everything in place to request a certificate from Let’s Encrypt. Use this certbot command to request a certificate:

sudo certbot certonly --standalone --preferred-challenges http \
--http-01-port 8080 -d example.com -d www.example.com \
--deploy-hook="/usr/local/bin/hitch-deploy-hook" \
--post-hook="systemctl reload hitch"

The first time you use certbot, it will ask for your email address and to accept the terms of service. Then the certificate will be obtained after the challenges are completed.

You also need to start the certbot-renew timer, which handles automatic certificate renewals once per day:

sudo systemctl enable certbot-renew.timer
sudo systemctl start certbot-renew.timer

The renewal service certbot-renew automatically reuses the settings used with the certbot command and these are saved in the folder /etc/letsencrypt/renewal/.

Step 6 - Start Hitch

You should now have a Hitch bundle consisting of the private key, the CA chain, and the pre-generated Diffie Hellman parameter file.

Add the resulting pem-file to your /etc/hitch/hitch.conf using your editor:

pem-file = "/etc/letsencrypt/live/example.com/hitch-bundle.pem"

Now start the Hitch daemon:

sudo systemctl start hitch

Conclusion

Hitch should now start. After opening a browser to the configured hostname, you can confirm that the connection is successfully encrypted using TLS. The certbot renewal process will ensure your certificates are automatically updated and that Hitch will reload when a new certificate is fetched.