Fixing Letsencrypt's `certbot` Authentication Error in GitLab

by Johannes Blaschke
in Coding

For months now, I've been dealing with an annoying problem relating to my GitLab instance: I could not renew my TLS certificate without disabling GitLab's http-to-https forwarding. certbot would fail with the error:

Failed authorization procedure

What made this hard to figure out is that:

  1. It worked fine before (but since I wasn't watching the logs, I didn't know which precise update broke everything).
  2. Nobody else seemed to be having the same problem.

But now I fixed it (finally)!

Background

First some background: I use GitLab ears now. Back in the pre-10.5 days, you had to set up TLS manually yourself, and supply GitLab with a certificate/key pair. Thus, I had Letsencrypt's certbot every day as a cron job. And GitLab's nginx server would host certbot's challenge. This was easy to set up, add:

nginx['custom_gitlab_server_config'] = "location ^~ /.well-known/ { root /var/www/letsencrypt; }"

to your /etc/gitlab/gitlab.rb.

Problem

But ever since version 10.5, this stopped working (well, not completely: if you bypassed https it would still work but I was not happy with that): GitLab's nginx would not allow access to ./well-known.

Solution

I stumbled accross the solution today: let GitLab manage the certbot. Remove the line above, and add:

letsencrypt['enable'] = true
letsencrypt['contact_emails'] = ['your@favorite.email']

to your /etc/gitlab/gitlab.rb. You can save yourself the cron job by also adding:

letsencrypt['auto_renew'] = true
letsencrypt['auto_renew_hour'] = "12"
letsencrypt['auto_renew_minute'] = "30"
letsencrypt['auto_renew_day_of_month'] = "*/15"

Presto!

Note

Don't foget to remove any external references to your certificates using the nginx['ssl_certificate'] settings.