Simplest HTTPS setup for your new Linux Server: Nginx Reverse Proxy+ Letsencrypt+ AWS Cloud + Docker

I needed a set of steps that would help me setup a server for any new project.

My projects would start by pointing a domain to my web-app running on a Linux VM in cloud(AWS, Azure, Oracle/OCI etc). Traffic should be served over https, with free Letsencrypt SSL certificates. Setup certificate auto-renewal. Use Nginx reverse proxy.

So I wrote down these minimum, easy to follow steps to get it done in less than 5 min ⌛. This helped me and anyone in my team quickly setup a completely new backend project. I have tested this setup with both Cloudflare and Namecheap DNS.

Hope this will save your precious time 😀… consider hitting 👏 at the end. !! You can appreciate my efforts… buy me a tea ☕.

Before starting, I’ll assume you already have following things ready:

1. You have a server/VM (doesn’t have to be AWS) with a public IP.
2. The ports 80 and 443 are open for all TCP inbound traffic. (80 for http and 443 for https)
3. Related to above, make sure you have tested it’s really open as some OS come with firewall by default and you need to open it to accept traffic on above ports.
4. Docker and docker-compose is installed (for ubuntu use this doc, or follow these minimal steps)
5. DNS setup: domain/sub-domain points to the IP address of server(check respective setup instructions for your domain registrar or DNS provider)

For any issues, jump to FAQ section at end.

*If you intend to get the job done quick, without getting into the details, just jump over to this github repo and follow the minimal steps mentioned there.

**Note: Now compose is a part of docker and hence all docker-compose commands can be written as docker compose i.e. without the - . To be able to run using docker-compose, the compose standalone should be installed first.

Step-1: Create project structure

Either clone this repo or follow the steps. If you clone the repo, it already has all files with default content.

We need to create files and folder structure like this:

nginx_https_docker
  |
  |-- config
  |     |-- nginx.conf
  |
  |-- docker
  |     |-- nginx.Dockerfile
  |
  |-- docker-compose.yaml
  |-- docker-compose-le.yaml

Run these commands

mkdir nginx_https_docker && cd nginx_https_docker
mkdir config
mkdir docker
touch config/nginx.conf docker/nginx.Dockerfile
touch docker-compose.yaml docker-compose-le.yaml

• nginx.conf : config for nginx reverse proxy
• nginx.Dockerfile : for nginx docker container
• docker-compose.yaml : compose file for our nginx server. Put any other services that are part of your application here.
• docker-compose-le.yaml : for the first time setup and certificate auto-renewal

Creating the files one by one can be useful if you already have your project repo with code. Then you can just add stuff to your docker-compose etc. and things should work. Or else, you can create the setup separately and then copy stuff from new setup to your existing codebase. I prefer latter.

Step-2: Configure for first time setup

Open config/nginx.conf and add below config.
**Replace test.leangaurav.devwith the domain you wish to configure. It can be subdomain like mine or apex domain.

server {
    listen 80;
    listen [::]:80;
    server_name test.leangaurav.dev;
    
    location ~ /.well-known/acme-challenge {
        allow all;
        root /tmp/acme_challenge;
    }
}

We need this config only for the first time to let certbot be able to get ssl certificates. Certbot places a token to validate the certificate request in the directory /.well-known/acme-challenge. Then certbot validates your domain by requesting the token at <yourdomain>/.well-known/acme-challenge/<some token>
Hence we have setup the location block to serve the challenge requests from /tmp/acme_challenge. This same location we will be mounting into our nginx container as well later.

In the docker-compose.yml paste below content as it is. For existing setup, just add the nginx service part to it. Remember to change the network name app with whatever you have. Otherwise nginx won’t be able to communicate with your application.

version: "3.3"

services:
  nginx:
    container_name: 'nginx-service'
    build:
      context: .
      dockerfile: docker/nginx.Dockerfile
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./config:/config
      - /etc/letsencrypt:/etc/letsencrypt:ro
      - /tmp/acme_challenge:/tmp/acme_challenge
    networks:
      - app
    restart: always

networks:
  app:
    driver: bridge

Now we need the docker/nginx.Dockerfile. This is pretty simple. Feel free to change the nginx version later.

FROM nginx:1.21.1-alpine
RUN rm /etc/nginx/conf.d/default.conf
COPY /config/nginx.conf /etc/nginx/conf.d

Last thing we need is the docker-compose-le.yaml . Again, here you need to replace your.email@email.com with the email where you wish to receive certificate renewal notifications and also replacetest.leangaurav.com with your domain name.

The expand option is optional. It helps if you add another domain to the same command at a later point.
Multiple domains can be added with a comma separator like test.leangaurav.com,about.leangaurav.com without any spaces.

version: "3.3" 
services:
  letsencrypt:
    container_name: 'certbot-service'
    image: certbot/certbot:v1.17.0
    command: sh -c "certbot certonly --expand --webroot -w /tmp/acme_challenge -d test.leangaurav.com --text --agree-tos --email your.email@email.com --rsa-key-size 4096 --verbose --keep-until-expiring --preferred-challenges=http"
    entrypoint: ""
    volumes:
      - "/etc/letsencrypt:/etc/letsencrypt"
      - "/tmp/acme_challenge:/tmp/acme_challenge"
    environment:
      - TERM=xterm

If you noticed, we have done a bind mount for the volume /tmp/acme_challenge in both the nginx and certbot docker-compose .

The two dockerfiles:
1. docker-compose.yml: contains nginx service. you can add other services which need to share network with nginx, or copy this service into your main docker-compose.yml
2. docker-compose-le.yml: runs the certbot image with our config. this will be needed later in the auto-renew step.

Step-3: The first run

From the nginx_https_docker folder on your server, run the command

docker compose up --build nginx

this will start our nginx server. Leave it running and you will also be able to see the logs here. Probably try accessing your server using the IP address or using http://yourdomain. Make sure you use http when using both. This should show you the nginx home page or a 404 not found page.

If you don’t see this, check the common issues section about this and how to fix this.

If at this point you get an error like
ERROR: for nginx Cannot create container for service nginx: Conflict. The container name “/nginx-service” is already in use by container “31bd6a36f4ecc7a43f1ea991c7686ae60a4ddb13c749b9c5b848cbacb69e89d5”. You have to remove (or rename) that container to be able to reuse that name.
ERROR: Encountered errors while bringing up the project.

then run this command to remove old container:
docker stop nginx-service && docker container rm nginx-service

Now in another terminal window go to the folder nginx_https_docker and run the letsencrypt container with the below command

docker compose -f docker-compose-le.yaml up --build

if things work well you will see something like this

Successfully received certificate.
letsencrypt_1  | Certificate is saved at: /etc/letsencrypt/live/yourdomain.com/fullchain.pem
letsencrypt_1  | Key is saved at:         /etc/letsencrypt/live/yourdomain.com/privkey.pem
letsencrypt_1  | This certificate expires on 2021-10-20.
letsencrypt_1  | These files will be updated when the certificate renews.
letsencrypt_1  |
letsencrypt_1  | NEXT STEPS:
letsencrypt_1  | - The certificate will need to be renewed before it expires. Certbot can automatically renew the certificate in the background, but you may need to take steps to enable that functionality. See https://certbot.org/renewal-setup for instructions.
letsencrypt_1  |
letsencrypt_1  | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
letsencrypt_1  | If you like Certbot, please consider supporting our work by:
letsencrypt_1  |  * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
letsencrypt_1  |  * Donating to EFF:                    https://eff.org/donate-le
letsencrypt_1  | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Generated certificates will be available under `/etc/letsencrypt/<your domain name>` directory on your machine.

The above container stops itself after installing the certificates. For the other container running in the first terminal, stop it by pressing Ctrl+C or CMD+C.

Step-4: Enable https

Open the config/nginx.conf again and delete everything. Paste below contents and remember again to remember to replacetest.leangaurav.com with your domain name at all the four places.

server {
    listen 80;
    listen [::]:80;
    server_name test.leangaurav.dev;
    location / {
        return 301 https://$host$request_uri;
    }
    location ~ /.well-known/acme-challenge {
        allow all;
        root /tmp/acme_challenge;
    }
}
server {
    listen 443 ssl;
    listen [::]:443 ssl http2;
    server_name test.leangaurav.dev;
    ssl_certificate /etc/letsencrypt/live/test.leangaurav.dev/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/test.leangaurav.dev/privkey.pem;
}

Now run the nginxcontainer again but with the -d flag this time.

docker compose up --build -d nginx

Now navigate to your domain and you should find an nginx 404 page served over https.

Congratulations 🎉. You are 99% done.

Step-4.1: Proxy requests(optional)

Next step would be telling nginx to proxy pass. See the bold items below

upstream default_app {
    server webserver:8000;
}
server {
    listen 80;
    listen [::]:80;
    server_name test.leangaurav.dev;
    
    location / {
        return 301 https://$host$request_uri;
    }
    location ~ /.well-known/acme-challenge {
        allow all;
        root /tmp/acme_challenge;
    }
}
server {
    listen 443 ssl;
    listen [::]:443 ssl http2;
    server_name test.leangaurav.dev;
    
    ssl_certificate /etc/letsencrypt/live/test.leangaurav.dev/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/test.leangaurav.dev/privkey.pem;
    
    location / {
        proxy_pass http://default_app;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_redirect off;
    }
    location /static/ {
        alias /static/;
    }
}

The first block sets up a set of upstream request handler by the tag default_app . One interesting thing here is the webserver:8000 part. Since I’m running nginx inside a container, I wanted it to access another container which actually handles any incoming request. The container was tagged with the name webserver and it handles requests on port 8000 .

For this to work, you need to ensure that both the nginx and the webserver containers are part of the same network. There’s another way to handle this as well. If your other container had it’s ports mapped to host or specify a port mapping, then you can use localhost:8000 as well.

The second server now contains a location block specifying the proxy settings. And also another separate location block for handing static.

All these things can be customized according to your needs. What I have provided is just an example.

Step-5: Setup certificate auto-renewal

One last and important step is to setup the cron-job to renew the ssl certificates.

The cron job should do two things:

1. Run the letsencrypt certbot container.
   docker compose -f <absolute path to folder>/docker-compose-le.yaml up
2. Tell nginx server to reload newly installed certificates
   docker exec -it nginx-service nginx -s reload

Couple of things to notice:

• You need to give an absolute path to the docker-compose-le.yaml.
• The container name is nginx-service which is the same as what was used in docker-compose.yaml If you did any changes there, use the same name here.

The command to launch the letsencrypt container in the docker-compose-le.yaml had the flag — keep-until-expiring. This will renew the certificates only when they are about to expire. Letsencrypt issues certificates valid for 90 days and certificates are marked for renewal 30 days before expiry.

We will run the crontab on 1st of every month at 12:00 GMT. We want nginx to reload only if renewal happened. I haven’t yet spent time to get that working, so we’ll just reload irrespective.

For the crontab, we will combine the two commands into one. Run the crontab -e command and paste the below thing (adding the correct path). Save and exit.

0 12 1 * * docker compose -f <absolute path to folder>/docker-compose-le.yaml up && docker exec -it nginx-service nginx -s reload

To understand the cron tab, check these:
- https://crontab.guru/#0_0_*_*_0
- https://stackoverflow.com/questions/350047/how-to-instruct-cron-to-execute-a-job-every-second-week

Optinally you can pipe the crontab logs to a specific file for debugging by adding this >> <path to log file> 2>&1 to thee

So complete crontab can look like this

0 12 1 * * docker compose -f /home/ubuntu/nginx_https_docker/docker-compose-le.yaml up && docker exec -it nginx-service nginx -s reload >> /home/ubuntu/crontab.log 2>&1

Thats it you are all set !!

Consider hitting 👏. You can find me on Linkedin and say 👋. Saved a few days of effort ? Buy me a couple coffee ☕

Common Issues

Re-directed too many times when using cloudflare DNS.

While installing certificate in steps 1 and 2 the cloudflare DNS setting should be set to Flexible.
After installation it can be either Full or Full(strict), otherwise you get a too many redirect error.

Nginx home page or an nginx 404 error page doesn’t show up.

There can be multiple reasons. Go step by steps:

1. check if the server IP address is public or not and also ports 80 and 443 should be open.
   Test this by running the nginx server with default config and then visiting http://<server ip>:80. This should display the nginx default home page
   OR
   Just run a simple python file server on port 80 using sudo python3 -m http.server 80 and then access http://<server ip>:80 . Sudo is needed to bind to port 80.
2. Make sure you have already added an A record do your DNS provider pointing to the server where you are setting up nginx .
3. Check firewall of the instance. For example this helped solve my problem on Oracle cloud
   Basically ran following commands

sudo iptables -I INPUT -m state --state NEW -p tcp --dport 80 -j ACCEPT
sudo iptables -I INPUT -m state --state NEW -p tcp --dport 443 -j ACCEPT
sudo netfilter-persistent save

Exec Format error

 ⠋ letsencrypt The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested  0.0s
Attaching to certbot-service
certbot-service  | exec /bin/sh: exec format error
certbot-service exited with code 1

Replace the certbot image tag in docker-compose-le.yaml with platform specific build like:

image: certbot/certbot:arm64v8-v1.32.2

Unable to connect to upstream service from nginx with network_mode host

If you use host network mode in nginx container and wish to connect to another service from nginx, then use 127.0.0.1 instead of localhost

upstream default_app {
    server 127.0.0.1:8000;
}

See this answer for more details.

Useful links

• https://stackoverflow.com/questions/66638368/how-to-do-auto-renewal-of-tls-certificates-with-certbot#:~:text=By%20running%20%22docker%2Dcompose%20%2D,it%20is%20about%20to%20expire
• https://certbot.eff.org/docs/using.html
• https://certbot.eff.org/docs/install.html#running-with-docker