# Nginx-Rest Proxy SSL Cert Setup

Overview

What this replaces

Previously, backend Spring Boot servers used a long-lived self-signed certificate. nginx pinned that exact cert file via and used to match its CN. Every cert rotation required touching nginx on every front-end machine, so rotation effectively never happened.

What this system does

<span>Each backend machine issues and renews its own Let's Encrypt certificates using certbot (DNS-01 challenges via Route53), running entirely in Docker alongside the Spring app. Certificates hot-reload into the running JVM with no restart. nginx trusts the public CA root and verifies a shared "pool alias" name, so </span>**nginx configuration never changes again**<span> — adding backends, rotating certs, and promoting environments require zero nginx cert changes.</span>

# Full How to Guide

The steps taken to get this setup are outlined below, note that I tested this on a real beta.web.tasking.rpp.cybercrucible.com beta domain and environment, the examples included are for this environment but replace with the actual values for wherever the new setup is. I attached a full write up from claude on the process that goes into more details if desired, this should be in the sidebar on this page to download.

1. Create new AWS IAM user and save the credentials. Assign this user the already created policy called "acme-dns01-cybercrucible", this is a tightly scoped policy used for this purpose only. Making a new user per machine would be a good practice
2. Make sure the domains you wants are created in AWS Route53, no record gets made for the alias used in certs such as "upstream.beta.web.tasking.rpp.cybercrucible.com"
3. Springboot application.properties update. Get rid of all the "server.ssl.key-store..." values, the new docker setup has environment variables for SPRING\_SSL\_BUNDLE\_PEM\_ACME\_KEYSTORE\_CERTIFICATE and related fields. Deploy to the machine
4. SSH to the machine where the upstreams are, this would be azul for rest-web for example, and cd to the docker directory for the upstream. Make new files listed here, this is the sample layout for rest-web upstream in the docker dir: 
    1. compose.yml
    2. certbot-deploy-hook.sh
    3. .env
    4. aws.env
    5. rest-web.war
    6. start.sh
    7. rest-mongo-keystore.jks
5. Edit the .env file, this is for cert variable names: 
    1. CERT\_NAME=rest-web-beta  
        PRIMARY\_DOMAIN=river.beaver.beta.web.tasking.rpp.cybercrucible.com  
        ALIAS\_DOMAIN=upstream.beta.web.tasking.rpp.cybercrucible.com  
        EXTRA\_DOMAINS=-d green.fox.beta.web.tasking.rpp.cybercrucible.com   
         -Note that if this docker dir has multiple upstreams then list each one here with '-d' before the domain like shown here, if only 1 upstream then this can be blank and just list the 1 under PRIMARY\_DOMAIN  
        ACME\_EMAIL=support@cybercrucible.com
6. Edit the aws.env file, also run sudo chmod 600 aws.env  
    AWS\_ACCESS\_KEY\_ID=AKIA...  
    AWS\_SECRET\_ACCESS\_KEY=...  
    AWS\_DEFAULT\_REGION=us-west-2  
    AWS\_USE\_DUALSTACK\_ENDPOINT=true #if the host machine only has IPV6 then leave this line here, if it has ipv4 you can uncomment, this is if the host only has ipv6 not the docker env that is important
7. Edit the certbot-deploy-hook.sh file, also run chmod +x on the file 
    1. \#!/bin/sh  
        set -eu  
        LIVE="/etc/letsencrypt/live/${CERT\_NAME}"  
        DEST=/etc/letsencrypt/deployed  
        mkdir -p "$DEST"  
        cp -L "$LIVE/privkey.pem" "$DEST/.privkey.tmp" &amp;&amp; chmod 600 "$DEST/.privkey.tmp"  
        cp -L "$LIVE/fullchain.pem" "$DEST/.fullchain.tmp" &amp;&amp; chmod 644 "$DEST/.fullchain.tmp"  
        mv -f "$DEST/.privkey.tmp" "$DEST/privkey.pem"  
        mv -f "$DEST/.fullchain.tmp" "$DEST/fullchain.pem"  
        echo "$(date -u +%FT%TZ) published ${CERT\_NAME} to $DEST" &gt;&gt; /etc/letsencrypt/deploy.log
8. Edit compose.yml file, see the attachment section 4.5 for what to put 
    1. You should only need to edit the <span style="color:rgb(184,10,24);">x-restServiceTemplate</span><span style="color:rgb(43,48,59);">:</span> <span style="color:rgb(43,48,59);">&amp;</span><span style="color:rgb(179,74,0);">restServiceTemplate</span> and services sections and copy paste the other sections in without change
9. You can test the certbot-init by adding --staging in the certbot-init command section in compose.yml and run it and make sure logs look good. Then remove the --staging and force production reissue with this command: 
    1. sudo docker compose run --rm certbot-init \\  
         "certbot certonly --dns-route53 --cert-name &lt;CERT\_NAME&gt; \\  
         -d &lt;PRIMARY\_DOMAIN&gt; -d &lt;ALIAS\_DOMAIN&gt; &lt;extra -d flags&gt; \\  
         --key-type ecdsa --non-interactive --agree-tos -m &lt;ACME\_EMAIL&gt; \\  
         --force-renewal &amp;&amp; CERT\_NAME=&lt;CERT\_NAME&gt; /etc/letsencrypt/renewal-hooks/deploy/publish.sh"
    2. Command I used in beta setup: 
        1. sudo docker compose run --rm certbot-init sh -c "certbot certonly --dns-route53 --cert-name rest-web-beta \\  
             -d river.beaver.beta.web.tasking.rpp.cybercrucible.com \\  
             -d upstream.beta.web.tasking.rpp.cybercrucible.com \\  
             -d green.fox.beta.web.tasking.rpp.cybercrucible.com \\  
             --key-type ecdsa --non-interactive --agree-tos -m support@cybercrucible.com \\  
             -force-renewal &amp;&amp; CERT\_NAME=rest-web-beta /etc/letsencrypt/renewal-hooks/deploy/publish.sh"
10. Make sure the dockers are running
11. Verify with these commands 
    1. \# Cert content, from inside the container:  
        sudo docker exec &lt;containerName&gt; sh -c \\  
         "openssl x509 -in /certs/deployed/fullchain.pem -noout -issuer -ext subjectAltName"  
        \# Want: issuer O=Let's Encrypt (no STAGING), all SANs listed.
        
        \# App started:  
        sudo docker logs &lt;containerName&gt; | grep -iE "started|error"
12. Updating nginx config. If the main nginx domain is ip split then update on both machines 
    1. Remove proxy\_ssl\_trusted\_certificate /etc/nginx/ssl/rest-web.crt; and proxy\_ssl\_name "Unknown"; settings
    2. Make sure to add these settings in: 
        1. proxy\_ssl\_verify on;  
            proxy\_ssl\_trusted\_certificate /etc/ssl/certs/ca-certificates.crt;  
            proxy\_ssl\_verify\_depth 3;  
            proxy\_ssl\_name &lt;ALIAS\_DOMAIN&gt;; #upstream.beta.web.tasking.rpp.cybercrucible.com; for example  
            proxy\_ssl\_server\_name on;
    3. Test and apply settings to nginx: 
        1. sudo docker exec &lt;nginx-container&gt; nginx -t &amp;&amp; sudo docker exec &lt;nginx-container&gt; nginx -s reload
13. Additional testing 
    1. echo | openssl s\_client -6 -connect \[&lt;backend IPv6&gt;\]:5050 \\  
         -servername &lt;ALIAS\_DOMAIN&gt; 2&gt;/dev/null \\  
         | openssl x509 -noout -issuer -dates -ext subjectAltName 
        1. \# Want: LE issuer, all SANs, ~90-day validity.
    2. Make sure public/health is working: curl https://&lt;public domain&gt;/public/health
    3. Test certbot renew: sudo docker compose exec certbot-renew certbot renew --dry-run