Renew certificates before they expire

Goal

Certificates renewed before anything notices, on a schedule, with the renewal proved by an actual TLS handshake rather than by the absence of an error.

When you finish expiry is a thing that gets handled at 2am by a scheduled run rather than discovered at 9am by a customer.

Why bother

Certificate expiry is the most predictable outage there is, which is what makes it embarrassing.

Without this	With this
A calendar reminder someone snoozes	A scheduled run
Renewal is remembered under time pressure	It happens with 30 days to spare
“Renewed” means the command exited zero	It means a handshake presented the new certificate
One host renewed, three forgotten	Every host enumerated and checked

Before you start

You need	Why
ACME reachable from the host, or a DNS credential	To answer the challenge
A reload that does not drop connections	The install is pointless if it costs an outage
The renewal window you want	30 days is the usual choice; below 14 leaves no room to retry

Step 1: find what is close to expiry

Renew on a window, not on a date. Anything inside the window gets renewed; everything else is left alone, which makes the run safe to execute daily.

Flow
Script

name: renew-certificates
list: true
vars:
  cert_path: /var/lib/letsencrypt
  window_days: 30
tasks:
  - name: inspect
    letsencrypt:
      op: info
      certpath: "{{cert_path}}"
      domain: "{{domain}}"
      setvar: cert

function main() {
  const info = letsencrypt.info({ certPath: cert_path, domain: domain });
  if (!info.success) throw new Error(info.error);

  const daysLeft = Math.floor(
    (new Date(info.notAfter).getTime() - Date.now()) / 86400000
  );
  log.info(`${domain}: ${daysLeft} days remaining`);

  if (daysLeft > Number(window_days)) {
    return { domain, renewed: false, daysLeft };
  }
  // …renew
}

  - name: renew
    letsencrypt:
      op: renew
      certpath: "{{cert_path}}"
      domain: "{{domain}}"
    retry:
      max_attempts: 3
      backoff_base: 30s
      backoff_max: 5m

The retry is not decoration. ACME providers rate-limit, and a renewal that fails on the first attempt at day 30 has 29 days of runway, but only if something tries again.

  const renewed = letsencrypt.renew({ certPath: cert_path, domain: domain });
  if (!renewed.success) throw new Error(`renew ${domain}: ${renewed.error}`);

Scripts have no retry policy of their own. Where renewal runs unattended, that is a reason to prefer the flow, see Which surface to use.

Step 3: install and reload

A renewed certificate on disk that no process has re-read is not a renewed certificate.

Flow
Script

  - name: reload
    ssh:
      host: "{{host}}:22"
      username: "{{user}}"
      privatekeypath: "{{keypath}}"
      commands: |
        set -e
        install -m 0644 {{cert_path}}/{{domain}}/fullchain.pem /etc/nginx/certs/{{domain}}.crt
        install -m 0600 {{cert_path}}/{{domain}}/privkey.pem   /etc/nginx/certs/{{domain}}.key
        nginx -t
        systemctl reload nginx

nginx -t before the reload is what stops a bad certificate taking the service down: the test fails, the reload never runs, and the old certificate keeps serving.

  const c = ssh.connect({ host: host, user: user, keyPath: keypath });
  try {
    const r = ssh.execute({
      clientId: c.clientId,
      commands: [
        'set -e',
        `install -m 0644 ${cert_path}/${domain}/fullchain.pem /etc/nginx/certs/${domain}.crt`,
        `install -m 0600 ${cert_path}/${domain}/privkey.pem   /etc/nginx/certs/${domain}.key`,
        'nginx -t',
        'systemctl reload nginx',
      ],
    });
    if (!r.success) throw new Error(`install ${domain}: ${r.error}`);
  } finally {
    ssh.close({ clientId: c.clientId });
  }

Note the key goes in at 0600 and the chain at 0644. They are not the same kind of file.

Step 4: prove it with a handshake

The only check that means anything:

echo | openssl s_client -connect "$DOMAIN:443" -servername "$DOMAIN" 2>/dev/null \
  | openssl x509 -noout -enddate -subject

notAfter=Nov  3 09:14:00 2026 GMT
subject=CN = www.example.com

If notAfter has not moved, the file changed and the process did not re-read it, go back to step 3. Checking the file on disk cannot tell you this, which is why the check is a connection.

Which surface to use

	Flow	Script
Retry on a rate-limited provider	`retry:` on the node	Write the loop yourself
Many domains	`map:` with bounded concurrency	Sequential
Which domains renewed	Recorded per node	Only what you logged
Runs unattended at 2am	Yes	Needs a wrapper

Renew with the flow. This is automation nobody watches, which puts a premium on the two things the flow has and the script does not: a retry policy, and a record of what happened while you were asleep.

Put it on a schedule

  - name: schedule
    cron:
      op: add
      name: renew-certificates
      schedule: "17 2 * * *"
      command: "kis flow -f /etc/kis/renew-certificates.yaml"

Daily, at an odd minute. Every day is right because the window makes it idempotent, nothing outside 30 days is touched, and an odd minute keeps you out of the crowd hitting the provider on the hour.

Verify

Check	Expect
`openssl s_client` after a run	`notAfter` roughly 90 days out
Run it again immediately	Nothing renewed: everything outside the window
Service log during reload	No dropped connections
The scheduled entry	`cron.list` shows it, `cron.logs` shows last night’s run

That second row is the one worth confirming, a renewal that renews every time it runs will hit the provider’s rate limit exactly when you need it most.

Adapt it

Change	How
Many domains	A CSV of domains and a `map:` node
DNS-01 challenge	Provide the DNS credential; the `dns:` operation can write the record
A different server	Replace `nginx -t` with that server’s config test. Keep the test
Certificates from an internal CA	Same shape, different issuing step

Rotate a secret across a fleet: the same overlap-then-retire shape
Restart a service safely
Secrets and security operations: letsencrypt, in both forms