Lerma — Deployment Guide

On this page

Architecture
Deployment Flow
Prerequisites
Environment Provisioning
- 1. Set up API credentials
- 2. Provision infrastructure with OpenTofu
- 3. Collect outputs for GitHub secrets
- 4. Run Ecto migrations
Ansible (Server Configuration)
- Inventory
- Roles
- Running Ansible manually
Docker Compose Files
API documentation (/docs)
GitHub Actions
- Required GitHub Secrets
Local Development
Testing
DNS & Routing
- Maintainer access
Project Structure
Troubleshooting
- Database connection errors
- Authentication 401 errors
- Container not healthy
- Rebuilding the Docker image
- Adding a new environment

Architecture

Lerma is a standalone Phoenix application running on Hetzner VMs via Docker Compose, deployed with Ansible. Infrastructure is provisioned with OpenTofu.

Two environments:

Environment	Domain	Server	Image tag	Trigger
Production	`lerma.paso4.io`	Hetzner CX23	`:latest`	Release published
Staging	`lerma-dev.paso4.io`	Hetzner CX23	`:latest-pre`	Push to `develop`

Each environment has its own isolated Supabase PostgreSQL database (free tier), SSH key, and Hetzner firewall.

Browser
    │
    ▼
Cloudflare DNS (proxied)
    │
    ├── lerma.paso4.io     ──►  Production Hetzner VM
    │                                           └── Docker Compose ──► Phoenix app ──► Supabase (prod)
    │
    └── lerma-dev.paso4.io ──►  Staging Hetzner VM
                                               └── Docker Compose ──► Phoenix app ──► Supabase (staging)

Maintainer SSH (DNS-only, not proxied):
    lerma-host.paso4.io         ──►  Production Hetzner VM :22
    lerma-staging-host.paso4.io ──►  Staging Hetzner VM :22

Deployment Flow

Push to develop               Release published
      │                              │
      ▼                              ▼
 Build image                  Pull latest-pre
 Push as :latest-pre          Tag as :latest + release tag
      │                       Push to GHCR
      ▼                              │
 Deploy to staging             Deploy to production
 (Ansible against              (Ansible against
  staging VM)                   production VM)

On every push to develop, the CI builds a fresh Docker image tagged :latest-pre and deploys it to the staging server. On release publish, the CI promotes the :latest-pre image to :latest (no rebuild) and deploys it to production.

Manual deployments to production are available via workflow_dispatch.

Prerequisites

OpenTofu 1.6+ (for infrastructure provisioning)
Ansible (for server configuration and deployment)
Docker (for local development)
Elixir 1.20 with Erlang/OTP 29 (for local Phoenix development)
Access to the Paso4 Cloudflare account (35afea16440634aa2350331d2a736eec)
Hetzner Cloud API token
Supabase access token (generate here)

Environment Provisioning

1. Set up API credentials

export HCLOUD_TOKEN="your-hcloud-api-token"
export LERMA_API_TOKEN="your-lerma-api-token"

2. Provision infrastructure with OpenTofu

Each environment uses a separate OpenTofu workspace for state isolation and sources its own env file.

cd apps/lerma/iac/tf

# Create a terraform.tfvars (copy from .example) with your SSH keys
cp terraform.tfvars.example terraform.tfvars
# Edit: fill in ssh_public_key and ssh_public_key_staging

# Provision production (default workspace)
tofu init
(set +a && source ../../.env && set -a && tofu apply)

# Provision staging
tofu workspace new staging
(set +a && source ../../.env.staging && set -a && tofu apply)

3. Collect outputs for GitHub secrets

# Production
tofu workspace select default
tofu output -raw server_ipv4_address    # → SERVER_IP
tofu output -raw database_url           # → DATABASE_URL

# Staging
tofu workspace select staging
tofu output -raw server_ipv4_address    # → STAGING_SERVER_IP
tofu output -raw database_url           # → STAGING_DATABASE_URL

Add these to the GitHub repository secrets along with SECRET_KEY_BASE and STAGING_SECRET_KEY_BASE (generate with mix phx.gen.secret). Invitation emails are sent through Cloudflare Email Service using LERMA_API_TOKEN as the SMTP password, so the token must include the Email Sending: Edit permission and the sender domain must be onboarded for Email Sending.

4. Run Ecto migrations

Migrations are not run automatically. Run them manually against each database:

# Production
DATABASE_URL="postgres://..." mix ecto.migrate -r Lerma.Repo

# Staging
DATABASE_URL="postgres://..." mix ecto.migrate -r Lerma.Repo

Ansible (Server Configuration)

Ansible configures each Hetzner VM: installs Docker, sets up the firewall, writes configuration, and deploys the app container.

Inventory

File	Environment	IP source
`iac/ansible/inventory.yml`	Production	`SERVER_IP` env var
`iac/ansible/inventory.staging.yml`	Staging	`STAGING_SERVER_IP` env var

Roles

Role	Purpose
`common`	Update apt, install base packages (curl, gnupg, ufw)
`firewall`	Open ports 22, 80, 443; enable UFW with deny-incoming default
`docker`	Install Docker CE + Compose plugin, start service
`app-config`	Copy docker compose file, write `.env` with secrets
`app`	Login to GHCR, pull image, restart container, health check

Running Ansible manually

cd apps/lerma/iac/ansible

# Production
SERVER_IP=116.203.112.210 ansible-playbook \
  -i inventory.yml \
  --private-key ~/.ssh/lerma_hetzner \
  --extra-vars "ghcr_owner=0xcab0" \
  --extra-vars "ghcr_username=0xCAB0" \
  --extra-vars "ghcr_token=$(gh auth token)" \
  --extra-vars "database_url=$DATABASE_URL" \
  --extra-vars "secret_key_base=$SECRET_KEY_BASE" \
  --extra-vars "cloudflare_api_token=$LERMA_API_TOKEN" \
  --extra-vars "cf_access_team_domain=$CF_ACCESS_TEAM_DOMAIN" \
  --extra-vars "cf_access_aud=$CF_ACCESS_AUD" \
  playbooks/site.yml

# Staging
STAGING_SERVER_IP=<staging-ip> ansible-playbook \
  -i inventory.staging.yml \
  --private-key ~/.ssh/lerma_staging \
  --extra-vars "compose_source_file=docker-compose.staging.yml" \
  --extra-vars "ghcr_owner=0xcab0" \
  --extra-vars "ghcr_username=0xCAB0" \
  --extra-vars "ghcr_token=$(gh auth token)" \
  --extra-vars "database_url=$STAGING_DATABASE_URL" \
  --extra-vars "secret_key_base=$STAGING_SECRET_KEY_BASE" \
  --extra-vars "cloudflare_api_token=$LERMA_API_TOKEN" \
  --extra-vars "cf_access_team_domain=$CF_ACCESS_TEAM_DOMAIN" \
  --extra-vars "cf_access_aud=$CF_ACCESS_AUD" \
  playbooks/site.yml

Docker Compose Files

File	Environment	Image tag	Container name
`docker-compose.prod.yml`	Production	`:latest`	`lerma`
`docker-compose.staging.yml`	Staging	`:latest-pre`	`lerma-staging`

Both expose port 80 → 4000, set CF_WORKER_MODE=false (standalone VM mode), and read secrets from .env.

API documentation (`/docs`)

The Dockerfile runs mix docs during the image build; ExDoc writes the HexDocs-format HTML into priv/static/docs, which the release serves at /docs (see the docs/0 config in mix.exs and the dedicated Plug.Static plug in LermaWeb.Endpoint). /docs remains behind Cloudflare Access like the rest of the dashboard. README/DEPLOY/STYLE are included as ExDoc extras, so anything documented in the README — including the Infrastructure as Code section — is also reachable from the hosted docs.

GitHub Actions

Two workflows share the Lerma pipeline (one workflow per deployable asset):

CI — .github/workflows/lerma.yml runs the quality gates on PRs and pushes to develop (mix format --check-formatted, mix credo --strict, mix license_audit, mix test).
CD — .github/workflows/deploy-dashboard.yml builds and deploys. Staging runs only after Lerma CI completes successfully on develop (workflow_run gate), so failing quality gates block deployment.

Trigger	Jobs
Lerma CI success on `develop`	`build` → `deploy-staging`
Release published	`promote` → `deploy-production`
`workflow_dispatch`	`deploy-manual` (build + deploy to production)

Required GitHub Secrets

Secret	Environment	Purpose
`DATABASE_URL`	Production	Supabase PostgreSQL connection string
`STAGING_DATABASE_URL`	Staging	Staging Supabase PostgreSQL connection string
`SECRET_KEY_BASE`	Production	Phoenix secret key base
`STAGING_SECRET_KEY_BASE`	Staging	Phoenix secret key base
`SERVER_IP`	Production	Production Hetzner VM IP
`STAGING_SERVER_IP`	Staging	Staging Hetzner VM IP
`SSH_PRIVATE_KEY`	Production	SSH key for production VM
`STAGING_SSH_PRIVATE_KEY`	Staging	SSH key for staging VM
`LERMA_API_TOKEN`	Both	Lerma API token
`CF_ACCESS_TEAM_DOMAIN`	Both	Cloudflare Access team domain
`CF_ACCESS_AUD`	Both	Cloudflare Access audience tag (local env). CI maps the GitHub secrets `CF_ACCESS_LERMA_AUD` / `STAGING_CF_ACCESS_LERMA_AUD` into the `CF_ACCESS_LERMA_AUD` env var.
`E2E_CF_ACCESS_TEAM_DOMAIN`	Both	Aliased CF Access team domain (for ansible)

Local Development

# Start PostgreSQL
docker compose up -d db

# Setup and run
mix setup
mix phx.server

The Phoenix app runs on http://localhost:4000.

Testing

# Phoenix tests
mix test

# Generate the HexDocs-format API docs served at /docs
mix docs

# Full repository pre-commit check (run from the repository root)
cd ../..
mix precommit

DNS & Routing

Cloudflare DNS records are provisioned by OpenTofu:

Domain	Environment	Type	Proxied	Purpose
`lerma.paso4.io`	Production	A record	Yes	Dashboard (Cloudflare Access)
`lerma-dev.paso4.io`	Staging	A record	Yes	Dashboard (Cloudflare Access)
`lerma-host.paso4.io`	Production	A + AAAA records	No	Maintainer SSH/direct access
`lerma-staging-host.paso4.io`	Staging	A + AAAA records	No	Maintainer SSH/direct access

Maintainer access

The lerma-host / lerma-staging-host records are DNS-only (grey cloud) so they resolve for SSH. The dashboard records are proxied and cannot be used for SSH — Cloudflare’s proxy terminates HTTP(S) only.

# Production (SSH key: ~/.ssh/lerma_hetzner)
ssh root@lerma-host.paso4.io

# Staging (SSH key: ~/.ssh/lerma_staging)
ssh root@lerma-staging-host.paso4.io

OpenTofu exposes the hostname as host_fqdn:

tofu output -raw host_fqdn   # lerma-host.paso4.io (or lerma-staging-host.paso4.io)

The Phoenix endpoint’s check_origin is configured in config/runtime.exs:

check_origin: ["https://lerma.paso4.io", "https://lerma-dev.paso4.io"]

Project Structure

lerma/
├── lib/                         # Phoenix backend (Elixir)
│   ├── lerma/        # Business logic (Ecto/PostgreSQL)
│   └── lerma_web/    # Web layer (LiveView, controllers)
├── assets/                      # Phoenix frontend (JS/CSS)
├── config/                      # Phoenix config (runtime.exs, prod.exs)
├── priv/repo/migrations/        # Ecto migrations
├── test/                        # Phoenix tests
├── iac/                         # Infrastructure as Code
│   ├── tf/                      # OpenTofu (Hetzner, DNS, Supabase)
│   └── ansible/                 # Ansible (server config, deployment)
│       ├── inventory.yml        # Production inventory
│       ├── inventory.staging.yml # Staging inventory
│       ├── playbooks/site.yml   # Master playbook
│       ├── roles/               # common, firewall, docker, app-config, app
│       ├── templates/           # Jinja2 templates
│       └── group_vars/          # Shared variables
├── Dockerfile                   # Elixir release image
├── docker-compose.yml           # Local dev (PostgreSQL only)
├── docker-compose.prod.yml      # Production compose
├── docker-compose.staging.yml   # Staging compose
└── mix.exs                      # Elixir project definition

Troubleshooting

Database connection errors

Check the .env file at /opt/fouria/.env on the server:

ssh root@lerma-host.paso4.io cat /opt/fouria/.env          # production
ssh root@lerma-staging-host.paso4.io cat /opt/fouria/.env  # staging

Verify the Supabase project is active and the transaction pooler is enabled (port 6543).

Authentication 401 errors

Ensure CF_ACCESS_AUD matches the Cloudflare Access application audience tag. Set SKIP_AUTH=true in .env to bypass auth for debugging.

Container not healthy

Check container logs on the server:

ssh root@lerma-host.paso4.io docker compose -f /opt/fouria/docker-compose.yml logs app

Rebuilding the Docker image

The Docker image is rebuilt on every push to develop. To force a rebuild without code changes, update the cache bust comment in the Dockerfile:

# Build cache bust: 2026-09-22-v6-exdoc-docs

Adding a new environment

cd apps/lerma/iac/tf
tofu workspace new <env-name>
(tofu workspace select <env-name> && set +a && source ../../.env && set -a && tofu apply)