# Lerma — Deployment Guide

## Architecture

Lerma is a standalone Phoenix application running on Hetzner VMs via Docker Compose, deployed with Ansible. Infrastructure is provisioned with OpenTofu.

Two environments:

| Environment    | Domain               | Server       | Image tag     | Trigger           |
| -------------- | -------------------- | ------------ | ------------- | ----------------- |
| **Production** | `lerma.paso4.io`     | Hetzner CX23 | `:latest`     | Release published |
| **Staging**    | `lerma-dev.paso4.io` | Hetzner CX23 | `:latest-pre` | Push to `develop` |

Each environment has its own isolated Supabase PostgreSQL database (free tier), SSH key, and Hetzner firewall.

```
Browser
    │
    ▼
Cloudflare DNS (proxied)
    │
    ├── lerma.paso4.io     ──►  Production Hetzner VM
    │                                           └── Docker Compose ──► Phoenix app ──► Supabase (prod)
    │
    └── lerma-dev.paso4.io ──►  Staging Hetzner VM
                                               └── Docker Compose ──► Phoenix app ──► Supabase (staging)

Maintainer SSH (DNS-only, not proxied):
    lerma-host.paso4.io         ──►  Production Hetzner VM :22
    lerma-staging-host.paso4.io ──►  Staging Hetzner VM :22
```

## Deployment Flow

```
Push to develop               Release published
      │                              │
      ▼                              ▼
 Build image                  Pull latest-pre
 Push as :latest-pre          Tag as :latest + release tag
      │                       Push to GHCR
      ▼                              │
 Deploy to staging             Deploy to production
 (Ansible against              (Ansible against
  staging VM)                   production VM)
```

On every push to `develop`, the CI builds a fresh Docker image tagged `:latest-pre` and deploys it to the staging server. On release publish, the CI promotes the `:latest-pre` image to `:latest` (no rebuild) and deploys it to production.

Manual deployments to production are available via `workflow_dispatch`.

## Prerequisites

- OpenTofu 1.6+ (for infrastructure provisioning)
- Ansible (for server configuration and deployment)
- Docker (for local development)
- Elixir 1.20 with Erlang/OTP 29 (for local Phoenix development)
- Access to the Paso4 Cloudflare account (`35afea16440634aa2350331d2a736eec`)
- Hetzner Cloud API token
- Supabase access token ([generate here](https://supabase.com/dashboard/account/tokens))

## Environment Provisioning

### 1. Set up API credentials

```bash
export HCLOUD_TOKEN="your-hcloud-api-token"
export LERMA_API_TOKEN="your-lerma-api-token"
```

### 2. Provision infrastructure with OpenTofu

Each environment uses a separate OpenTofu workspace for state isolation and sources its own env file.

```bash
cd apps/lerma/iac/tf

# Create a terraform.tfvars (copy from .example) with your SSH keys
cp terraform.tfvars.example terraform.tfvars
# Edit: fill in ssh_public_key and ssh_public_key_staging

# Provision production (default workspace)
tofu init
(set +a && source ../../.env && set -a && tofu apply)

# Provision staging
tofu workspace new staging
(set +a && source ../../.env.staging && set -a && tofu apply)
```

### 3. Collect outputs for GitHub secrets

```bash
# Production
tofu workspace select default
tofu output -raw server_ipv4_address    # → SERVER_IP
tofu output -raw database_url           # → DATABASE_URL

# Staging
tofu workspace select staging
tofu output -raw server_ipv4_address    # → STAGING_SERVER_IP
tofu output -raw database_url           # → STAGING_DATABASE_URL
```

Add these to the GitHub repository secrets along with `SECRET_KEY_BASE` and `STAGING_SECRET_KEY_BASE` (generate with `mix phx.gen.secret`). Invitation emails are sent through Cloudflare Email Service using `LERMA_API_TOKEN` as the SMTP password, so the token must include the `Email Sending: Edit` permission and the sender domain must be onboarded for Email Sending.

### 4. Run Ecto migrations

Migrations are not run automatically. Run them manually against each database:

```bash
# Production
DATABASE_URL="postgres://..." mix ecto.migrate -r Lerma.Repo

# Staging
DATABASE_URL="postgres://..." mix ecto.migrate -r Lerma.Repo
```

## Ansible (Server Configuration)

Ansible configures each Hetzner VM: installs Docker, sets up the firewall, writes configuration, and deploys the app container.

### Inventory

| File                                | Environment | IP source                   |
| ----------------------------------- | ----------- | --------------------------- |
| `iac/ansible/inventory.yml`         | Production  | `SERVER_IP` env var         |
| `iac/ansible/inventory.staging.yml` | Staging     | `STAGING_SERVER_IP` env var |

### Roles

| Role         | Purpose                                                       |
| ------------ | ------------------------------------------------------------- |
| `common`     | Update apt, install base packages (curl, gnupg, ufw)          |
| `firewall`   | Open ports 22, 80, 443; enable UFW with deny-incoming default |
| `docker`     | Install Docker CE + Compose plugin, start service             |
| `app-config` | Copy docker compose file, write `.env` with secrets           |
| `app`        | Login to GHCR, pull image, restart container, health check    |

### Running Ansible manually

```bash
cd apps/lerma/iac/ansible

# Production
SERVER_IP=116.203.112.210 ansible-playbook \
  -i inventory.yml \
  --private-key ~/.ssh/lerma_hetzner \
  --extra-vars "ghcr_owner=0xcab0" \
  --extra-vars "ghcr_username=0xCAB0" \
  --extra-vars "ghcr_token=$(gh auth token)" \
  --extra-vars "database_url=$DATABASE_URL" \
  --extra-vars "secret_key_base=$SECRET_KEY_BASE" \
  --extra-vars "cloudflare_api_token=$LERMA_API_TOKEN" \
  --extra-vars "cf_access_team_domain=$CF_ACCESS_TEAM_DOMAIN" \
  --extra-vars "cf_access_aud=$CF_ACCESS_AUD" \
  playbooks/site.yml

# Staging
STAGING_SERVER_IP=<staging-ip> ansible-playbook \
  -i inventory.staging.yml \
  --private-key ~/.ssh/lerma_staging \
  --extra-vars "compose_source_file=docker-compose.staging.yml" \
  --extra-vars "ghcr_owner=0xcab0" \
  --extra-vars "ghcr_username=0xCAB0" \
  --extra-vars "ghcr_token=$(gh auth token)" \
  --extra-vars "database_url=$STAGING_DATABASE_URL" \
  --extra-vars "secret_key_base=$STAGING_SECRET_KEY_BASE" \
  --extra-vars "cloudflare_api_token=$LERMA_API_TOKEN" \
  --extra-vars "cf_access_team_domain=$CF_ACCESS_TEAM_DOMAIN" \
  --extra-vars "cf_access_aud=$CF_ACCESS_AUD" \
  playbooks/site.yml
```

## Docker Compose Files

| File                         | Environment | Image tag     | Container name  |
| ---------------------------- | ----------- | ------------- | --------------- |
| `docker-compose.prod.yml`    | Production  | `:latest`     | `lerma`         |
| `docker-compose.staging.yml` | Staging     | `:latest-pre` | `lerma-staging` |

Both expose port 80 → 4000, set `CF_WORKER_MODE=false` (standalone VM mode), and read secrets from `.env`.

## API documentation (`/docs`)

The Dockerfile runs `mix docs` during the image build; ExDoc writes the
HexDocs-format HTML into `priv/static/docs`, which the release serves at
`/docs` (see the `docs/0` config in [mix.exs](https://github.com/0xCAB0/fouria/blob/develop/apps/lerma/mix.exs) and the dedicated
`Plug.Static` plug in `LermaWeb.Endpoint`). `/docs` remains behind Cloudflare
Access like the rest of the dashboard. README/DEPLOY/STYLE are included as
ExDoc `extras`, so anything documented in the README — including the
[Infrastructure as Code section](/llms/platform/apps/lerma/index.md#infrastructure-as-code) — is also
reachable from the hosted docs.

## GitHub Actions

Two workflows share the Lerma pipeline (one workflow per deployable asset):

- **CI** — `.github/workflows/lerma.yml` runs the quality gates on PRs and
  pushes to `develop` (`mix format --check-formatted`, `mix credo --strict`,
  `mix license_audit`, `mix test`).
- **CD** — `.github/workflows/deploy-dashboard.yml` builds and deploys.
  Staging runs only after Lerma CI completes successfully on `develop`
  (`workflow_run` gate), so failing quality gates block deployment.

| Trigger                          | Jobs                                        |
| -------------------------------- | ------------------------------------------- |
| Lerma CI success on `develop`    | `build` → `deploy-staging`                  |
| Release published                | `promote` → `deploy-production`             |
| `workflow_dispatch`              | `deploy-manual` (build + deploy to production) |

### Required GitHub Secrets

| Secret                      | Environment | Purpose                                       |
| --------------------------- | ----------- | --------------------------------------------- |
| `DATABASE_URL`              | Production  | Supabase PostgreSQL connection string         |
| `STAGING_DATABASE_URL`      | Staging     | Staging Supabase PostgreSQL connection string |
| `SECRET_KEY_BASE`           | Production  | Phoenix secret key base                       |
| `STAGING_SECRET_KEY_BASE`   | Staging     | Phoenix secret key base                       |
| `SERVER_IP`                 | Production  | Production Hetzner VM IP                      |
| `STAGING_SERVER_IP`         | Staging     | Staging Hetzner VM IP                         |
| `SSH_PRIVATE_KEY`           | Production  | SSH key for production VM                     |
| `STAGING_SSH_PRIVATE_KEY`   | Staging     | SSH key for staging VM                        |
| `LERMA_API_TOKEN`           | Both        | Lerma API token                               |
| `CF_ACCESS_TEAM_DOMAIN`     | Both        | Cloudflare Access team domain                 |
| `CF_ACCESS_AUD`             | Both        | Cloudflare Access audience tag (local env). CI maps the GitHub secrets `CF_ACCESS_LERMA_AUD` / `STAGING_CF_ACCESS_LERMA_AUD` into the `CF_ACCESS_LERMA_AUD` env var. |
| `E2E_CF_ACCESS_TEAM_DOMAIN` | Both        | Aliased CF Access team domain (for ansible)   |

## Local Development

```bash
# Start PostgreSQL
docker compose up -d db

# Setup and run
mix setup
mix phx.server
```

The Phoenix app runs on `http://localhost:4000`.

## Testing

```bash
# Phoenix tests
mix test

# Generate the HexDocs-format API docs served at /docs
mix docs

# Full repository pre-commit check (run from the repository root)
cd ../..
mix precommit
```

## DNS & Routing

Cloudflare DNS records are provisioned by OpenTofu:

| Domain                        | Environment | Type                        | Proxied | Purpose                        |
| ----------------------------- | ----------- | --------------------------- | ------- | ------------------------------ |
| `lerma.paso4.io`              | Production  | A record                    | Yes     | Dashboard (Cloudflare Access)  |
| `lerma-dev.paso4.io`          | Staging     | A record                    | Yes     | Dashboard (Cloudflare Access)  |
| `lerma-host.paso4.io`         | Production  | A + AAAA records            | No      | Maintainer SSH/direct access   |
| `lerma-staging-host.paso4.io` | Staging     | A + AAAA records            | No      | Maintainer SSH/direct access   |

### Maintainer access

The `lerma-host` / `lerma-staging-host` records are **DNS-only** (grey cloud) so
they resolve for SSH. The dashboard records are proxied and cannot be used for
SSH — Cloudflare's proxy terminates HTTP(S) only.

```bash
# Production (SSH key: ~/.ssh/lerma_hetzner)
ssh root@lerma-host.paso4.io

# Staging (SSH key: ~/.ssh/lerma_staging)
ssh root@lerma-staging-host.paso4.io
```

OpenTofu exposes the hostname as `host_fqdn`:

```bash
tofu output -raw host_fqdn   # lerma-host.paso4.io (or lerma-staging-host.paso4.io)
```

The Phoenix endpoint's `check_origin` is configured in `config/runtime.exs`:

```elixir
check_origin: ["https://lerma.paso4.io", "https://lerma-dev.paso4.io"]
```

## Project Structure

```
lerma/
├── lib/                         # Phoenix backend (Elixir)
│   ├── lerma/        # Business logic (Ecto/PostgreSQL)
│   └── lerma_web/    # Web layer (LiveView, controllers)
├── assets/                      # Phoenix frontend (JS/CSS)
├── config/                      # Phoenix config (runtime.exs, prod.exs)
├── priv/repo/migrations/        # Ecto migrations
├── test/                        # Phoenix tests
├── iac/                         # Infrastructure as Code
│   ├── tf/                      # OpenTofu (Hetzner, DNS, Supabase)
│   └── ansible/                 # Ansible (server config, deployment)
│       ├── inventory.yml        # Production inventory
│       ├── inventory.staging.yml # Staging inventory
│       ├── playbooks/site.yml   # Master playbook
│       ├── roles/               # common, firewall, docker, app-config, app
│       ├── templates/           # Jinja2 templates
│       └── group_vars/          # Shared variables
├── Dockerfile                   # Elixir release image
├── docker-compose.yml           # Local dev (PostgreSQL only)
├── docker-compose.prod.yml      # Production compose
├── docker-compose.staging.yml   # Staging compose
└── mix.exs                      # Elixir project definition
```

## Troubleshooting

### Database connection errors

Check the `.env` file at `/opt/fouria/.env` on the server:
```bash
ssh root@lerma-host.paso4.io cat /opt/fouria/.env          # production
ssh root@lerma-staging-host.paso4.io cat /opt/fouria/.env  # staging
```

Verify the Supabase project is active and the transaction pooler is enabled (port 6543).

### Authentication 401 errors

Ensure `CF_ACCESS_AUD` matches the Cloudflare Access application audience tag. Set `SKIP_AUTH=true` in `.env` to bypass auth for debugging.

### Container not healthy

Check container logs on the server:
```bash
ssh root@lerma-host.paso4.io docker compose -f /opt/fouria/docker-compose.yml logs app
```

### Rebuilding the Docker image

The Docker image is rebuilt on every push to `develop`. To force a rebuild without code changes, update the cache bust comment in the Dockerfile:
```dockerfile
# Build cache bust: 2026-09-22-v6-exdoc-docs
```

### Adding a new environment

```bash
cd apps/lerma/iac/tf
tofu workspace new <env-name>
(tofu workspace select <env-name> && set +a && source ../../.env && set -a && tofu apply)
```