FourIA — Environment Variables Reference
On this page
- Variable Classification
- Dynamic Secrets (F4E1-Encrypted R2 Storage)
- AI Provider Configuration
- Authentication & Access Control
- Persistence & Storage
- Cloudflare Tunnel
- Browser Automation (CDP)
- Container Rescue (/debug/rescue-container)
- Billing API (Lerma, server-side only)
- Lerma Provisioning API Keys
- Feature Flags
- Stripe Billing Integration
- Communication Schema (Dispatch Worker → User Worker)
- Dashboard Tenant Provisioning (apps/lerma)
- Legacy Channel Secrets (E2E Testing Only)
- Internal Variables (Auto-Set)
- Setting Variables
Complete reference of all environment variables used across FourIA Workers. Derived from apps/fouria/src/gateway/env.ts and apps/fouria/.dev.vars.example.
Variable Classification
FourIA uses three classifications for configuration values:
| Classification | Storage | Binding Type | Managed By | Examples |
|---|---|---|---|---|
| Instance Secret | CF encrypted secret_text | secret_text | lerma provisioner | Gateway token, CDP secret, AI keys, Stripe |
| Variable | wrangler.jsonc / WFP plain_text | plain_text | Provisioner / IaC | DEV_MODE, URLs, account IDs, sandbox config |
| Dynamic Secret | F4E1-encrypted R2 object in BACKUP_BUCKET | — | End-user via integrations UI | Channel tokens, OAuth, custom integrations |
Instance Secrets
These are automatically generated and deployed by the lerma provisioner as secret_text bindings on each tenant User Worker. For the full inventory of required secrets, see the secrets manifest.
| Secret | Description |
|---|---|
MOLTBOT_GATEWAY_TOKEN | Auto-generated gateway token for CLI auth. Mapped to OPENCLAW_GATEWAY_TOKEN in container. |
CF_ACCESS_PLATFORM_AUD | Platform-level Access AUD (always present, from dispatch worker). |
CF_ACCESS_CLIENT_AUD | Per-tenant Access AUD (optional, provisioned by lerma). |
Secrets reference: See docs/schemas/secrets-manifest.schema.json for the canonical credential inventory.
Variables (Non-Secret Config)
Operational configuration, safe to store in wrangler.jsonc or .dev.vars. Deployed as plain_text bindings by the provisioner.
| Variable | Container | Required | Description |
|---|---|---|---|
DEV_MODE | Yes† | No | Skips CF Access auth + bypasses device pairing. Local dev only. |
OPENCLAW_DEV_MODE | Yes | No | Explicit override for the dev-agents flag. Normally derived from the instance’s own -dev hostname suffix (WORKER_URL), which enables the dev-only Fouria Builder/Q&A agents, client-doc materialization, the dev-only ROOT “view as” role preview, and the opencode-credits-check automation. Does not bypass Cloudflare Access auth. Falls back to DEV_MODE for local development. |
E2E_TEST_MODE | Yes‡ | No | Skips CF Access auth but keeps device pairing. For automated E2E. |
BILLABLE_EVENTS | No | No | Opt-in for billable-action emission on a dev instance (the billable-action Analytics Engine rows + BILLING_QUEUE enqueue lerma reconciles into credits). Billable production and the staging platform always emit and cannot be opted out (staging mirrors production so the billing pipeline is exercised before release; its rows land in the staging Analytics Engine dataset). A dev environment is silent by default and emits only with true — useful for exercising billing end-to-end from a dev instance. Emission only: the platform/environment cost-row dimensions and the container sleep policy are unchanged, so an opted-in dev instance still auto-sleeps (and staging keeps its 15m idle timeout). |
DEBUG_ROUTES | No | No | Enables /debug/* endpoints (ROOT role only). Off in production. lerma provisions dev instances with DEBUG_ROUTES=true by default; toggle per instance from the dashboard. |
DEMO_MODE | No | No | Enables demo-specific UI behaviors (read-only, sample data). |
SANDBOX_RUNTIME | No | No | Intended container runtime: legacy (default) or v1. v1 selects the side-by-side SandboxV1 (ctx.container) runtime; until it implements the full Sandbox surface the legacy class keeps serving and the flag is reported as pending (/debug/config → sandbox_runtime). The container idle timeout is derived, not configured: staging 15m / dev 2h / production 6h (the Sandbox SDK 1.0 platform cap), from the instance hostname — there is no SANDBOX_SLEEP_AFTER binding. Only non-billable instances auto-sleep: the billable production environment never idle-sleeps; it is stopped only by an explicit lerma freeze (POST /control → sandbox.destroy()). Waking a sleeping container is the ActivityScheduler DO alarm’s job. |
SANDBOX_INSTANCE_TIMEOUT_MS | No | No | Bound for container VM provisioning (default: 20000). Fail-fast detection of a degraded/off container. |
SANDBOX_PORT_TIMEOUT_MS | No | No | Bound for container port readiness (default: 90000). Keep generous — gateway cold start is slow. |
SANDBOX_POLL_INTERVAL_MS | No | No | Poll interval for container readiness (default: 300). |
SANDBOX_LOG_LEVEL | No | No | Container log level: debug, info, warn, error |
SANDBOX_LOG_FORMAT | No | No | Container log format: pretty or json |
WORKER_URL | Yes | No | Public Worker URL (used for CDP and OTel endpoint) |
WORKER_NAME | Yes | No | Worker name for config bucket key resolution |
CF_AI_GATEWAY_ACCOUNT_ID | Yes | Yes* | Cloudflare account ID for AI Gateway URL construction |
CF_AI_GATEWAY_GATEWAY_ID | Yes | Yes* | AI Gateway ID for URL construction |
CF_AI_GATEWAY_CUSTOM_DOMAIN | No | No | AI Gateway custom domain (e.g. ai.example.com). Routes unified billing through https://<domain>/compat instead of api.cloudflare.com; required for Cloudflare Access identity-aware controls and User Insights attribution. |
CF_ACCESS_TOKEN | No | Yes* | Cloudflare Access user JWT forwarded as cf-access-token on an Access-protected custom domain. AI Gateway records cf.user_id, which User Insights uses to attribute spend per identity. |
CF_ACCESS_CLIENT_ID | No | Yes* | Cloudflare Access service-token client ID (headless alternative; service tokens do not produce cf.user_id). |
CF_ACCESS_CLIENT_SECRET | No | Yes* | Cloudflare Access service-token client secret. |
CF_ACCESS_TEAM_DOMAIN | No | Yes* | Cloudflare Access team domain (e.g., myteam.cloudflareaccess.com) |
CLOUDFLARE_ACCOUNT_ID | Yes | No | Cloudflare account ID. Read by the Sandbox SDK for R2 presigned URL signing (backup/restore). Legacy CF_ACCOUNT_ID is deprecated (still mapped into the container by buildEnvVars). |
BACKUP_BUCKET_NAME | Yes§ | No | R2 backup bucket name (default: fouria-data). Required by the Sandbox SDK presigned-URL backup path; mapped to R2_BUCKET_NAME in container. |
BACKUP_MAX_VERSIONS | No | No | Max squashfs snapshot versions to retain (default: 3) |
BACKUP_INTERVAL_MINUTES | No | No | Minimum scheduled-snapshot cadence (default: 5). The effective interval is max(this, 2 × the last snapshot's end-to-end duration), so a tenant whose snapshot outlasts the cadence is not snapshotted back-to-back (which starved the automation wake and held the R2 lease continuously). See resolveEffectiveIntervalMs in src/persistence.ts. |
BACKUP_CHANGE_DETECTION | No | No | Feature flag (default: off). 'true' skips scheduled snapshots of unchanged tenant state. Set on lerma; deployed to tenant Workers. |
BACKUP_CHANGE_WATCH | No | No | Feature flag (default: off). 'true' runs the per-tenant BackupWatch loop. Set on lerma, which also binds BACKUP_WATCH. |
BACKUP_CHANGE_CHECK_MINUTES | No | No | BackupWatch check cadence while armed (default: 2). Worker-only. |
BACKUP_CHANGE_DEBOUNCE_MS | No | No | Quiet window before a still-changing tenant is snapshotted, in ms (default: 60000). Worker-only. |
BACKUP_LOCAL_BUCKET | No | No | Set to true to store R2 backups in the Sandbox local binding bucket instead of remote R2 (needed for local dev without presigned-URL support). Explicit; falls back to DEV_MODE only when unset, so a real deployment — including the cloud E2E, which sets DEV_MODE=true merely to bypass Access — must set it to false or backups land somewhere the Worker cannot restore from. |
LEGACY_STAGE_TIMEOUT_MS | No | No | Bound (ms) for the legacy-state staging shell that runs before sandbox.createBackup; the stage is retried once on a transient timeout. Default: 600000 (10 min) |
ANALYTICS_DATASET | No | No | Analytics Engine dataset holding emitted billable-action rows, read back by GET /api/admin/costs/v2/summary for the dashboard “Usage & Billing” widget. Default: fourai_cost_metrics; the staging provisioner sets fourai_cost_metrics_staging. |
ANALYTICS_ENGINE_BASE_URL | No | No | Overrides the Analytics Engine SQL API base URL for cost-summary queries (default https://api.cloudflare.com/client/v4/accounts). Local billing-workflow E2E only — points the query at a fake AE. |
ANALYTICS_ENGINE_API_TOKEN | No | No | Dedicated Analytics Engine SQL token (Account Analytics:Read) for cost-summary queries. Provisioned by Lerma as a per-tenant secret_text binding from its own environment (listed in tenant_provisioner.ex @platform_secrets). Falls back to CLOUDFLARE_API_TOKEN (the AI Gateway REST alias, which lacks AE read — the query then 403s and the widget shows zeros). |
† DEV_MODE is a local-development switch (auth bypass). The dev-only agents surface is derived from a dev instance’s -dev hostname (or an explicit OPENCLAW_DEV_MODE override) without bypassing auth.
‡ Mapped to OPENCLAW_E2E_TEST_MODE in container.
§ Mapped to R2_BUCKET_NAME in container.
Mode behavior summary:
| Mode | CF Access Auth | Device Pairing | Debug Routes | Write API |
|---|---|---|---|---|
| Production | Required | Required | Disabled | RBAC |
DEV_MODE | Bypassed | Bypassed | Configurable | ROOT |
E2E_TEST_MODE | Bypassed | Required | Configurable | ROOT |
DEMO_MODE | Required | Required | Disabled | Read-only |
Dynamic Secrets (F4E1-Encrypted R2 Storage)
Channel tokens, tool OAuth credentials, and custom integration secrets are managed by end-users through the integrations dashboard. They are stored as F4E1-encrypted JSON objects in the BACKUP_BUCKET R2 bucket under the secrets/<customerSlug>/ prefix.
Secret categories: ai_provider, channel, integration
The encryption uses the shared BACKUP_ENCRYPTION_KEY (same master key used for backup snapshots) with per-object HKDF-derived AES-256-GCM keys (domain-separated via fouria-r2-secrets/v1). The master key never leaves the Worker and is never passed to the container.
Unlike the old vault, there is no unlock/lock mechanism: secrets are always accessible when the key is configured. The /lock endpoint is a no-op kept for client compatibility. The WebSocket API (/ws) has been removed (returns 410 Gone).
See
apps/fouria/src/services/secrets/R2SecretsStore.tsfor the full implementation.
AI Provider Configuration
At least one provider must be configured. Priority: CF AI Gateway > Anthropic > OpenAI.
CI-Only Secret Mapping
The CI workflow maps GitHub secrets (prefixed with E2E_) to their corresponding env variables. See .github/workflows/test.yml for the authoritative mapping.
| GitHub Secret | Maps To |
|---|---|
E2E_CLOUDFLARE_AI_GATEWAY_API_KEY | CLOUDFLARE_AI_GATEWAY_API_KEY |
E2E_CF_AI_GATEWAY_ACCOUNT_ID | CF_AI_GATEWAY_ACCOUNT_ID |
E2E_CF_AI_GATEWAY_GATEWAY_ID | CF_AI_GATEWAY_GATEWAY_ID |
E2E_TEARDOWN_API_TOKEN | CLOUDFLARE_TEARDOWN_API_TOKEN |
E2E_TELEGRAM_BOT_TOKEN | E2E_TELEGRAM_BOT_TOKEN |
There is no GitHub secret named
CLOUDFLARE_AI_GATEWAY_API_KEY. Staging and production Lerma deploys (.github/workflows/deploy-dashboard.yml) both readE2E_CLOUDFLARE_AI_GATEWAY_API_KEYand deploy it as theCLOUDFLARE_AI_GATEWAY_API_KEYruntime env. Referencing a non-existentsecrets.CLOUDFLARE_AI_GATEWAY_API_KEYexpands to empty and ships a dashboard whose tenant provisioning fails with:missing_ai_gateway_credential— the deploy workflow fails fast when the secret is empty.
E2E_TEARDOWN_API_TOKENis used only by the E2E teardown for destructive cleanup (wrangler delete+ R2 bucket empty/delete + container/Access/service-token deletes). It must carry Workers Scripts, Workers KV Storage, Workers R2 Storage, Cloudflare Containers, and Access (Apps + Service Tokens) Edit permissions. When unset, teardown falls back toCLOUDFLARE_API_TOKEN(DEPLOY_WORKER_API_TOKENin CI).
E2E_TELEGRAM_BOT_TOKENis an optional real Telegram bot token (from@BotFather) for the Telegram webhook-ingress E2E (q_telegram_webhook.txt). The suite self-skips when it is unset. It is never passed to the container or used in production.Local cloud E2E file.
bun run secrets:generateproduces the gitignoredapps/fouria/.dev.vars.e2ewith the mandatory cloud E2E vars —CLOUDFLARE_API_TOKEN(fromLERMA_API_TOKEN, since E2E provisions real Workers/Containers/R2/Access/KV resources),CLOUDFLARE_ACCOUNT_ID,WORKERS_SUBDOMAIN,CF_ACCESS_TEAM_DOMAIN,R2_ACCESS_KEY_ID,R2_SECRET_ACCESS_KEY— plus the AI Gateway trio and Tunnel token when set. Vars the registry does not track (CLOUDFLARE_TEARDOWN_API_TOKEN,SUPABASE_*, channel tokens, …) live in the file’s preserved manual section.Secrets reference: See docs/schemas/secrets-manifest.schema.json for the canonical credential inventory.
Runtime AI Credentials
| Variable | Class | Container | Description |
|---|---|---|---|
CLOUDFLARE_AI_GATEWAY_API_KEY | Platform Secret | Yes | Real AI-scoped Cloudflare credential supplied by Lerma. It is never generated randomly per tenant. |
CLOUDFLARE_API_TOKEN | Tenant Runtime Alias | Yes | Bearer credential for the AI Gateway REST API. Lerma derives this tenant binding from CLOUDFLARE_AI_GATEWAY_API_KEY; it is not Lerma’s administrative provisioning token. |
The native cloudflare-ai-gateway provider uses CLOUDFLARE_AI_GATEWAY_API_KEY. The unified-billing plugin uses CLOUDFLARE_API_TOKEN for api.cloudflare.com/client/v4/accounts/{account}/ai/v1 requests. When CF_AI_GATEWAY_CUSTOM_DOMAIN is set, the plugin switches to https://{domain}/compat and authenticates with CF_ACCESS_TOKEN (or a service token) when the domain is behind Cloudflare Access — this is what enables User Insights identity attribution via cf.user_id.
* At least one provider must be configured.
Authentication & Access Control
| Variable | Class | Container | Required | Description |
|---|---|---|---|---|
MOLTBOT_GATEWAY_TOKEN | Instance Secret | Yes† | Yes | Gateway token for CLI auth. Auto-generated per tenant by lerma provisioner. |
CF_ACCESS_PLATFORM_AUD | Instance Secret | No | Yes* | Platform-level Access AUD (always present, from dispatch worker). |
CF_ACCESS_CLIENT_AUD | Instance Secret | No | No | Per-tenant Access AUD (optional, provisioned by lerma). |
CF_ACCESS_TEAM_DOMAIN | Variable | No | Yes* | Cloudflare Access team domain (e.g., myteam.cloudflareaccess.com). |
Secrets reference: See docs/schemas/secrets-manifest.schema.json for the canonical credential inventory.
* Required for production. Not required when DEV_MODE=true or E2E_TEST_MODE=true.
† Mapped to OPENCLAW_GATEWAY_TOKEN by buildEnvVars().
Persistence & Storage
| Variable | Class | Container | Required | Description |
|---|---|---|---|---|
CLOUDFLARE_ACCOUNT_ID | Variable | Yes | No | Cloudflare account ID. Read by the Sandbox SDK for R2 presigned URL signing (backup/restore). Legacy CF_ACCOUNT_ID is deprecated (still mapped into the container by buildEnvVars). |
BACKUP_BUCKET_NAME | Variable | Yes† | No | R2 backup bucket name (default: fouria-data). Required by the Sandbox SDK presigned-URL backup path alongside the BACKUP_BUCKET binding; mapped to R2_BUCKET_NAME in container. |
R2_ACCESS_KEY_ID | Secret | No‡ | Yes | Deprecated for the container. Worker binding only: required by the Sandbox SDK presigned-URL backup path. Resolved per-instance by the lerma provisioner (shared lerma credential or a generated per-instance token). No longer forwarded into the container — sandbox R2 access uses the outboundByHost bridges (releases.r2.internal / docs.r2.internal / leads.r2.internal). |
R2_SECRET_ACCESS_KEY | Secret | No‡ | Yes | R2 secret key, paired with R2_ACCESS_KEY_ID. Worker binding only (see above); not forwarded into the container. |
BACKUP_MAX_VERSIONS | Variable | No | No | Max squashfs snapshot versions to retain (default: 3) |
BACKUP_INTERVAL_MINUTES | Variable | No | No | Minimum scheduled-snapshot cadence (default: 5). The effective interval stretches to at least twice the last observed snapshot duration so a slow tenant cannot snapshot back-to-back. See resolveEffectiveIntervalMs in src/persistence.ts. |
BACKUP_LOCAL_BUCKET | Variable | No | No | Set to true to store backups in the Sandbox local binding bucket (local dev only). Explicit; falls back to DEV_MODE when unset. Real deployments (including the cloud E2E) set it to false so backups land in the remote BACKUP_BUCKET. |
LEGACY_STAGE_TIMEOUT_MS | Variable | No | No | Bound (ms) for the legacy-state staging shell in createSnapshot; retried once on a transient timeout. Default: 600000 (10 min) |
BACKUP_ENCRYPTION_KEY | Secret | No | Recommended | Base64 32-byte AES-256 master key for client-side R2 backup encryption (F4E1). Worker-side only, never passed to the container. Generate: openssl rand -base64 32 |
BACKUP_ENCRYPTION_REQUIRED | Variable | No | No | Set to true to fail closed (skip backup + error) when BACKUP_ENCRYPTION_KEY is missing (default: false, legacy plaintext + warning) |
† Mapped to R2_BUCKET_NAME in container.
‡ Deprecated for the container: kept as a Worker binding for the Sandbox SDK presigned-URL backup path, never exposed inside the container. Sandbox R2 access goes through the outboundByHost bridges (releases.r2.internal / docs.r2.internal / leads.r2.internal).
Secrets reference: See docs/schemas/secrets-manifest.schema.json for the canonical credential inventory.
Cloudflare Tunnel
Secrets reference: See docs/schemas/secrets-manifest.schema.json for the canonical credential inventory.
Browser Automation (CDP)
| Variable | Class | Container | Required | Description |
|---|---|---|---|---|
WORKER_URL | Variable | Yes | No | Public Worker URL (used for CDP and OTel endpoint) |
Secrets reference: See docs/schemas/secrets-manifest.schema.json for the canonical credential inventory.
Container Rescue (/debug/rescue-container)
Recovers a container stuck in Error mode (e.g. the fouria:unstable-<sha> image it references was deleted by the stale-image cleanup cron). The tool probes the container via the Sandbox SDK, re-points the application image to the expected tag through the Cloudflare Containers API (rollout), and destroys the stuck container so the platform provisions a fresh one. See apps/fouria/src/diagnostics.ts.
| Variable | Class | Container | Required | Description |
|---|---|---|---|---|
CLOUDFLARE_API_TOKEN | Runtime Secret | Yes | No | AI Gateway REST Bearer token on tenant workers. On ROOT/ops deployments the same binding may also be used by /debug/rescue-container when it has Containers:Edit. |
CONTAINER_APPLICATION_ID | Variable | No | No | Cloudflare Containers application id for this worker’s Sandbox container (set by the provisioner). |
CONTAINER_IMAGE_TAG | Variable | No | No | Expected container image tag (e.g. unstable-<sha>); used to derive the repair target registry.cloudflare.com/<account>/fouria:<tag>. |
CONTAINER_IMAGE_NAME | Variable | No | No | Container image repo name for the tenant sandbox image (default: fouria). |
Secrets reference: See docs/schemas/secrets-manifest.schema.json for the canonical credential inventory.
| Variable | Class | Container | Required | Description |
|---|---|---|---|---|
OTEL_SHARED_SECRET | Instance Secret | Yes | No | Shared secret for OTel metric ingest authentication |
OTEL_EXPORTER_OTLP_METRICS_ENDPOINT | Auto | Yes | Auto | Auto-derived from WORKER_URL + /api/otel/v1/metrics |
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT | Auto | Yes | Auto | Auto-derived from WORKER_URL + /api/otel/v1/traces (same E2E gating) |
OTEL_EXPORTER_OTLP_PROTOCOL | Auto | Yes | Auto | Auto-set to http/json (never grpc/protobuf) |
OTEL_TRACES_ENABLED | Variable | Yes | No | 'true' enables diagnostics.otel.traces (default: disabled) |
OTEL_SERVICE_NAME | Auto | Yes | Auto | Auto-set to customer slug or worker name |
ANALYTICS_ENGINE_API_TOKEN | Instance Secret | No | No | Account Analytics:Read token for the cost-summary Analytics Engine SQL query (GET /api/admin/costs/v2/summary, dashboard “Usage & Billing”). Read from Lerma’s env and bound per tenant. |
PROMOTION_PUSH_TOKEN | Instance Secret | No | No | Shared Worker→Worker secret for the dev→production agent push (POST /api/admin/releases/import). When set, the import endpoint requires it as a bearer token. Optional. |
The OTEL_SHARED_SECRET above is the shared secret that authenticates the
tenant worker’s OTLP receiver (POST /api/otel/v1/traces, /v1/metrics) and is
forwarded to the container as OTEL_EXPORTER_OTLP_HEADERS. When it is unset the
receiver rejects every export and no billable traces are produced. It is read
from Lerma’s environment and deployed as a per-tenant secret_text binding.
ANALYTICS_ENGINE_API_TOKEN is likewise read from Lerma’s environment and
deployed per tenant, but is Worker-only (never passed to the container). It
must be scoped to Account Analytics:Read. Without it
GET /api/admin/costs/v2/summary falls back to CLOUDFLARE_API_TOKEN — the AI
Gateway alias, which lacks Analytics read — so the Analytics Engine SQL query
returns 403 and the dashboard “Usage & Billing” widget shows zeros (the
request degrades to an empty summary and never fails).
PROMOTION_PUSH_TOKEN (optional) is the shared secret for the dev→production
agent push: the dev instance’s POST /api/admin/releases/promote exports the
promotable manifest and POSTs it to the production instance’s
POST /api/admin/releases/import, which applies it as a custom backup. When the
variable is set on the production instance the import requires it as a bearer
token; when unset the standard Access/RBAC identity is the only gate. See
apps/fouria/docs/WORKER-DEVELOPMENT.md.
Billing API (Lerma, server-side only)
These are Lerma-dashboard environment variables — never deployed to tenant workers.
| Variable | Class | Required | Description |
|---|---|---|---|
BILLING_INGEST_TOKEN | Runtime Secret | No | Bearer token guarding the internal billing API: L2 ingest (POST /api/credits/v1/ingest) and L5 reconcile/usage (POST /api/credits/v1/reconcile, GET /api/credits/v1/usage/:id). Unset → every request is rejected (401). |
ANALYTICS_DATASET | Variable | No | Analytics Engine dataset holding billable rows (fourai_cost_metrics default; fourai_cost_metrics_staging when CF_ENVIRONMENT=staging). |
ANALYTICS_ENGINE_BASE_URL | Variable | No | Overrides the Analytics Engine SQL API base URL for Lerma.BillingReconcile/Lerma.Analytics.CFAnalytics. Local billing-workflow E2E only — points them at the local fake AE (apps/lerma/test/e2e_billing_local/). Unset everywhere else. |
BILLING_INGEST_TOKEN permissions: none. It is not a Cloudflare API token and needs no Cloudflare scopes — it is an opaque shared bearer secret. Generate it with openssl rand -hex 32 and give the same value to every caller of the internal billing API (the billing queue consumer, operator tooling, the staging billing-trace e2e). The endpoints compare it in constant time and fail closed (401) when it is unset. Rotate by updating the value in the callers and in Lerma’s environment; no scope changes are involved.
Lerma Provisioning API Keys
Lerma supports both a platform-wide compatibility key and per-application keys:
| Name | Storage | Scope | Description |
|---|---|---|---|
PROVISIONING_API_KEY | Lerma runtime secret | Platform-wide | Optional global bearer for /api/provisioning/* and /api/ai-gateways/*. When unset, global-only API operations fail closed (401). |
STAGING_PROVISIONING_API_KEY | GitHub Actions secret only | Staging Fouria application | Used by .github/workflows/promote-staging.yml to upgrade the staging client tenant. It is not injected into Lerma’s runtime. |
Per-application keys are generated, rotated, and revoked by ROOT users on
/dashboard/applications/:id. Lerma stores only the SHA-256 digest in the
application row and reveals a newly generated key once. Copy the staging
Fouria application’s value to the GitHub secret STAGING_PROVISIONING_API_KEY;
never reuse it in production. Calls using an application key must send
X-FourIA-Application: <application-slug>, and Lerma filters the operation to
that application’s instances. The platform-wide key, when configured, remains a
full-scope compatibility credential.
Feature Flags
| Variable | Class | Container | Required | Description |
|---|---|---|---|---|
MOLTLAZY_FEATURE_FLAGS | Auto | Yes | Auto | JSON feature flags for moltlazy config generation (auto-generated). Includes the softwareProvisioning key (Worker-side: reconcile the declared software manifest at boot). |
WORKER_NAME | Variable | Yes | No | Worker name for config bucket key resolution. |
STARTUP_LOG_VERBOSITY | Variable | Yes | No | Clean boot-log verbosity for the loading page: steps | detailed (default) | verbose. |
OPENCODE_API_KEY | Secret | Yes | Dev | OpenCode (opencode-go) subscription key for dev instances; authenticates the dev agents and the opencode-credits-check automation. Never used for API billing. |
MOLTLAZY_AGENTS | Variable | Yes | Paso4 | Comma-separated premade agent roster to inject (moltlazy MOLTLAZY_AGENTS). Empty/unset means the platform default (all non-opt-in agents). |
MOLTLAZY_PASO4_AGENTS | Variable | Yes | Paso4 | true makes the Paso4-exclusive agent registry eligible (the opt-in paso4-dev agent). Set by lerma only on scope: "paso4" instances. |
GITHUB_TOKEN | Secret | Yes | Paso4 | GitHub token for the paso4-dev agent’s gh CLI. Deployed by the provisioner to Paso4-exclusive instances only; never present on client tenants. |
PASO4_TELEGRAM_GROUP_ID | Variable | Yes | Dev | Paso4 general Telegram group id for platform alerts (global lerma env; default -5442477312). |
TELEGRAM_INGRESS_HOST | Variable | No | No | Public Telegram webhook ingress host (hooks.fouria.io / hooks.dev.fouria.io). The Dispatch Worker matches it and routes to the tenant Worker without an Access JWT. |
TELEGRAM_INGRESS_QUEUE | Queue | No | Yes | Cloudflare Queue (fouria-telegram-ingress) buffering Telegram updates that arrive while the container is cold; the consumer wakes/heals and forwards with at-least-once delivery. |
TELEGRAM_INGRESS | DO | No | Yes | Per-tenant TelegramIngress Durable Object owning the fixed-window rate limiter for inbound webhook updates. |
Telegram webhook ingress (#364). Managed tenants switch Telegram to webhook mode. The webhook secret + bot token live in a Worker-readable F4E1 R2 registry (
telegram-ingress/<slug>/<account>); moltlazy writeschannels.telegram.accounts.<account>.webhook*from theTELEGRAM_WEBHOOK_MANIFESTenv var the Worker injects. The Dispatch Worker addsTELEGRAM_INGRESS_HOST(its own ingress routing) and the tenant Worker adds theTELEGRAM_INGRESSDO +TELEGRAM_INGRESS_QUEUEqueue bindings.
Stripe Billing Integration
Instance secrets for the Stripe subscription billing and payment management features in the Admin UI. Only accessed by the Worker (not the container).
| Variable | Class | Container | Required | Description |
|---|---|---|---|---|
STRIPE_SECRET_KEY | Instance Secret | No | No | Stripe secret key (sk_live_...) for creating Checkout and Customer Portal sessions. |
STRIPE_PUBLISHABLE_KEY | Instance Secret | No | No | Stripe publishable key (pk_live_...), exposed to the frontend for Stripe.js. |
BILLING_QUEUE | Queue binding | No | Yes | Cloudflare Queue producer for canonical billable actions to Lerma. |
Billing data (plan tier, status, seats, credits, Stripe customer ID) is sourced from the Supabase database via the Dispatch Worker. The Worker does not persist billing state in Durable Objects.
Communication Schema (Dispatch Worker → User Worker)
Billing metadata is passed from the Dispatch Worker to the User Worker via one of three resolution methods (in priority order):
| Priority | Method | Header / Source | Format |
|---|---|---|---|
| 1 (preferred) | Cloudflare for SaaS custom hostname metadata | request.cf.hostMetadata | JSON object — set via PATCH /zones/:id/custom_hostnames/:id with custom_metadata |
| 2 | Single metadata header | X-FourIA-Metadata | Base64url-encoded JSON of BillingMetadata |
| 3 (legacy) | Individual headers | X-FourIA-Plan, X-FourIA-SeatsUsed, etc. | String values (deprecated) |
Preferred approach (1): Set custom_metadata on each customer’s custom hostname via the Cloudflare API. The Dispatch Worker reads request.cf.hostMetadata and the User Worker reads it when trusted_workers: true is set on the dispatch namespace.
Fallback approach (2): The Dispatch Worker encodes the BillingMetadata object as base64url JSON and sets it as the X-FourIA-Metadata header when forwarding the request.
BillingMetadata Schema
interface BillingMetadata {
plan_tier: string; // "free" | "basic" | "pro" | "enterprise"
plan_name: string; // Human-readable name
billing_status: string; // "active" | "past_due" | "canceled" | "trialing" | "incomplete"
seats_used: number; // Current seat count
seats_max: number; // Plan seat limit
credits_used: number; // Credits consumed this period
credits_max: number; // Credit limit (0 = unlimited/pay-as-you-go)
stripe_customer_id: string | null; // Stripe customer for payments
period_end: string | null; // ISO 8601 — current billing period end
trial_end: string | null; // ISO 8601 — trial end (if trialing)
}RBAC Role Resolution
The user’s RBAC role is forwarded by the Dispatch Worker via the X-FourIA-Role header (one of ROOT, ADMIN, BASE_USER), when the Dispatch Worker can resolve it:
- Paso4 ROOT users are always
ROOT. A user qualifies when their JWTaudis the platform AUD (the*.fouria.iowildcard Access application) or their email is a member of the sharedPaso4 Root UsersAccess group. The group check matters because per-instance Access applications include the ROOT group policy (so ROOT operators are not locked out at{slug}.fouria.io); Cloudflare matches the most-specific application, so those tokens carry the per-instance AUD rather than the platform AUD. In staging this is the only permitted audience/identity. - Client users (per-instance Access application AUDs) get their role only when
client_users.status = validatedandclient_user_instanceslinks them to the requested instance. Pending, rejected, production-only, or dev-only assignments do not cross environment boundaries.
The User Worker’s resolveCustomerRole resolves the role in priority order:
- JWT
audmatchesCF_ACCESS_PLATFORM_AUD→ROOT X-FourIA-Roleheader (from the Dispatch Worker)- JWT
custom.roleclaim (IdP-provided) LOCAL_ROLE_MAP(local dev fallback)
Dashboard Tenant Provisioning (apps/lerma)
Variables required by the Phoenix dashboard to create and decommission tenant User Workers.
CF Access Local Development
Variables for local CF Access integration via cloudflared tunnel. See apps/lerma/bin/setup-cf-access-local.sh for one-time setup.
| Variable | Class | Required | Description |
|---|---|---|---|
CF_ACCESS_TEAM_DOMAIN | Variable | Yes* | Cloudflare Access team domain (e.g., paso4). Used by lerma’s dashboard auth and deployed to tenant workers. |
CF_ACCESS_AUD | Variable | Yes* | Dashboard’s own Access AUD (JWT audience for lerma’s dashboard). Populated by the deploy flow from CF_ACCESS_LERMA_AUD/STAGING_CF_ACCESS_LERMA_AUD. |
CF_ACCESS_PLATFORM_AUD | Variable | Yes* | Platform-level Access AUD (Paso4 Root app). Deployed to every tenant worker as the CF_ACCESS_PLATFORM_AUD secret so its auth middleware can validate JWTs forwarded by the Dispatch Worker. |
CF_ACCESS_SERVICE_TOKEN_ID | Instance Secret | No | CF Access service token Client ID for local CI/CD (non-interactive). |
CF_ACCESS_SERVICE_TOKEN_SECRET | Instance Secret | No | CF Access service token Client Secret for local CI/CD. |
CLOUDFLARED_TUNNEL_TOKEN | Instance Secret | No | Tunnel token for lerma-local (routes to port 4001). |
SKIP_AUTH | Variable | No | Set to "true" to bypass CF Access JWT verification (escape hatch). |
Lerma staging healthcheck tunnel
The staging Hetzner host runs a remotely managed Cloudflare Tunnel sidecar at
lerma-health-dev.paso4.io. Its ingress forwards only the minimal
GET /health response (ok) to Lerma; all other paths return 404. The existing
dashboard hostname and Access policies are unchanged. The tunnel does not replace
the existing staging A record or close its current origin ports.
| Variable | Class | Required | Description |
|---|---|---|---|
TF_VAR_healthcheck_tunnel_api_token | Local IaC Secret | Apply | Dedicated Cloudflare token used only by the staging OpenTofu apply; requires Account Tunnel:Edit and paso4.io Zone DNS:Edit. Do not reuse LERMA_API_TOKEN. |
LERMA_STAGING_HEALTHCHECK_TUNNEL_TOKEN | GitHub Secret | Staging | Remote-managed Tunnel run token passed through Ansible to the staging-only cloudflared sidecar. It is not passed to the Lerma Phoenix app. |
Invitation Email
Lerma delivers invitation confirmation emails through Cloudflare Email Service over authenticated SMTP submission (smtp.mx.cloudflare.net:465, implicit TLS). It reuses the existing provisioning credential as the SMTP password, so no additional secret is required.
| Variable | Class | Required | Description |
|---|---|---|---|
LERMA_API_TOKEN | Secret | Yes | Cloudflare API token. Must include the Email Sending: Edit permission; used as the SMTP password (username is the literal api_token). The From domain must be onboarded for Email Sending. |
Local development continues to use Swoosh’s local mailbox adapter; tests use the Swoosh test adapter.
Provisioning Configuration
| Variable | Class | Required | Description |
|---|---|---|---|
CLOUDFLARE_API_TOKEN | ENV only | Yes | API token with Workers:Edit, R2:Edit, Containers:Edit, and D1 edit permissions. Containers:Edit is required — the dashboard generates short-lived registry credentials (POST /containers/registries/{host}/credentials) to list versions, and uses R2:Edit to fetch the worker bundle from the fouria-bundles R2 bucket at provisioning time. Without it the calls fail with 403 Authentication error. |
CLOUDFLARE_ACCOUNT_ID | Variable | Yes | Paso4 Cloudflare account ID for provisioning calls. Must be set or the registry-health check reports it missing and registry-credentials calls fail with Could not route to /accounts/containers/registries/... — object identifier is invalid (the account ID is interpolated into the request path). On the fouria User Worker it is a plain var: the Sandbox SDK reads it for R2 presigned URL signing (backup/restore) — missing it throws InvalidBackupConfigError (Backup requires R2 presigned URL credentials). The dashboard health check also validates that CLOUDFLARE_API_TOKEN grants the required permission groups (Workers Scripts Write, DNS Write, Access: Apps and Policies Write, Workers Containers Write), flagging a missing one as “Limited access”. Legacy CF_ACCOUNT_ID is deprecated. |
CONTAINER_REGISTRY_HOST | Variable | No | Cloudflare managed container registry host (default: registry.cloudflare.com). Mirrors CLOUDFLARE_CONTAINER_REGISTRY in wrangler. |
CONTAINER_IMAGE_NAME | Variable | No | Container image repo name for the tenant sandbox image (default: fouria). |
CONTAINER_BUNDLE_R2_BUCKET | Variable | No | R2 bucket that publishes the worker bundle fouria.mjs (default: fouria-bundles). CI uploads the bundle to <version>/fouria.mjs and the dashboard fetches it from R2 at provisioning time. |
FOURIA_DISPATCH_NAMESPACE | Variable | No | Workers for Platforms dispatch namespace name (default: fouria-dispatch). |
FOURIA_R2_PUBLIC_ENDPOINT | Variable | No | R2 public endpoint template, e.g. https://pub-{bucket_id}.r2.dev. |
ANALYTICS_DATASET | Variable | No | Analytics Engine dataset name (default: fourai_cost_metrics, staging: fourai_cost_metrics_staging). |
CONTAINER_IMAGE | Variable | No | Container image reference for the tenant’s container application (linked to the Sandbox DO namespace). Example: registry.cloudflare.com/<account_id>/fouria:latest. When unset, lerma derives it from the published bundle version as registry.cloudflare.com/<account_id>/fouria:<image_tag>. |
CONTAINER_MAX_INSTANCES | Variable | No | Max container instances for tenant container apps (default: 3). |
DEV_INSTANCE_SSH_PUBLIC_KEY | Secret | No | Optional single-line ssh-ed25519 public key Lerma applies to Cloudflare Container authorized_keys only for dev instances. It is never a Worker/container environment binding; keep the private key on the operator workstation. Access uses wrangler containers ssh (Cloudflare-authenticated; no public SSH port). Existing dev apps receive key changes on their next Lerma redeploy. |
WORKER_OBSERVABILITY_ENABLED | Variable | No | Enable Workers Observability (logs + traces) on deployed tenant User Workers (default: true). Mirrors apps/fouria/wrangler.jsonc. |
WORKER_OBSERVABILITY_SAMPLING_RATE | Variable | No | Head sampling rate for traces on tenant User Workers, 0..1 (default: 1). |
WORKER_OBSERVABILITY_LOGS | Variable | No | Enable logs on tenant User Workers (default: true). |
WORKER_OBSERVABILITY_INVOCATION_LOGS | Variable | No | Enable invocation logs on tenant User Workers (default: true). |
WORKER_OBSERVABILITY_TRACES | Variable | No | Enable traces on tenant User Workers (default: true). |
WORKER_OBSERVABILITY_LOGS_DESTINATIONS | Variable | No | Comma-separated account-level Cloudflare Observability Logs destination names (Observability → Destinations), referenced by name in the worker upload metadata. Empty → no export destination. |
WORKER_OBSERVABILITY_TRACES_DESTINATIONS | Variable | No | Comma-separated account-level Cloudflare Observability Traces destination names. Empty → no export destination. |
WORKER_OBSERVABILITY_LOGS_SAMPLING_RATE | Variable | No | Logs head sampling rate, 0..1. Falls back to WORKER_OBSERVABILITY_SAMPLING_RATE. Use 0.1 for a 10% pilot. |
WORKER_OBSERVABILITY_TRACES_SAMPLING_RATE | Variable | No | Traces head sampling rate, 0..1. Falls back to WORKER_OBSERVABILITY_SAMPLING_RATE. The uptime widget/API report this rate and treat “no events” as unknown, never downtime. |
WORKER_OBSERVABILITY_LOGS_PERSIST | Variable | No | Keep the Cloudflare-side copy when an export destination is configured (default: false — Lerma is the destination of record). |
WORKER_OBSERVABILITY_TRACES_PERSIST | Variable | No | Keep the Cloudflare-side trace copy when an export destination is configured (default: false). |
CONTAINER_OBSERVABILITY_LOGS | Variable | No | Enable log capture on the tenant’s container application (default: true). |
CONTAINER_REGIONS | Variable | No | Container application placement regions (deployment zone), comma-separated (default: WEUR). Valid values: ENAM, WNAM, EEUR, WEUR, APAC, SAM, ME, OC, AFR. |
R2_TOKEN_STRATEGY | Variable | No | How lerma resolves the mandatory per-tenant R2 S3 backup credential: auto (default), shared, or per_instance. auto uses R2_TOKEN_MAX_PER_ACCOUNT: a known limit <= 250 selects shared, > 250 or unknown selects per_instance. per_instance mints a scoped R2 token per instance at provisioning. See Lerma.Provisioning.R2Credentials. |
R2_TOKEN_MAX_PER_ACCOUNT | Variable | No | Account API-token limit used by :auto (default: unknown → :per_instance). Cloudflare exposes no endpoint for the maximum, so pin a known value here if desired. |
R2_ACCESS_KEY_ID | Secret | Cond. | Lerma-level shared R2 S3 access key, required when the effective strategy is shared. Kept as a Worker binding for the SDK presigned-URL backup path (never forwarded to the container — sandbox R2 access uses the outboundByHost bridges). Generate with mix lerma.r2_token --all-buckets. |
R2_SECRET_ACCESS_KEY | Secret | Cond. | Lerma-level shared R2 S3 secret key, paired with R2_ACCESS_KEY_ID. Required when the effective strategy is shared; provisioning fails with :missing_shared_r2_credentials otherwise. Worker binding only (see above). |
Observability destination ingestion (Lerma)
Cloudflare account-level Observability → Destinations push Workers Logs and Workers Traces to Lerma over OTLP/HTTP JSON. This is a push stream: it adds no per-tenant polling, does not replace HealthMonitor/billing, and never wakes a sleeping container. HealthMonitor continues to drive instances.status/clients.status; destination telemetry only supplies diagnostics and uptime statistics.
| Variable | Class | Required | Description |
|---|---|---|---|
OTEL_INGEST_TOKEN | Runtime Secret | No | Shared bearer secret configured as the destination’s Authorization: Bearer … header. The ingest endpoints compare it in constant time and fail closed (401) when unset. Generate with openssl rand -hex 32. |
OTEL_INGEST_MAX_EVENTS | Variable | No | Maximum sanitized rows accepted per ingest request (default 1000). The remainder is reported as skipped. |
OTEL_INGEST_RETENTION_DAYS | Variable | No | Retention for stored destination telemetry (default 90, matching PRIVACY.md). |
OTEL_INGEST_PRUNE_INTERVAL_MS | Variable | No | Retention sweep interval (default 86400000 / 24 h). A first sweep runs one minute after boot, then every interval. |
Destination names per environment:
| Environment | Logs destination | Traces destination | Lerma OTLP base |
|---|---|---|---|
| staging | lerma-worker-logs | lerma-worker-traces | https://lerma-dev.paso4.io/api/ingest/otel/v1 |
| production | lerma-worker-logs-prod | lerma-worker-traces-prod | https://lerma.paso4.io/api/ingest/otel/v1 |
apps/fouria/wrangler.jsonc references the staging names in env.staging and
the production names in env.production; Lerma’s CD passes the same names to
each environment’s tenant-worker metadata. The account-level destinations must
exist before a Worker that references them is deployed.
-
In Observability → Destinations, create the pair for the environment you are rolling out (staging first), pointing at that environment’s Lerma host. Add the custom header
Authorization: Bearer <OTEL_INGEST_TOKEN>to both. The API equivalent is:# staging values shown; for production use LERMA_HOST=lerma.paso4.io and # the `-prod` destination names. LERMA_HOST=lerma-dev.paso4.io LOGS_NAME=lerma-worker-logs TRACES_NAME=lerma-worker-traces for pair in "$LOGS_NAME opentelemetry-logs logs" "$TRACES_NAME opentelemetry-traces traces"; do set -- $pair curl -sS "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/workers/observability/destinations" \ -H "Authorization: Bearer $CF_OBSERVABILITY_TOKEN" -H 'Content-Type: application/json' \ -d "{\"name\":\"$1\",\"enabled\":true,\"configuration\":{\"type\":\"logpush\",\"logpushDataset\":\"$2\",\"url\":\"https://$LERMA_HOST/api/ingest/otel/v1/$3\",\"headers\":{\"Authorization\":\"Bearer $OTEL_INGEST_TOKEN\"}}}" doneUse a credential with Workers Observability Write (the standard
LERMA_API_TOKENmay need that permission added). Cloudflare’s destination preflight needs a live, decompressing receiver, so create the staging pair after staging Lerma is on the gzip-aware release, and the production pair after production Lerma is.The dashboard CD runs
mix lerma.observability.destinations --environment <env> --warn-onlybefore each deploy (staging and production). It lists the destinations, creates missing ones, and PATCHes existing ones so a rotatedOTEL_INGEST_TOKENpropagates. It prefersCF_OBSERVABILITY_TOKENand falls back toLERMA_API_TOKEN; either must carry Workers Observability Write. With--warn-only, a missing permission logs a warning instead of failing the deploy, so add that permission to the token (or set a dedicatedCF_OBSERVABILITY_TOKEN) to make the rollout fully self-healing. -
Set
OTEL_INGEST_TOKENon that Lerma deployment and setWORKER_OBSERVABILITY_LOGS_DESTINATIONS/WORKER_OBSERVABILITY_TRACES_DESTINATIONSto the environment’s two names. The CD does this automatically (staging: un-suffixed names; production:-prodnames). -
Redeploy the tenant workers so their upload metadata references the destinations (new tenants inherit them automatically). Existing tenants pick them up on their next Lerma upgrade/redeploy.
-
Confirm delivery on the destination status (Last: n minutes ago) and query
GET /api/analytics/v1/uptime/tenant-<slug>?window=24h.
The ingest endpoints accept OTLP/HTTP JSON only (Cloudflare gzips the body while keeping Content-Type: application/json, so LermaWeb.Plugs.OtlpBodyReader decompresses it before parsing) and answer 202 with {accepted, skipped} so Cloudflare does not retry-loop; binary protobuf is rejected with 415 and a hint. Ingestion is idempotent — the destinations deliver at least once, and a replayed batch adds no rows. Only allowlisted metadata is stored: log message text, stack traces, URLs/query strings, user agents, request/response bodies and DB query text are dropped before persistence.
Health Monitoring
Lerma’s HealthMonitor periodically probes each tenant’s public GET /api/health endpoint and keeps clients.status and instances.status accurate. /api/health is the unified health endpoint (returns startup/readiness/liveness probes plus the canonical status and processId); GET /api/status is a legacy alias over the same snapshot. Both are exposed by every User Worker without auth and are made publicly reachable via path-scoped Cloudflare Access applications (*.fouria.io/api/health, *.fouria.io/api/status, see apps/fouria/iac/access.tf). Probing also cold-starts asleep containers (ensureGateway), so a stopped VM recovers on the next check — pass ?passive=1 (used by ROOT diagnostics) to probe without warming.
Each instance is probed at its own environment hostname (acme.fouria.io for production, acme-dev.fouria.io for dev).
up means the gateway is running. A probe is up only when the snapshot reports status: "running". A 200 status: "sleeping" (an idle container whose gateway is not running) is not up: it records a down transition for the uptime widget and marks the instance sleeping (alive, but not running). A 503 (stopped) or a transport failure records a down transition but leaves the instance status untouched — health is observability-only, so a probe miss (no traffic, a slow cold start, or an Access challenge) never flips a tenant to error or takes it out of service. A sleeping/stopped probe also does not recover a client left in error; only a genuine running probe does.
An exact-host per-tenant Access app shadows the wildcard health app, so the provisioner (Lerma.Cloudflare.AccessPolicy) also creates a per-tenant health application ({slug}.fouria.io/api/health) that carries no role requirement (any authenticated user) and allows the health-monitor service token. The monitor authenticates with HEALTH_MONITOR_ACCESS_CLIENT_ID / HEALTH_MONITOR_ACCESS_CLIENT_SECRET.
The dashboard CD (.github/workflows/deploy-dashboard.yml) wires these credentials for both staging and production: it passes HEALTH_MONITOR_ACCESS_CLIENT_ID, HEALTH_MONITOR_ACCESS_CLIENT_SECRET, and CF_ACCESS_HEALTH_MONITOR_TOKEN_ID to Ansible as health_monitor_access_client_id / health_monitor_access_client_secret / cf_access_health_monitor_token_id. When they are omitted, the deployed Lerma probes go out unauthenticated and every Access-protected tenant answers 302 (an Access login redirect), which the probes surface as an Access challenge.
| Variable | Class | Required | Description |
|---|---|---|---|
HEALTH_MONITOR_ENABLED | Variable | No | Enable the periodic health monitor (default: true). Always disabled in test env. |
HEALTH_MONITOR_INTERVAL_MS | Variable | No | Poll interval in milliseconds (default: 120000). |
HEALTH_MONITOR_TIMEOUT_MS | Variable | No | Per-probe HTTP timeout in milliseconds (default: 15000). |
HEALTH_MONITOR_PLATFORM_DOMAIN | Variable | No | Domain the tenant workers are served under (default: fouria.io). |
HEALTH_MONITOR_SLUG_PREFIX | Variable | No | Hostname prefix for staging tenants (default: "", staging auto-set to staging-). |
HEALTH_MONITOR_ACCESS_CLIENT_ID | Secret | No | Cloudflare Access service-token client ID sent on each probe (cf-access-client-id). Required for probing Access-protected tenants. |
HEALTH_MONITOR_ACCESS_CLIENT_SECRET | Secret | No | Cloudflare Access service-token client secret (cf-access-client-secret). |
CF_ACCESS_HEALTH_MONITOR_TOKEN_ID | Variable | No | Pins the service-token id granted access to each per-tenant health app. When empty, AccessPolicy resolves the FourIA Health Monitor token by name. |
INTERNAL_DISPATCH_INVALIDATE_TOKEN | Secret | No | Shared bearer Lerma sends on the Dispatch Worker’s internal routes (/_control/*, /_invalidate/*). Must equal the Dispatch Worker’s hardcoded key (internal-fouria-dispatch-key); without it those calls return 403 and the pre-replacement checkpoint cannot run. Defaults to that key in the deploy templates. |
Dashboard connectivity checks (StartupHealth)
Lerma’s StartupHealth runs external connectivity checks (Cloudflare API token validity/permissions and the container image registry) and caches the result in ETS. The result drives the dashboard connectivity banner (/dashboard) and the /health/liveness + /health/readiness probes. Checks run at startup and then on a recurring interval so a transient boot-time failure (e.g. a registry request timing out) clears itself once connectivity is restored: the banner updates live over PubSub and the readiness probe recovers, without restarting the app.
| Variable | Class | Required | Description |
|---|---|---|---|
STARTUP_HEALTH_INTERVAL_MS | Variable | No | Re-check interval in milliseconds (default: 300000). Set to 0 to run the connectivity checks only once at startup (startup-only behaviour). |
Provisioning health probe (HealthPoller)
Health verification is asynchronous and post-deployment: once a provisioning
run (or upgrade) is finalized, the provisioner schedules a background task that
runs the non-passive /api/health probe (HealthPoller). It is not a
deployment step, so a slow container cold start (the Worker’s boot budget is
STARTUP_TIMEOUT_MS, 300s) never blocks, times out, or rolls back a deployment.
A failed probe is observability-only: the downtime is recorded for the
instance uptime widget, but the instance status is left untouched (a probe miss
is not an outage). The poll succeeds (:ok) only when the snapshot reports
status: "running" — a 200 status: "sleeping" is not ready and keeps
retrying until the deadline. The background check is disabled with
skip_health_poll: true.
The probe targets the instance’s environment hostname — the client slug plus
the -dev environment suffix for a dev instance, then the HEALTH_MONITOR_SLUG_PREFIX
platform prefix ({prefix}{slug}[-dev].fouria.io). The platform suffix (-staging)
is deliberately not part of the hostname; hostname construction must never
reuse the routable_slug used for Cloudflare resource names (R2/AI Gateway),
or the platform axis is applied twice.
Upgrade rollback never deletes the existing worker script / container application, so a transient failure cannot destroy a running tenant. Fresh provisions still clean up the resources created for the new tenant.
| Variable | Class | Required | Description |
|---|---|---|---|
HEALTH_PROBE_TIMEOUT_MS | Variable | No | Provisioning health-probe deadline in milliseconds (default: 360000). Must exceed the Worker cold-start budget (300s). |
HEALTH_PROBE_BACKOFF_MS | Variable | No | Delay between provisioning health-probe attempts in milliseconds (default: 3000). |
How Secrets Are Deployed
The provisioner deploys all instance secrets as secret_text bindings directly on the User Worker. These are encrypted at rest by Cloudflare and available at runtime as plain string values (not Secrets Store objects).
The provisioner also creates an empty Cloudflare Secrets Store (fouria-{slug}) bound to the Worker for future dynamic secret use. Instance secrets are never placed in this store.
# Per-tenant secrets (generated by lerma provisioner)
%{type: "secret_text", name: "MOLTBOT_GATEWAY_TOKEN", text: gateway_token}
%{type: "secret_text", name: "CDP_SECRET", text: value}
%{type: "secret_text", name: "CF_ACCESS_PLATFORM_AUD", text: platform_aud}
%{type: "secret_text", name: "CF_ACCESS_CLIENT_AUD", text: tenant_aud} # Optional
# Platform-level secrets (from Lerma ENV, optional)
%{type: "secret_text", name: "CLOUDFLARE_AI_GATEWAY_API_KEY", text: ai_gateway_key}
%{type: "secret_text", name: "CLOUDFLARE_API_TOKEN", text: ai_gateway_key} # Tenant REST alias
%{type: "secret_text", name: "STRIPE_SECRET_KEY", text: value}
# ... etc.
See apps/lerma/lib/lerma/provisioning/tenant_provisioner.ex for the authoritative binding list.
Container-Backed Durable Objects
The tenant User Worker’s Sandbox DO class is a container-backed Durable Object (it extends
Container from @cloudflare/sandbox). lerma deploys the worker into the dispatch namespace
with a containers array in the upload metadata (declaring Sandbox), which enables the
container runtime for that class — without it, Cloudflare throws
Containers have not been enabled for this Durable Object class when the worker calls into the
container (e.g. /api/doctor). lerma also creates/updates a container application for the
tenant (image + instance type + max instances) linked to the Sandbox DO namespace.
Container app image comes from CONTAINER_IMAGE, and the container instance type is derived
from the instance type chosen at provisioning time. When none is chosen it defaults to basic
(4 GB disk — lite’s 2 GB disk is too small to unpack the sandbox image).
CONTAINER_MAX_INSTANCES configures the max instances (see table above).
The tenant container application is pinned to the WEUR deployment zone by default
(CONTAINER_REGIONS) and container log capture is enabled (CONTAINER_OBSERVABILITY_LOGS).
Observability for the User Worker itself (logs + traces + invocation logs) mirrors the
apps/fouria/wrangler.jsonc block and is enabled by default
(WORKER_OBSERVABILITY_* vars above).
Legacy Channel Secrets (E2E Testing Only)
Deprecated. Channel secrets are now managed through the integrations dashboard (F4E1-encrypted R2 vault). These env vars are kept only for E2E test fixtures. Do not use in production.
Secrets reference: See docs/schemas/secrets-manifest.schema.json for the canonical credential inventory.
Internal Variables (Auto-Set)
These are automatically derived by the Worker and should never be manually configured:
| Variable | Source |
|---|---|
OPENCLAW_GATEWAY_TOKEN | Mapped from MOLTBOT_GATEWAY_TOKEN |
OPENCLAW_DEV_MODE | Derived from the -dev hostname (WORKER_URL) or an explicit OPENCLAW_DEV_MODE override; DEV_MODE for local development |
OPENCLAW_E2E_TEST_MODE | Mapped from E2E_TEST_MODE |
CUSTOMER_SLUG | Set from tenant resolution |
OTEL_EXPORTER_OTLP_HEADERS | Auto: Authorization=Bearer {OTEL_SHARED_SECRET} |
OTEL_EXPORTER_OTLP_METRICS_ENDPOINT | Auto: {WORKER_URL}/api/otel/v1/metrics |
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT | Auto: {WORKER_URL}/api/otel/v1/traces (unset when WORKER_URL unset or E2E mode) |
OTEL_TRACES_ENABLED | Forwarded when explicitly set ('true' enables diagnostics.otel.traces; default: disabled) |
MOLTLAZY_FEATURE_FLAGS | Set as per-customer binding (managed via FourIA API); the staging wrangler.jsonc env default enables {"softwareProvisioning":true} |
MOLTLAZY_ANALYTICS_DATASET | Auto-set to tenant Analytics Engine dataset |
CUSTOMER_ANALYTICS_DATASET | Auto-set to tenant Analytics Engine dataset |
STARTUP_LOG_VERBOSITY | Passed through from Worker env when set (detailed default in container) |
OPENCODE_API_KEY | Deployed by the provisioner to dev instances (lerma env); authenticates the opencode-go dev agents + credits automation |
MOLTLAZY_AGENTS | Deployed by the provisioner when an instance has a selected agent roster; comma-separated premade agent ids |
MOLTLAZY_PASO4_AGENTS | Deployed by the provisioner ("true") only on Paso4-exclusive (scope: "paso4") instances |
GITHUB_TOKEN | Deployed by the provisioner to Paso4-exclusive instances only; authenticates the paso4-dev agent’s gh CLI |
PASO4_TELEGRAM_GROUP_ID | Deployed by the provisioner to dev instances (lerma env, default -5442477312); Telegram group for platform alerts |
OPENCLAW_CONFIG_PATH | Baked into image: /root/openclaw/.openclaw/openclaw.json — the REAL config path (see below) |
OPENCLAW_STATE_DIR | Baked into image: /root/openclaw/.openclaw — the REAL state dir (see below) |
Why
OPENCLAW_CONFIG_PATH/OPENCLAW_STATE_DIRpoint at/root/openclaw/.openclaw:HOME=/home/openclawand/home/openclaw/.openclawis a symlink to the real config dir at/root/openclaw/.openclaw(kept under the/home/openclawsnapshot root for R2 backups). OpenClaw 2026.8.2+ fs-safe refuses atomic file replaces when the immediate parent directory is a symlink (“Atomic replace parent must be a real directory”), so config/state writes through the/homesymlink fail. These two env vars point OpenClaw at the real path for its own writes; the/home/openclaw/.openclawsymlink is preserved so backups and the Worker’s/home-based config reads/writes keep working. Set in the Dockerfile (apps/fouria/Dockerfile), not per-tenant.
Setting Variables
Local Development (.dev.vars)
Copy from .dev.vars.example:
cp apps/fouria/.dev.vars.example apps/fouria/.dev.vars
# Edit .dev.vars with your values
# .dev.vars is gitignored — never commit it
In local dev, instance secrets are plain strings (simulating secret_text bindings). Dynamic secrets need BACKUP_ENCRYPTION_KEY configured with the F4E1 R2 store.
Production Deployment
Instance secrets are deployed automatically by lerma when a tenant is provisioned. No manual wrangler secret put commands are needed.
Platform-level secrets should be set as environment variables in the dashboard’s deployment:
STRIPE_SECRET_KEY=sk_live_...
OTEL_SHARED_SECRET=your-secret
# ... etc.OpenTofu (IaC) Mapping
Secrets reference: See docs/schemas/secrets-manifest.schema.json for the canonical credential inventory.