Deploy M2 — Public-internet exposure (v4.5)

Status: Released in v4.5 (M2 of the Whilly Distributed v5.0 mission). Pairs with docker-compose.control-plane.yml + docker-compose.demo.yml (M2 adds the profile-gated funnel sidecar service to both), the bootstrap_tokens table (migration 009), the funnel_url table (migration 010), and the whilly admin CLI namespace. The single-host workshop demo (docker-compose.demo.yml + workshop-demo.sh) is unchanged when the new funnel profile is OFF — see DEMO.md.

Pivot note (2026-05-02). The earlier draft of M2 fronted the control-plane with either Caddy + Let’s Encrypt or Tailscale Funnel. Both paths were CANCELLED. M2 now ships exactly one public- exposure mechanism: a localhost.run sidecar (free anonymous SSH reverse tunnel, wildcard *.lhr.rocks cert managed upstream). This doc reflects that decision.

This is the operator-facing deploy + ops doc for v4.5. It covers:

  1. The two supported topologies (laptop-host vs VPS-host).
  2. The localhost.run “staging vs prod” decision matrix (free anonymous tier vs SSH-key stable URL).
  3. Bringing up the funnel sidecar end-to-end.
  4. Adjacent runbooks: cert renewal (docs/Cert-Renewal.md) and admin token rotation (docs/Token-Rotation.md).

Contents

  1. What’s new in v4.5 (M2)
  2. Decision matrix — pick a topology and a tier
  3. staging vs prod warning callout
  4. Topology A — laptop-host control-plane
  5. Topology B — VPS-host control-plane
  6. Worker-side URL re-discovery
  7. Adjacent runbooks
  8. Reference: env vars added in v4.5

What’s new in v4.5 (M2)

Surface Change
docker-compose.demo.yml Adds funnel service under profiles: ["funnel"]. Default docker compose up is byte-equivalent to v4.4 — funnel only starts with --profile funnel.
docker-compose.control-plane.yml Adds the same profile-gated funnel service for VPS deployments.
Dockerfile.funnel New small alpine image (alpine + openssh-client + bash + curl + postgresql-client, ≤ 32 MB).
Migration 008 workers.owner_email column + partial index.
Migration 009 bootstrap_tokens table — per-operator bootstrap minting.
Migration 010 funnel_url singleton table — sidecar publishes the live lhr.rocks URL here.
whilly admin bootstrap mint\|revoke\|list Per-operator bootstrap CLI; never reveals plaintext after mint.
whilly admin worker revoke <id> Live worker eviction + RELEASE of in-flight tasks.
make_admin_auth factory DB-backed admin bearer with is_admin scope check; legacy WHILLY_WORKER_BOOTSTRAP_TOKEN falls back to non-admin (one-minor compat window).
Worker env WHILLY_FUNNEL_URL_SOURCE static (default, back-compat) / postgres / file — re-discovers the funnel URL on rotation.

Decision matrix — pick a topology and a tier

There are two independent choices: where the control-plane runs, and which localhost.run tier you use. Pick one row from each.

Topology — where does the control-plane live?

Topology When to use it What it costs Reference section
A. Laptop-host Hands-on demo, working session with one or two colleagues, you don’t own a VPS. Your laptop must stay online; sidecar consumes one outbound TCP/22 to localhost.run. Topology A
B. VPS-host Long-running cluster, several workers, control-plane survives laptop sleep. A VPS (≥ 1 GB RAM, ≥ 2 GB disk, public outbound TCP/22). Topology B

Either way the funnel sidecar runs on whichever host owns the control-plane (laptop or VPS) and publishes a rotating https://<random>.lhr.rocks URL that workers anywhere on the public internet can reach.

localhost.run tier — staging vs prod

The localhost.run service offers two operationally distinct tiers. Treat them like Let’s Encrypt’s staging vs prod environments: one is free + ephemeral + safe-to-poke-at, the other is for real users.

Tier URL shape Lifetime Auth required Use when
Free anonymous (staging) https://<random>.lhr.rocks Rotates “after a few hours” of session lifetime None — nokey@localhost.run over SSH Demos, proofs of concept, M2 sign-off, anything you can re-share a fresh URL for. Default in v4.5.
SSH-key stable (prod) https://<your-name>.lhr.rocks (or your own custom domain on the paid tier) Stable across reconnects Free localhost.run account + dedicated SSH key registered with them A cluster you want to put into a colleague’s ~/.bashrc. Deferred to M3 in this mission, but supported by the sidecar today if you wire your own key in.

staging vs prod warning. The free anonymous tier rotates the public URL on a cadence localhost.run does not pin (their FAQ says “after a few hours”). Do NOT bake the live lhr.rocks URL into any persistent artifact — .bashrc, GitHub Actions secret, wiki page, kubeconfig, monitoring config, etc. Workers must read the URL through WHILLY_FUNNEL_URL_SOURCE=postgres or =file so rotations are absorbed transparently. If you need a stable URL, upgrade to the SSH-key path (deferred to M3) — do not pretend the anonymous tier is stable.

A second consequence of the rotation cadence: validators, dashboards, and monitoring systems reading /health or /api/v1/tasks over the public URL must re-fetch the URL from postgres / the shared file between attempts; never cache it across runs.

staging vs prod warning callout

⚠️ Read this before pasting any *.lhr.rocks URL anywhere.

The free-tier (staging) localhost.run URL is ephemeral by design. It rotates on a cadence the upstream documents as “after a few hours” of session connection. Treat it the same way you treat a Let’s Encrypt staging certificate: useful for end-to-end smoke tests, never trust-on-first-use as a permanent endpoint.

The prod path (SSH-key stable URL via a free localhost.run account, deferred to M3 in this mission) is what you want for anything that survives a laptop sleep. M2 ships the sidecar; M3 adds the SSH-key wiring + funnel_url-stability promise.

Workers absorb rotations when WHILLY_FUNNEL_URL_SOURCE is postgres (poll the funnel_url table every 30 s) or file (poll /funnel/url.txt every 5 s). Default is static for back-compat; if you set static you must restart the worker by hand on every rotation.

Topology A — laptop-host control-plane

End-to-end recipe: control-plane + sidecar on a macbook, workers on a VPS / second laptop / phone-tethered colleague.

A.1. Bring up the control-plane + sidecar 

cd /opt/develop/whilly-orchestrator
git checkout v4.5.0     # or `main` for unreleased

export WHILLY_WORKER_BOOTSTRAP_TOKEN="$(openssl rand -hex 32)"

# Profile-gated; default `up` is unchanged from v4.4.
docker compose -f docker-compose.demo.yml \
    --profile funnel \
    up -d

# Verify postgres + control-plane are healthy:
docker compose -f docker-compose.demo.yml ps

# Verify the sidecar parsed an `lhr.rocks` URL within ~10 s:
docker compose -f docker-compose.demo.yml logs funnel | grep -E 'https://[a-z0-9-]+\.lhr\.life'

A.2. Mint a per-operator bootstrap token

Replace the legacy shared WHILLY_WORKER_BOOTSTRAP_TOKEN env var with per-operator rows from the bootstrap_tokens table.

# WHILLY_DATABASE_URL must point at the control-plane Postgres
# (same DSN your control-plane is configured with — see ./scripts/db-up.sh
# and docker-compose.demo.yml for the demo credentials shape).
whilly admin bootstrap mint --owner alice@example.com --expires-in 30d
# ⇒ token: <plaintext>     ← capture this once, never reprintable
#   owner: alice@example.com
#   token_hash: <sha256-prefix>
#   is_admin: false
#   expires_at: 2026-06-02T...

The plaintext is shown once. Re-running whilly admin bootstrap list only ever shows the truncated token_hash — capture the plaintext at mint time and hand it to alice through whatever secure channel you’d normally use for shared secrets.

A.3. Discover the live lhr.rocks URL 

URL=$(psql "$WHILLY_DATABASE_URL" -t -A -c \
    "SELECT url FROM funnel_url ORDER BY updated_at DESC LIMIT 1")
echo "$URL"     # → https://abc123def456.lhr.rocks

A.4. Worker (any host) joins via the lhr.rocks URL 

# Either pip install (Python path) or use the Docker worker image.
pip install 'whilly-orchestrator[worker]'

whilly worker connect "$URL" \
    --bootstrap-token "$ALICE_BOOTSTRAP_TOKEN" \
    --plan demo \
    --hostname "$(hostname)"

# No --insecure needed — localhost.run terminates a real
# Let's Encrypt prod cert at the edge. The worker's URL-scheme
# guard accepts the URL because it's HTTPS.

The --insecure loopback-bypass is not needed here. That flag exists for plain-HTTP-to-non-loopback edge cases (LAN demos before M2 lands). Once you’re on https://*.lhr.rocks, drop it.

For URL-rotation tolerance, the worker host should run with WHILLY_FUNNEL_URL_SOURCE=postgres (preferred) or =file. See Worker-side URL re-discovery below.

Topology B — VPS-host control-plane

The same flow, but the control-plane + sidecar run on a public VPS. Useful when you want the cluster to survive your laptop sleeping.

export VPS_HOST=vps.example.com
ssh root@$VPS_HOST
cd /root/whilly
git checkout v4.5.0

export WHILLY_WORKER_BOOTSTRAP_TOKEN="$(openssl rand -hex 32)"

# Postgres + control-plane bound to loopback (default), sidecar
# bridges them to localhost.run.
docker compose -f docker-compose.control-plane.yml \
    --profile funnel \
    up -d

docker compose -f docker-compose.control-plane.yml logs funnel \
    | grep -E 'https://[a-z0-9-]+\.lhr\.life'

Workers on laptops then read the URL the same way as Topology A.4.

VPS resource note. The 964 MB-RAM VPS profile keeps the stack under 600 MB total — postgres ~256 MB, control-plane ~256 MB, funnel sidecar ~32 MB. Validate after bring-up: docker stats --no-stream.

Worker-side URL re-discovery

The localhost.run free-tier URL rotates “after a few hours”. Workers absorb the rotation transparently when one of the two non-static discovery modes is enabled:

WHILLY_FUNNEL_URL_SOURCE Default Behaviour
static yes — back-compat Use WHILLY_CONTROL_URL verbatim. Operator must restart the worker on rotation.
postgres no Poll funnel_url table every WHILLY_FUNNEL_URL_POLL_SECONDS (default 30 s). On change: release in-flight task, re-register against the new URL with the same worker_id, resume long-poll.
file no Same, but reads WHILLY_FUNNEL_URL_FILE (default /funnel/url.txt). 5 s default cadence.

Re-registration is idempotent server-side: the same worker_id row gets its last_heartbeat column updated in place. The bearer in the OS keychain is reused if still valid; otherwise the worker re-runs the bootstrap flow with its stored bootstrap token.

The publishing side (sidecar → funnel_url table + /funnel/url.txt) ships with m2-localhostrun-funnel-sidecar. The worker-side polling loop that consumes the URL ships in m2-worker-url-refresh-on-rotation: opt in by setting WHILLY_FUNNEL_URL_SOURCE=postgres or =file on the worker. See docs/Distributed-Setup.md § “Two-host via localhost.run” for the static-URL fallback recipe operators can also use.

Adjacent runbooks

Two operational concerns are split into their own runbooks so you can find them by name:

  • docs/Cert-Renewal.md — TLS-cert lifecycle for the localhost.run wildcard *.lhr.rocks cert (what to check when workers start failing handshake), and a forward-looking section on the BYO-cert path you’ll want when you outgrow localhost.run.
  • docs/Token-Rotation.md — separate playbooks for per-user bootstrap leaks (one operator’s plaintext got pasted in Slack — limited blast radius) and admin / shared legacy-token leaks (the cluster-wide secret got out — full rotation required).

Both are linked from the README quick-start and from the in-tree CHANGELOG.md v4.5 entry so operators can grep them by name.

Reference: env vars added in v4.5

Variable Default Purpose
WHILLY_FUNNEL_URL_SOURCE static Worker URL discovery mode: static / postgres / file. See Worker-side URL re-discovery.
WHILLY_FUNNEL_URL_FILE /funnel/url.txt Path to the shared-volume file in file mode.
WHILLY_FUNNEL_URL_POLL_SECONDS 30 (postgres) / 5 (file) Poll cadence on the chosen source.
FUNNEL_LOCAL_HOST control-plane Sidecar — host the SSH reverse tunnel forwards to.
FUNNEL_LOCAL_PORT 8000 Sidecar — port the SSH reverse tunnel forwards to.
FUNNEL_SERVER_ALIVE_INTERVAL 60 Sidecar — SSH ServerAliveInterval seconds.
FUNNEL_RETRY_BACKOFF_SECONDS 5 Sidecar — sleep between SSH reconnect attempts.
FUNNEL_URL_FILE /funnel/url.txt Sidecar — file the parsed URL is atomically rewritten to.

WHILLY_WORKER_BOOTSTRAP_TOKEN (the legacy shared secret) is still honoured in v4.5 as a one-minor-version backwards-compat fallback on POST /workers/register. Setting it produces a one-shot deprecation warning at startup; the per-operator bootstrap flow (whilly admin bootstrap mint) is the supported path.