Host specs — hosts/<name>.yml
A host spec says where a factory runs: which accounts, which tokens (by
name), where the Pulumi state lives. It is owned by the host owner, who is
not necessarily the repo that runs on it — clankerlabs owns
hosts/dev-host.yml; clankerengineer merely points at it.
It is non-secret by construction. Every credential is a secret:NAME
reference into the secrets provider the spec names; no value ever appears in
the file. That is what lets a host spec sit in a public repo, travel through
repo: and stack: references, and be pasted into a support ticket. It is
also what makes the rule enforceable: no committed PLAINTEXT credential,
anywhere — the secure: ciphertext a pulumi config set --secret writes
into the stack file is allowed, a pasted token is not (plan D24;
tests/no-committed-blobs.test.ts).
- Example:
examples/hosts/dev-host.yml(the clankerlabs host, built fresh under the legacy names, with the productiongateway:block). - Schema source:
packages/engine/src/schema/host.ts. - What you need to own before you can write one: byoc.md.
How a manifest finds a host
host.ref in factory.yml:
| Form | Use when |
|---|---|
./hosts/<name>.yml | the caller repo carries the spec (you own the host, or vendored the owner’s copy) |
repo:<owner>/<repo>@<ref>:<path>.yml | the owner keeps it in their repo; pin @<ref> to a tag. Private repo → checkout_token_from (read into FACTORY_GITHUB_TOKEN) |
stack:<org>/<project>/<stack> | you only know the host exists; the engine reads the stack’s non-secret hostSpec output. Container verbs then read nothing else from the stack — the secrets they need come from the provider the spec names, never from stack outputs |
Keys
id, namePrefix
| Key | Type | Default | Meaning |
|---|---|---|---|
id | slug | — | Names the Pulumi stack (<org>/<project>/<id>), the DNS ownership tag managed-by=factory/<id>, and the Tailscale hostname of the host — clankerlabs’ host joins the tailnet as dev-host |
namePrefix | slug | — | Prefix of every cloud resource name: {namePrefix}-key, {namePrefix}-firewall, and {namePrefix} itself for the server and the tunnel |
A prefix carries names, never state. Every host is built fresh by
factory host up; there is no state import and no frozen logical name
(plan D23) — a pre-factory stack is torn down, never imported. clankerlabs keeps namePrefix: clanker-dev-host so
the fresh server, key, firewall and tunnel answer under the names every
runbook and URL already uses. The target names are plan D24’s
clanker-<thing>-development family (namePrefix: clanker-development,
bucket clanker-storage-development, Redis clanker-redis-development,
host id development): a one-line edit of the spec plus the clankerengineer
rename in T081, once the legacy dev-host is gone.
A stack this program did not build is refused. Before any verb runs the
program, the engine pulumi stack exports the stack and stops if it holds a
resource outside the logical names layout.ts declares for the spec
(assertStackOwned, packages/engine/src/host/engine.ts). The stack and
provider resources are ignored; an empty stack passes. This exists because
whitehatgg/clanker-infra/dev-host IS the live legacy clankerengineer stack
until plan T023 step 3 empties it — a plan host against that checkpoint
would preview replacing the running dev-host, and a host down would
destroy it.
compute
| Key | Type | Default | Meaning |
|---|---|---|---|
provider | hetzner | existing-host | — | Create the VM, or converge one you already run |
hetzner.serverType | slug | cpx32 | |
hetzner.location | slug | hel1 | fsn1 is refused (disabled for the account) |
hetzner.image | string | ubuntu-24.04 | The converge script targets Ubuntu 24.04 |
hetzner.tokenFrom | secret:NAME | secret:hetznerToken | |
existingHost.address | string | — | Required for existing-host |
existingHost.sshUser | string | root | |
sshKeyFrom.privateKey | secret:NAME | secret:hostSshPrivateKey | The host root key lives in the secrets provider, never as a stack output (plan D19). Minted by host up on the runner when both names are unset; half a pair is refused |
sshKeyFrom.publicKey | secret:NAME | secret:hostSshPublicKey | For existing-host the pair is yours to set (the engine mints only for hosts it creates) |
firewall.ssh | boolean | — | Open TCP/22 publicly. Must be explicitly true for access.provider: ssh-direct |
firewall.mosh | boolean | true | Open UDP/60000–61000 |
ingress
| Key | Type | Default | Meaning |
|---|---|---|---|
provider | cloudflare-tunnel | none | — | A tunnel to Traefik :80 with TLS at the edge, or no public ingress |
accountId | string | — | Cloudflare account id (non-secret). Required for the tunnel; {host:accountId} in manifests |
tokenFrom | secret:NAME | secret:cloudflareApiKey | Tunnel permissions |
ingress: none is legal (a Tailscale-only host); a manifest with public:
ports on it validates with a warning.
dns
| Key | Type | Default | Meaning |
|---|---|---|---|
provider | cloudflare | cloudflare | The only v1 provider |
zones | zone[] (≥1) | — | Zones the factory may write into. hostnames.zone must be one of them; the first is the default |
tokenFrom | secret:NAME | secret:cloudflareApiKey | Zone:DNS edit on the listed zones |
Only records carrying comment: managed-by=factory/<id> are ever created,
updated or deleted — see dns-ownership.md (T031).
access
| Key | Type | Default | Meaning |
|---|---|---|---|
provider | tailscale | ssh-direct | — | How developers reach containers |
tailscale.tag | tag:… | — | ACL tag every minted key carries; required for tailscale |
tailscale.oauthFrom.clientId | secret:NAME | secret:tailscaleOAuthClientId | Pre-auth keys are minted on the runner from this, never delivered as the OAuth client in cloud-init |
tailscale.oauthFrom.clientSecret | secret:NAME | secret:tailscaleOAuthClientSecret |
secrets
| Key | Type | Meaning |
|---|---|---|
provider | pulumi-config | pulumi-esc | factory-secrets | Where every secret:NAME resolves |
pulumiEsc.environment | project/environment | Required for pulumi-esc. CI needs only PULUMI_ACCESS_TOKEN to read it |
pulumi-config— the reference deployment (plan D24):pulumi config set --secret NAMEvalues on the stack, the ciphertext committed inpulumi/Pulumi.<id>.yamlexactly as clankerengineer’s stacks of the same project carry theirs. Onpulumi-cloudthe stack’s own key decrypts andPULUMI_ACCESS_TOKENis the only credential CI needs; on ans3backend the passphrase/KMS key does. The engine reads the whole set withpulumi config --show-secrets --json --cwd pulumi(the CLI needs the project directory) and strips the<project>:namespace, so the spec names stay bare. Seeding: operator/seeding.md.pulumi-esc— a BYOC option: an ESC environment referenced from a blob-free stack file. Needsstate.provider: pulumi-cloud(refused otherwise).factory-secrets— a JSONname → valuemap handed to the reusable workflow as ONE secret (FACTORY_SECRETS). Aworkflow_callhas no open-ended secrets map, so this is how a caller passes many names.
state
| Key | Type | Default | Meaning |
|---|---|---|---|
provider | pulumi-cloud | s3 | — | |
pulumiCloud.org | string | — | The Pulumi org — spelled here, never in code (plan D3) |
pulumiCloud.project | string | clanker-infra | Must equal pulumi/Pulumi.yaml’s name: (the directory the engine opens). clankerlabs reuses clankerengineer’s clanker-infra project (plan D24) |
s3.url | s3://bucket[?endpoint=…] | — | R2 works via ?endpoint= |
s3.secretsProvider | passphrase | awskms://… | — | passphrase reads FACTORY_PASSPHRASE at run time |
State necessarily holds host secrets (the SSH key, the tunnel token). The
control is who holds the backend and the decryption key, not “no secrets
in state” — with pulumi-cloud the owner accepts Pulumi Inc. as a key
holder (byoc.md).
image.pull.tokenFrom
GHCR read:packages token. Needed while CLANKERNET’s packages are private
(plan T051) — a v1 limitation recorded in byoc.md.
gateway (optional)
The GFP/1 gateway as a host-stack service (plan T041, D22). factory host converge deploys host/docker-compose.gateway.yml + host/traefik.gateway.yml
(on-host dynamic/gateway.yml) only when this block is present; a host
without it runs no gateway, and removing the block stops the service on the
next converge. For clankerlabs this service IS production: api.clanker.net
on dev-host (docs/operator/gateway.md).
| Key | Type | Default | Meaning |
|---|---|---|---|
host | hostname | — | What the gateway answers. Must be under one of dns.zones; the CNAME to the tunnel is { protect: true } |
publicOrigin | scheme://host | — | Required. The APP origin the OAuth callback is computed from (${publicOrigin}/auth/github/callback is the one URL on the OAuth app). NOT derived from host: clankerlabs answers api.clanker.net but logs in on https://app.clanker.net, whose Pages worker proxies /auth/* here |
coordinationRepo | owner/name | — | The GFP/1 log it serves → COORDINATION_REPO_URL |
org | login | — | GitHub org whose members may log in → GITFWD_ORG |
imageTag | tag | 1 | ghcr.io/clankerlabs/factory-gateway:<tag>. Rollback = converge with the previous tag |
oauthFrom.clientId | secret:NAME | secret:gatewayGithubClientId | A dedicated OAuth app per gateway — an OAuth app has one callback URL (plan D18) |
oauthFrom.clientSecret | secret:NAME | secret:gatewayGithubClientSecret | |
gitTokenFrom | secret:NAME | secret:GIT_TOKEN | Reads + pushes refs/heads/gitfwd/** on the coordination repo (plan D11) |
bridgeSecretFrom | secret:NAME | — | Optional stable OAuth-state key; unset, a restart turns an in-flight login into a retry |
How the secrets travel. The four secret values are resolved from the
host’s secrets provider on the runner and delivered inside the converge
script over SSH stdin into a root-owned 0600 env file
(/opt/factory/gateway.env) that docker compose --env-file interpolates
from. They are never written into the compose file, never in cloud-init and
never a stack output — packages/engine/test/gateway.test.ts pins each
negative. The GHCR pull token (image.pull.tokenFrom) is used inside a
throwaway DOCKER_CONFIG for the pull and removed; nothing persists it on
the host.
resources (optional)
Shared backends the host PROVIDES to its containers (plan T021/D4). A
manifest names one by name and maps its exports to env vars
(factory-yml.md §resources); it cannot create one — a bucket only one
caller repo’s file knew about would be planned for deletion the first time
the owner ran host up without it.
| Key | Type | Default | Meaning |
|---|---|---|---|
r2[].name | slug | — | The bucket name, exactly as manifests reference it. Created on the host stack { protect: true } |
r2[].location | WNAM | ENAM | WEUR | EEUR | APAC | OC | WNAM | Immutable after create |
r2[].credentialsFrom.{accessKeyId,secretAccessKey} | secret:NAME | secret:R2_ACCESS_KEY_ID / secret:R2_SECRET_ACCESS_KEY | The S3 token pair (minted once in the dashboard; not a Pulumi resource). Read by containers and by resources destroy |
upstash[].name | slug | — | The logical name manifests reference. Then EXACTLY ONE of the two forms below |
upstash[].existing.restUrlFrom | secret:NAME | https://… | — | Existing form — a database you already run. The REST endpoint, by name or in the clear (it is an address). The host stack declares NO resource; containers get the pair straight from the two references; resources destroy refuses it (“not ours to destroy”) |
upstash[].existing.restTokenFrom | secret:NAME | — | The REST token — always a name |
upstash[].databaseName | slug | {namePrefix}-{name} | Create form — the host program creates a global database { protect: true }. Immutable after create |
upstash[].primaryRegion | string | us-east-1 | |
upstash[].credentialsFrom.{email,apiKey} | secret:NAME | secret:upstashEmail / secret:upstashApiKey | The Upstash management credential (console → Account → Management API) that creates the database and that containers use to read its REST pair back by name. factory validate warns on every create-form entry: whether the provider holds the key is not knowable offline |
An entry that mixes the two forms (existing beside databaseName,
primaryRegion or credentialsFrom) is refused. clankerlabs’ dev-host
uses the existing form — the one database staging and production already
share, seeded as REDIS_URL / REDIS_TOKEN from staging’s own rows
(operator/seeding.md) — so no Upstash management key
exists for it anywhere. The create form stays for adopters who own a
management key and want the factory to own the database.
Refusals
| Condition | Why |
|---|---|
access.provider: ssh-direct without compute.firewall.ssh: true | Opening TCP/22 to the world must be a line the owner wrote |
secrets.provider: pulumi-esc with state.provider: s3 | ESC is a Pulumi Cloud feature; nothing for a self-managed backend to reference |
ingress: cloudflare-tunnel without accountId | The tunnel is created in an account |
compute.hetzner.location: fsn1 | Disabled for the account |
existing-host with a hetzner block, or tailscale without its block | A spec that thinks it is two things |
gateway.host outside dns.zones | A record that could never be created |
gateway: without publicOrigin | The OAuth callback cannot be derived from host (T114 erratum) |
resources.upstash[] mixing existing with the create-form keys | One entry is one database, either yours or the factory’s |
a literal token anywhere a secret:NAME is expected | That is a committed secret |
| any unknown key | Objects are closed; a typo is an error, not an ignored line |
Which secrets a provision needs
Derived from the two specs, with a reason per name — factory plan prints
the list, and the workflow preflight (T050) checks it before touching a
cloud. A Hetzner + tunnel + Tailscale host needs: hetznerToken,
hostSshPrivateKey, hostSshPublicKey (minted by the first host up),
cloudflareApiKey, tailscaleOAuthClientId, tailscaleOAuthClientSecret;
plus image.pull, the gateway’s four names when gateway: is present, the
manifest’s env.git.tokenFrom, GITHUB_ORG_READ_TOKEN when
access.github.org is set, and every name in env.secrets. An
existing-host + ssh-direct host needs neither the Hetzner nor the
Tailscale credentials.
The host engine (T021)
factory host up|down|converge, factory plan host <id>, factory resources destroy, factory secrets set — packages/engine/src/host/engine.ts
over the inline Pulumi program packages/engine/src/program.ts, opened with
the Automation API from pulumi/ (project clanker-infra, one stack per
host). Every verb that opens the stack exports it first and refuses a
checkpoint this program did not build (above).
| Verb | What happens |
|---|---|
plan host <id> | The keypair check below (the preview needs the public half), then resolve every needed secret through the spec’s provider → open the stack → preview → the DNS deletion gate (host/dns-gate.ts, fails closed; ALLOW_DNS_DELETION="<reason>" / ALLOW_EMPTY_PREVIEW="<reason>" are the only overrides). No apply. |
host up --host <id> [--sysbox] | First, the root keypair: if neither sshKeyFrom name is set in the secrets provider, ssh-keygen -t ed25519 runs on the runner and both halves are stored there (half a pair is refused; existing-host never mints). Then plan + up, then the runner-side half: fetch the tunnel token from the Cloudflare API, probe tailscale status over SSH, mint a tagged single-use pre-auth key on the runner only if the host has not joined, run the idempotent converge over SSH stdin, prune stale <device> / <device>-N tailnet records (exact or -N, never a prefix). |
host converge --host <id> [--sysbox] | The runner-side half only, from existing outputs: Traefik, cloudflared, the Tailscale join, and — when the spec has gateway: — the gateway service with its 0600 env file. Re-run freely. |
host down --host <id> [--dry-run] | Refuses while the host provides any resources.r2[] or create-form resources.upstash[] (they are { protect: true }, plan D4). An existing database is not a stack resource and does not block it. Nothing else is protected: the box is the factory’s to rebuild. |
resources destroy --host <id> --name <n> | The ONLY path that deletes a shared backend: (1) up with protect off, (2) empty the bucket on the runner (providers/resources/empty-r2.ts), (3) up with the resource left out. Then remove it from the spec. Refuses an existing database — not ours to destroy; remove the entry from the spec instead. |
secrets set <NAME> --host <id> | Value on stdin. pulumi-config → pulumi config set --stack <org>/<project>/<id> --secret <NAME> --cwd pulumi (commit the changed row); pulumi-esc → pulumi env set … pulumiConfig.<NAME> - --secret; factory-secrets has no write path. |
Stack outputs are exactly serverIp, tunnelId, tailscaleHostname,
hostSpec — none secret (packages/engine/test/no-secret-outputs.test.ts).
The SSH key lives in the secrets provider, the tunnel token is fetched on the
runner, and the Tailscale OAuth client never leaves it; nothing secret is in
cloud-init (host/cloud-init.ts is a golden, secret-free string — userData
is ForceNew, so it is written to never change).
The engine never writes plaintext into the stack file. Secrets reach the
default cloud providers as the workspace’s process environment
(HCLOUD_TOKEN, CLOUDFLARE_API_TOKEN, and UPSTASH_EMAIL/UPSTASH_API_KEY
only when the spec CREATES a Redis),
which is also why no explicit Provider resource exists — it would be a
second place the credential lives. The stack file holds ciphertext only
(pulumi-config) and the one-time seed is
operator/seeding.md.
Legacy names, never legacy state (plan D23). The logical names are fixed
(host-ssh-key, host-firewall, host, host-tunnel, host-tunnel-config,
host-tunnel-secret, host-gateway-dns, r2-<name>, upstash-<name>) and
the cloud names are {namePrefix}-…; { protect: true } sits on the
data-bearing backends and the gateway record only, and nothing carries
ignoreChanges. packages/engine/test/host-program.test.ts pins that
hosts/dev-host.yml plans server, key, firewall, tunnel and bucket under
clanker-dev-host / clanker-storage-dev-host with no state-import
machinery — and NO Redis resource: clanker-redis is the existing form
(the database staging and production already share), so the stack holds
eight resources, not nine.