Environments

A repository describes the infrastructure it needs in one file, replicator.config.ts, and a person or a workflow runs it with one command per stage. The same file builds staging and production, so the two never drift apart.

import { replicator } from "@clankernet/replicator";

export default replicator.app({
  name: "acme",
  stages: { staging: {}, production: {} },
  run(ctx) {
    const host = replicator.hostnames("acme.example", ctx.stage, {
      labels: { staging: "next" },
    });
    new replicator.StaticSite("public", {
      accountId: ctx.secret("accountId"),
      project: ctx.stage === "production" ? "acme-public" : "acme-public-next",
      productionBranch: "main",
      domains: [host()],
    });
    new replicator.Bucket("uploads", {
      accountId: ctx.secret("accountId"),
      bucket: `acme-uploads-${ctx.stage}`,
    });
    return { site: host() };
  },
});

Every option of the file is listed, with its default, in the configuration reference; complete files are in Examples.

The stages

stages names every environment. production protects its data by default: buckets, databases and sites are never deleted or replaced by a deploy, and the stage cannot be removed without saying so. Any other stage is disposable. A stage can set protect itself.

run(ctx) builds one stage. It receives:

• ctx.stage and ctx.app, to name things;
• ctx.protect, whether this stage’s data is protected;
• ctx.preview, whether this stage is a preview (one per pull request);
• ctx.secret(name), a value you stored for this stage, which fails by name when it is missing;
• ctx.optionalSecret(name), the same, or nothing;
• ctx.output(app, stage, key), a plain value another app published (see Reading another app’s outputs).

What run returns are the stage’s outputs: names and addresses. An output that holds a stored secret is refused. A stored value that is only a setting, such as an account id, a project name or a hostname, can be named in plain: ["accountId"] on the app; the output check skips just those names, so addresses built from them may be returned.

Previews

A pull request can have an environment of its own. Declare the pattern once, next to stages:

export default replicator.app({
  name: "acme",
  stages: { staging: {}, production: {} },
  preview: { pattern: "pr-{number}", ttl: "7d" },
  run(ctx) {
    if (ctx.preview) {
      // smaller, cheaper, disposable
    }
    // …
  },
});

Any stage name the pattern makes (pr-12) is a preview. It is not listed in stages; there is one per pull request. A preview:

• is never protected, and holds nothing that outlives it: every resource in it is removed with it, whatever a component says about keeping its data, so a bucket or a database made for a pull request goes with the pull request;
• cannot be named like production or like any protected stage, and no listed stage may match the pattern. The check names its rule (preview-protected, preview-collision, preview-pattern, preview-ttl) and stops the run before anything is planned;
• is removed with replicator infra remove --stage pr-12, with no typed confirmation. Any other stage needs the confirmation in the workflow, and a protected one needs more.

ttl (hours or days, at most 30d, default 7d) states how long a preview is meant to live. Closing the pull request removes it; the lifetime is what removes a preview whose pull request closed without anything noticing. replicator infra stage --config <file> --stage <name> prints what a name is (a listed stage or a preview, and whether it is protected) without opening any state.

The release pipeline deploys a preview when a pull request opens or changes and removes it when it closes; which pull requests start it is your workflow’s choice.

Preview expiry

replicator infra previews --config replicator.config.ts            # list, with each age against the ttl
replicator infra previews --config replicator.config.ts --expired  # only the ones past it
replicator infra previews --config replicator.config.ts --remove --yes

It lists the app’s stages in the state bucket and keeps only those the preview pattern makes. Each shows how long ago it was last written (its state or its published outputs, whichever is newer) against the ttl; one past it is EXPIRED. --remove --yes removes the expired ones, one at a time, through the same path as replicator infra remove: the preview’s resources, then its published outputs. --remove without --yes removes nothing. A stage the pattern does not make (a listed stage, a protected one, a name nobody declared) is never listed as removable and is never touched; it is only named as left alone. One stage failing does not stop the others, and the exit status is 1 when any failed. --json prints the report. The token and the state account are the same as for any infra verb.

To run it on a schedule, call the reusable workflow from a cron. remove defaults to false, so the first runs only list:

on:
  schedule:
    - cron: "17 4 * * *"
jobs:
  previews:
    uses: clankerlabs/CLANKERNET/.github/workflows/replicator-previews.yml@<40-character SHA>
    with:
      replicator_ref: <40-character SHA>
      remove: "true"
    secrets:
      REPLICATOR_STATE_TOKEN: ${{ secrets.REPLICATOR_STATE_TOKEN }}

A complete caller is in examples/workflows/replicator-previews.yml.

Presets

A convention an organisation states once should read the same in every repository that follows it. replicator.presets are plain functions you call at the top of run(ctx); each registers a rule over every resource declared after it (a component’s own resources included) or builds components under the convention:

replicator.transform(type, edit) is the primitive underneath: edit({ type, name, props, opts }) returns the props or opts to use instead, or nothing to leave the resource alone. A rule lives for one stage’s run and nothing carries over to another. Two repositories calling one preset differ only in their own names, which is what a preset is for. Shared YAML defaults (extends) are for the fleet file and the dev-container block, never for infrastructure.

Hostnames

replicator.hostnames(zone, stage, rules) gives each stage its own names. The production stage owns the apex and plain subdomains (api.acme.example); every other stage is flat-dash, so one certificate and one record per name is enough (api-next.acme.example). rules.labels renames a stage’s label.

Components

Server

const origin = new replicator.Server("origin", {
  sshPublicKey: ctx.secret("originPublicKey"),
  cloudflareRanges: edgeRanges, // the edge network's published IPv4 and IPv6 ranges
  serverType: "cx22",
  location: "hel1",
  userData: bootScript,
  protect: ctx.protect,
});
return { ip: origin.ip };

One machine and the firewall in front of it. The firewall lets in ping, and web traffic (ports 80 and 443) only from the address ranges you pass as cloudflareRanges, so the machine cannot be reached around the edge. Nothing else is open, port 22 included: reach the machine over a private network, or set ssh: true to open 22 to everyone. An empty list of ranges is an error, because it would close the machine to everyone.

The image defaults to ubuntu-24.04. userData is the first-boot script; changing it replaces the machine, so a stage that must keep its server leaves it alone. The machine is protected unless you pass protect: false; pass ctx.protect to follow the stage. Its names derive from the component’s name (origin-ssh-key, origin-firewall, origin-server), so renaming the component replaces them. Give the login key as the public half only; the private key stays with whoever runs the deploy.

Hyperdrive

new replicator.Hyperdrive("db", {
  accountId: ctx.secret("accountId"),
  origin: {
    host: "db.acme.example",
    database: "app",
    user: "app",
    password: dbPassword.value,
  },
  protect: ctx.protect,
});

A pooled, nearby connection from edge workers to a relational database. Query caching is off unless you pass caching: { disabled: false, maxAge: 60 }. The password may be a plain value or the output of another component, such as a GeneratedSecret; it is stored as a secret and is never an output. The component’s outputs are id, which a worker’s binding uses, and name.

SecretsStore

const kms = new replicator.SecretsStore("kms", {
  accountId: ctx.secret("accountId"),
  secretName: "KMS_MASTER_KEY",
  value: ctx.secret("kmsMasterKey"),
  protect: ctx.protect,
});
return { kmsStore: kms.storeId };

A store holding one secret: the key that wraps your other secrets, readable by edge workers (scopes defaults to ["workers"]). The value may be a plain value or another component’s secret output, is stored as a secret, and is never an output; storeId and secretName are. Both the store and the key are protected unless you pass protect: false, because losing the key loses everything it wraps.

An account may hold only one store. When another stack already owns it, pass existingStoreId and this component creates only the key inside that store, and can never delete the store itself.

Worker

const api = new replicator.Worker("api", {
  accountId: ctx.secret("accountId"),
  scriptName: "acme-api",
  module: "dist/api.mjs", // your build's output; the component uploads it as it is
  compatibilityDate: "2025-01-01",
  compatibilityFlags: ["nodejs_compat"],
  bindings: {
    vars: { APP_ENV: ctx.stage },
    secrets: { SESSION_KEY: ctx.secret("sessionKey") },
    secretsStore: {
      KMS: { storeId: kms.storeId, secretName: "KMS_MASTER_KEY" },
    },
    r2: { UPLOADS: uploads.name },
    hyperdrive: { DB: db.id },
    kv: { CACHE: "namespace-id" },
    services: { BILLING: "acme-billing" },
    durableObjects: { ROOMS: "Room" },
  },
  durableObjects: {
    adoptedTag: "v2", // the tag the live script already reports; omit on a first deploy
    migrations: [
      { tag: "v1", newSqliteClasses: ["Room"] },
      { tag: "v2", newClasses: ["Lobby"] },
      { tag: "v3", renamedClasses: [{ from: "Lobby", to: "Hall" }] },
    ],
  },
  routes: [{ zoneId, pattern: "api.acme.example/*" }],
  domains: [{ zoneId, hostname: "api.acme.example" }],
});
return { api: api.scriptName };

An edge worker. You build the script yourself (any bundler); module is the path of the finished file, which must exist when replicator infra runs, and it is uploaded unchanged: a rebuilt file with different bytes is the only thing that redeploys the code. scriptName is the name on your account; never change it after the first deploy.

Bindings are grouped by kind, each as binding name: what it points at. vars are plain text; secrets take a stored value (ctx.secret(...)) and are sent as secrets, never shown in a plan and never an output. secretsStore binds a secret that lives in a store (see SecretsStore) so the worker reads it without the value ever passing through your stack. r2, hyperdrive, kv and services bind a bucket, a pooled database connection, a key-value namespace and another worker. durableObjects binds a class this worker exports (it must appear in the migration list), or { className, scriptName } for another worker’s. A name used twice is an error.

Durable objects keep their data in the account, so their history is a list you maintain. Each migration has a tag and says what it did (newClasses, newSqliteClasses, deletedClasses, renamedClasses); add one to the end whenever the exported classes change, and never edit or reorder the ones already there. On a first deploy every step runs. For a worker that already exists, set adoptedTag to the tag it reports now: only the steps after it run, and an adoptedTag that is not in your list is an error naming the list. When the two agree, no migration is sent at all.

A worker with durable objects is protected unless you pass protect: false, and a deploy that would delete or replace it is refused on every stage unless you name it (the same decision as for a bucket). A worker without them is an ordinary resource. Routes and domains point at the script; give domains for a hostname the worker owns outright and routes for a path pattern on an existing zone.

Database

const db = new replicator.Postgres("db", {
  apiKey: ctx.secret("databaseApiKey"),
  project: `acme-${ctx.stage}`,
  region: "aws-eu-central-1",
});

One project with its first database and role, protected by default. db.host, db.pooledHost, db.database and db.user are plain; db.password and db.uri are secret and cannot be returned from run (an output that holds a stored secret is refused). The role’s password exists only when the project is created, so a stage that needs a database is created, not adopted: recreate it rather than import it. Region, version, database and role cannot change in place; sizes and the name can.

Handing the application its environment

A stage often has values only the stack knows (a database address and password, a generated key). Return them under the reserved key __env, a flat record of environment names to strings:

return {
  site: host(),
  __env: { DATABASE_URL: db.uri, SESSION_SECRET: session.value },
};

replicator infra deploy --env-out .env writes them to that file (mode 0600) after registering every value so CI logs hide it. --env-format docker writes the lines verbatim for docker run --env-file, which does not unquote. They are never printed and never part of the stage’s plain outputs: the program writes them to a private file while it runs, and the command fails if it wrote none. A value holding a newline is refused: store it base64-encoded.

Reading another app’s outputs

A repository sometimes needs an address or an id that another repository’s stage creates: the API’s public URL, a project id. Instead of copying it into your own file, declare the edge and read it:

export default replicator.app({
  name: "web",
  stages: { staging: {}, production: {} },
  dependsOn: [{ app: "api", stage: "production", key: "publicUrl" }],
  run(ctx) {
    const api = ctx.output("api", "production", "publicUrl");
    return { api };
  },
});

Every deploy publishes the stage’s plain outputs (the object run returned, without the reserved __env) next to its state, and ctx.output(app, stage, key) reads one of them. The rules:

• Declared up front. Each edge is a dependsOn entry, so the engine reads the values before the stage runs, and who depends on whom can be read from the files alone. ctx.output for an edge you did not declare fails and shows the line to add. An app cannot depend on its own outputs.
• Read-only, plain values. You get a string, a number or a boolean (handed over as a string). An output that holds a value the other app stored as a secret is refused when it is published, and so is an output named like a stored secret: a secret is never readable from another app.
• The same state account only. One account is one trust domain (see the account is the trust boundary). An edge that names another account with accountId, or an object written into another account, is refused with the rule’s name, same-state-account.
• A missing value fails the run, by name. If the other app has not deployed that stage, or does not return that key, the run stops before anything is built and says which app reads which key of which app and stage, and what to deploy first.
• Removal cleans up. Removing a stage deletes what it published. A removal still goes ahead when an edge cannot be read, because the other app may be gone.

Deploy the app that publishes before the app that reads it. A diff of the reading app fails the same way until the other app has deployed once.

Running it

replicator infra secrets set accountId --config replicator.config.ts --stage staging
replicator infra diff   --config replicator.config.ts --stage staging
replicator infra deploy --config replicator.config.ts --stage staging

secrets set reads the value on standard input. Add --if-absent to leave a value that is already stored alone, so a script that fills in a missing value can run again safely.

A value every stage should start from (a shared account id, one provider key) is stored once with --all-stages:

replicator infra secrets set sharedKey --all-stages --config replicator.config.ts --stage staging

Every stage of the app reads it until that stage stores a value of its own with the same name; the stage’s value always wins. --if-absent with --all-stages looks only at the shared value. --stage still names the stage the command opens; an all-stages value is not specific to it.

A later step of a workflow sometimes needs one stored value for another tool, or the stage’s addresses. replicator infra secrets get <NAME> --out <file> writes that one value to a file (mode 0600, masked in CI logs, never printed), and replicator infra deploy --outputs-out <file> writes the stage’s plain outputs as JSON; a deploy with nothing to change is a safe way to read them.

Declaring the secrets

A repository that reads secrets lists them in the secrets block of replicator.app, so a missing one is found before anything is planned and not half way through a deploy:

secrets: {
  sessionKey: { generate: "random-hex-32" },
  zoneId: { description: "The DNS zone's id" },
},

replicator infra secrets check  --config replicator.config.ts --stage staging
replicator infra secrets ensure --config replicator.config.ts --stage staging --values-file secrets.env

check prints the required names the stage lacks (names only, never a value) and exits 1 if there are any. ensure stores what it can: a value from the values file (NAME=value lines, a file only its owner can read), else one generated on the runner when the declaration says generate. A name the stage already holds is kept, so it is safe to run again. diff and deploy run the same check first and stop with every missing name and the exact replicator infra secrets set NAME … line for each. Every key of the block is in the configuration reference; replicator doctor warns when the block and what the environments read disagree.

diff prints every step and any policy violation, and never applies. deploy applies only a diff that passes the policy:

• a data resource (bucket, database, cache, site) is never deleted or replaced unless you name it with --allow-destroy;
• a DNS record is never deleted or replaced silently;
• on a protected stage, nothing is deleted at all.

The host owner can add rules of their own in the policy: block of the fleet file (Fleet): the regions a resource may be created in, the labels a server must carry, kinds of resource that are never deleted on any stage, and how many stages a repository may declare. Pass the file with --policy clanker.yml; every refusal names its rule (region, label, protected-kind or stage-count). A protected kind is not lifted by --allow-destroy, because the person running the command is not the host owner.

replicator infra remove --stage <stage> tears a stage down; a protected stage needs --allow-destroy '*'.

Finding drift

A resource someone changed by hand is invisible to a diff, which compares the program with what the last deploy recorded. drift asks the cloud first:

replicator infra drift --config replicator.config.ts --stage production \
  --policy clanker.yml --summary-out drift.md

It refreshes the stage’s recorded state from what is actually there, then compares in two ways, and never applies anything to the cloud:

• changed outside: a resource whose recorded properties moved since the last run, or that is gone. Only the property names are printed, never a value;
• deploy would: every step a deploy would now take against the refreshed state. A change you committed but have not deployed shows here too, which is why it is a separate list.

The exit status is 0 when the stage matches its program, 2 when there is drift, and 1 when the run itself failed, so a scheduled job can tell a finding from a failure. --summary-out writes a short Markdown table for the job summary. Refreshing writes the cloud’s truth into the stage’s recorded state (that is what a refresh is); the second list keeps reporting the difference until a deploy puts it right or the program is changed to match. A deploy of the stage is the fix.

Drift on a schedule

replicator-drift.yml is the reusable workflow that runs it nightly. It takes the stages (stages: staging,production), checks each one even if an earlier one drifted, writes every stage’s table to the job summary and keeps them as the artifact replicator-drift. It fails the job when a stage drifted or a check could not run, so the run is red where you look; it opens and updates no issue, because what to do about drift is your repository’s decision. Its one secret is REPLICATOR_STATE_TOKEN. A complete caller is in examples/workflows/replicator-drift.yml.

Each kept summary is named <owner>__<repo>__<stage>.md and ends with the time it was checked. replicator fleet status --drift-dir <directory of those files> then shows, beside each repository of the fleet, the last result per stage (drift: production DRIFT, staging clean). It only reads the directory; how the files get there (downloaded artifacts, a results branch) is yours.

Running it from a workflow

A repository does not have to run these commands by hand. The control-plane workflow you already call for hosts and pods (see the quickstart) also has three actions for a repository’s own environments. Each one runs exactly one command against replicator.config.ts in your checkout, for one stage, with your own credentials. They never touch a host, a pod or a fleet file.

The inputs these actions read, all strings:

A value that does not belong to the chosen action is refused rather than ignored, so a stray allow_destroy on a diff cannot look like it did something.

The credential that opens the state is REPLICATOR_STATE_TOKEN, passed as a secret of that name. It reaches the command as an environment variable and never as an argument. A token that reads a private engine checkout can be passed as the optional FACTORY_GITHUB_TOKEN secret.

Runs for one repository and stage queue behind each other (infra:<repository>:<stage>) and are never cancelled half way. The run summary names the repository, the environment file, the stage and the action, and never a value.

The workflow does not hand you the application’s environment file: that file holds secrets, so it is never uploaded as a build artifact. If your own job needs it, run replicator infra deploy --env-out <file> there, in a job of yours that checks out the engine at the same pinned reference.

To release a whole repository (the stage, the static sites, your own commands and a smoke check) in one call, see the release pipeline.

A repository calling the workflow, with a diff on every pull request and a deploy you start by hand:

name: Environments
on:
  pull_request:
    paths: ["replicator.config.ts"]
  workflow_dispatch:
    inputs:
      stage:
        type: choice
        default: staging
        options: [staging, production]
      allow_destroy:
        description: "Resources this run may delete or replace, comma-separated"
        default: ""
permissions:
  contents: read
jobs:
  diff:
    if: github.event_name == 'pull_request'
    uses: clankerlabs/CLANKERNET/.github/workflows/replicator-deploy.yml@<40-character SHA> # v1
    permissions:
      contents: read
      packages: read
    with:
      action: infra-diff
      stage: staging
      replicator_ref: <40-character SHA> # v1
      state_account: 0123456789abcdef0123456789abcdef
    secrets:
      REPLICATOR_STATE_TOKEN: ${{ secrets.REPLICATOR_STATE_TOKEN }}
  deploy:
    if: github.event_name == 'workflow_dispatch'
    uses: clankerlabs/CLANKERNET/.github/workflows/replicator-deploy.yml@<40-character SHA> # v1
    permissions:
      contents: read
      packages: read
    with:
      action: infra-deploy
      stage: ${{ inputs.stage }}
      allow_destroy: ${{ inputs.allow_destroy }}
      replicator_ref: <40-character SHA> # v1
      state_account: 0123456789abcdef0123456789abcdef
    secrets:
      REPLICATOR_STATE_TOKEN: ${{ secrets.REPLICATOR_STATE_TOKEN }}

Pin the 40-character SHA on the uses: line and in replicator_ref, as the quickstart does. The only secret you must pass is REPLICATOR_STATE_TOKEN; add FACTORY_GITHUB_TOKEN: ${{ secrets.FACTORY_GITHUB_TOKEN }} when the engine or a private file has to be read with a token of yours, or name a stored GitHub App with github_app_from: <app>/<stage> to have the run mint a read-only token from it instead (quickstart, “If something the workflow reads is private”).

Where the state lives

Each stage keeps its state, and the secrets you store for it, in a bucket in your own cloud account. The runner needs one credential, REPLICATOR_STATE_TOKEN, to open it. Nothing leaves your account.

The account is the trust boundary

The bucket has one fixed name per account, and the token that opens it opens every app’s and every stage’s secrets in that account. So isolation between teams is by account, not by repository: give each trust domain (an organisation, a team that must not read another’s secrets) its own account and its own token. A repository says which account holds its state with state.accountId, and the runner says which account its token belongs to with REPLICATOR_STATE_ACCOUNT (the workflow sets it beside the token). When the two differ, every infra verb refuses before it opens anything, naming both accounts:

acme-api: state.accountId is 5ddc…, but this runner's token belongs to account 1111…. A state
account is a trust boundary: one token opens every secret in it.

A config with no state.accountId uses the runner’s account.

State tokens

The boundary is the account that holds the bucket, so a token is scoped to one account, and a repository’s workflows hold only that account’s token. There is no isolation between two apps inside one account: they share a bucket, and a token that opens it opens both. Keep apps that must not read each other’s secrets in different accounts.

Give the token the least it needs, and nothing account-wide:

• storage: read and write on the one state bucket, and no other bucket;
• whatever the stages’ own resources need, added deliberately and per resource kind (for example site hosting and DNS edit for a repository that publishes a site, edge-script edit for one that deploys workers), scoped to the account and, where the provider allows it, to the zone or project;
• never account-wide administrator, never a token that spans several accounts.

replicator infra token-check asks whether the token in the environment is that token:

REPLICATOR_STATE_ACCOUNT=<account id> replicator infra token-check \
  --config replicator.config.ts --stage staging

It checks three things and exits 1 if any fails: the config and REPLICATOR_STATE_ACCOUNT name the same account; the token opens that account and no other (a token that opens two is refused, and nothing is read with a token that does not open the declared one); and the token can read the state bucket for this app. If the token is not allowed to list accounts, that is a warning and the bucket read stands as the evidence: a token that reads this account’s bucket belongs to it. The report names account ids and never the token. It proves nothing about what else the token may do; the list above is how you keep that small.

A config can also say which engines it was written for, engine: ">=1.25.0 <2.0.0"; an engine outside the range refuses every infra command with the range named. Start a repository with replicator init, and check that its files agree with replicator doctor (Modular repositories).