Belay — your agent writes the code; you review ten sentences Belay — your agent writes the code; you review ten sentences

Your agent writes the code. You review ten sentences.

However fast an LLM writes code, whether that code still does what the spec says is a separate question. Belay, a black-box scenario harness for vibe coding, puts the system under test in src/ untouched, and builds executable specs in test/ that only ever knock from the outside, through interfaces a user can reach. A browser screen, a gRPC stream — every surface a person touches counts. A spec that existed only as prose becomes a contract that actually runs, every time. And because that contract is written as sentences rather than code, the artifact you have to read line by line is the spec, not the diff.

Gherkin .feature TypeScript step definitions HTTP · GraphQL · gRPC · WebSocket Browser control · capture · visual checks Mock infra · fault injection · clock control Language and framework agnostic

01 Getting started

Install once, run belay init once. After that it is just belay run.

Belay installs as a global CLI. The harness and the system under test never share a runtime, so it does not matter what src/ is written in. Node is what Belay runs on — it is not a constraint on your project.

Requirements

ItemNeededNotes
Node.js 22+requiredFor the Belay CLI and step definitions. Unrelated to src/, which can be any language
DockeroptionalFor containerised infra. Without it, stub and process mode still give you a full environment
Browser engineoptionalOnly if you pick the browser surface; init downloads just what you selected

Install

terminal
npm install -g @beoks/belay

belay --version
belay doctor        # check ports, Docker and engines up front

Initialise

The command is the same whether you start from an empty directory or a project that is already running. init looks around and proposes what it finds as the defaults.

terminal
mkdir my-project && cd my-project
belay init

  # Press enter to accept the value in brackets.
  # detected: npm --prefix src start

  Surfaces to verify (comma separated: http, graphql, ws, browser, cli)
  [http] > http, ws

  Command that starts src/
  [npm --prefix src start] >

  URL that proves it is ready
  [http://localhost:8080/health] >

  Add an external API stub? (in-process, no Docker)
  [y/N] > y

  Add a mail capture server? (SMTP)
  [y/N] > y

   src/                   # created if absent — where the system under test lives
   belay.config.ts        # prefilled with the surfaces and infra you chose
   test/support/world.ts
   test/support/steps/common.steps.ts
   test/features/L0-smoke/health.feature   # the first anchor
   test/features/L0-smoke/health.steps.ts
   AGENTS.md              # the rule that keeps agents out of test/
   .github/workflows/belay.yml
   .gitignore
   package.json           # test · test:smoke · test:watch scripts

  Next steps
    1. Put your project in src/ and check app.start in belay.config.ts
    2. belay run --only L0   # get the first anchor green
    3. belay new L1-<domain>/<feature>   # pin a spec as a scenario

What gets created

my-project/
├── belay.config.ts        # prefilled from the surfaces and infra you chose
├── AGENTS.md              # rules for agents, including "test/ is read-only"
├── src/                   # your existing code moves here wholesale
└── test/
    ├── support/           # world · hooks · drivers for the surfaces you picked
    ├── contracts/         # schema files, if you chose GraphQL or gRPC
    ├── baselines/         # created only if you chose the browser
    └── features/
        └── L0-smoke/
            └── health.feature

The first scenario it writes is deliberately trivial.

test/features/L0-smoke/health.feature
Feature: service boots

  The first anchor: proof that the harness itself is alive.
  Once green, the whole path works — infra up, build, readiness probe,
  scenario execution, and teardown.

  Scenario: it answers the health check
    When I GET "/health"
    Then the response succeeds
When that one passes, the harness is alive. Infrastructure came up, src/ built, the readiness probe passed, a scenario executed, and everything came back down. Every other scenario is stacked on top of that — it is checking the anchor is in before you weight the rope.

Commands

CommandWhat it does
belay initScaffold the config and the test/ skeleton. Safe to run over an existing project
belay runThe whole thing — infra up, scenarios, teardown
belay run --upto L1Cut the run at a layer for fast feedback (--only L2 also works)
belay run --grep signupFilter scenarios by name (--tags "@smoke" too)
belay run --watchRe-run on change. The environment stays up, so a scenario edit re-executes in milliseconds
belay run --bailStop at the first failure
belay upBoot the environment and hold it in the foreground, for poking by hand
belay downClean up an environment kept after a failure (child processes, containers)
belay new L2-order/checkoutCreate a .feature and its step file together
belay approve cart-emptyAccept a visual baseline, after showing you the diff
belay report --openOpen the latest report
belay doctorCheck ports, Docker, browser engines and orphaned containers

Language

Messages, CLI help and the generated scaffolding all follow BELAY_LANG (en, ko), falling back to your locale and then to English.

terminal
belay init                # English Gherkin, steps and AGENTS.md
BELAY_LANG=ko belay init  # the same, in Korean

The Gherkin parser reads en, ko and ja dialects regardless of that setting. A suite written in Korean runs fine for someone whose tooling speaks English, and the repository's own examples use both, so each dialect is exercised on every CI run.

Handing it to an agent

init writes an AGENTS.md whose rules are short. The point of all of them is one thing — keep the examiner and the candidate apart.

AGENTS.md
## Belay harness rules

- Change the implementation only inside `src/`.
- `test/` is read-only. Never edit a scenario to make it pass.
- Run `belay run` before reporting that you are done, and attach the output verbatim.
- If you believe a spec is wrong, do not change it — report it with your reasoning.

With that in place the loop is three lines. The agent edits src/, belay run decides, and the failure list is the todo list. It cannot rewrite the grading criteria, so the only thing that can converge is the implementation. That is what safe vibe coding means here.

terminal
belay run --only L0          # check the anchor
belay new L1-account/signup  # pin the spec as a scenario first
belay run                    # the failure list is the remaining work

02 Adopting a project you already have

The projects that most need a safety net are the ones already in flight.

Belay does not need a fresh start. Point it at the service you are already running, and it reads the code to fill in the parts you would otherwise have to describe from scratch.

terminal
belay init --from ../my-api --link symlink

   src → ../my-api

  Read from the project
    stack      node
    start      npm start --prefix src  (high: scripts.start = "node src/index.js")
    port       4321                    (high: PORT=4321)
    health     /healthz
    surfaces   http
    infra      db · cache              (from compose — declare what you need)

The port, the start command and the health path all come out of the code, so the first scenario knocks on the endpoint this service actually serves rather than a convention it may not follow. Every reading carries the evidence it came from — a proposal you can check beats one you have to trust.

How src/ connects

The four modes differ in who owns the code afterwards, and that decides whether CI can even check the suite out. That is not a choice to make on someone's behalf.

--linkWhat happensUse when
symlinkThe project stays where it is; edits apply immediatelyYou are still working on it locally
submodulePins a commit, so a run states which revision it passed againstIt is a shared repository, or CI must check it out
moveThe harness becomes the project repositoryYou want one repository
copyA snapshot, which will drift from the originalYou want a frozen reference

A symlinked src is added to .gitignore: it points outside the repository, and committing that helps nobody. A submodule is committed, and CI needs actions/checkout@v4 with submodules: true.

Turning existing code into scenarios

belay analyze reads the connected project and writes a checklist to .belay/analysis.md: the endpoints it found, grouped into the domains that become L1- directories, with a success candidate and a refusal candidate for each.

.belay/analysis.md
### `L1-orders`

| Surface | Operation | Where |
|---|---|---|
| http | `GET /api/orders`         | src/routes/orders.js |
| http | `POST /api/orders`        | src/routes/orders.js |
| http | `ANY /api/orders/:param`  | src/routes/orders.js |

Candidates:

- [ ] `POST /api/orders` — the success path: what is true afterwards that was not before?
- [ ] `POST /api/orders` — the refusal: which input is rejected, and with which error code?
The report says what it cannot see. Routes assembled at runtime, middleware behaviour, methods it could not read next to their path — and above all, what an endpoint is for. A scan can list POST /api/orders; it cannot tell you that placing an order must not charge the customer twice. That sentence has to come from you, and a tool that pretended otherwise would be handing you a suite that looks complete and checks nothing.

Skills for the agent doing the work

init writes the same skill to .claude/skills/belay/ and .agent/skills/belay/, so a coding agent loads the boundary before it starts: it may rewrite src/, and it may not edit test/ to make something pass. Belay's whole value rests on that separation, and an agent will not honour it from a README it was never asked to read.

The order matters. Run belay run --only L0 before writing any scenario. If the harness cannot boot your project yet, every later failure will be about that and not about your code.

03 Why Belay

The bottleneck in vibe coding is not how fast you produce code. It is the fear of regression — and the cost of reading what the agent wrote.

Hand a feature to an agent and the code appears. The problem starts immediately after. Is it safe to let it refactor? To switch frameworks? Did this commit quietly break the checkout flow written two months ago? With no way to answer, someone ends up clicking through it by hand every time — and the speed you gained disappears there.

Each of the usual answers leaks, for a different reason.

leaks

Specs that live only as documents

They never execute. When the implementation betrays them nothing says so, and given time the document rots first.

leaks

Unit tests bound to internals

Coupled to function signatures and class names. Change the structure and the tests break first — the very thing meant to enable refactoring blocks it.

leaks

Manual QA

Not reproducible, not recorded. If it cannot run on every commit it is not a safety net.

Belay

Executable specs, kept outside the implementation

Scenarios live outside src/. Replace everything inside and they remain, as the answer key.

Reading the code is the other half of the bottleneck

Vibe coding did not only make code cheaper to write; it made it more expensive to trust. An agent hands you a five-hundred-line diff in a minute, and approving it by inspection means reading all five hundred lines — again on the next commit, and the one after that. The artifact you have to review grows exactly as fast as the thing that made you fast.

Belay hands you a smaller artifact instead, written in a language you already read.

test/features/L2-cart/coupon.feature
  Scenario: a coupon that has been used cannot be used again
    Given the coupon "WELCOME" has already been redeemed
    When I apply it to my cart
    Then the request is refused
    And the cart total is unchanged

That is what goes under review, line by line. No framework, no class names, no control flow — only the behaviour you are agreeing to. Ten sentences pin down what a thousand lines of implementation are allowed to do, and they stay legible to the person who asked for the feature in the first place.

And these sentences are not a description of the code, they are a check on it. The steps knock only on surfaces a user can reach, so a green run means the sentences are true of the running system — not that something was mocked into agreeing with them. Review the spec; let the suite review the implementation.

As-is, a person reads the whole diff on every commit; to-be, the person reads ten sentences once and the suite reads the diff Left: the agent's 500-line diff lands in front of you and you read all of it, again on the next commit. Right: you read a ten-sentence scenario, written once, while belay run reads the implementation on every commit and answers 50/50. AS-IS — REVIEW THE IMPLEMENTATION src/ diff · +512 −348 ⋯ 500 more lines you read all of it — again on the next commit TO-BE — REVIEW THE SPEC Scenario: a used coupon is refused Given the coupon was already redeemed When I apply it to my cart Then the request is refused And the cart total is unchanged you read ten sentences — written once src/ diff +512 −348 belay run 50/50 the suite reads the implementation — every commit
Both sides receive the same diff, on every commit. What changes is who has to read it: the suite takes the five hundred lines, and the part left for a person is ten sentences — the ones that were worth reading in the first place.
What a green run guarantees is everything you wrote down, and nothing you did not. Belay cannot tell you that a behaviour is missing — that sentence has to come from you. What it does guarantee is that once written, it holds on every commit, against any implementation.
The goal is confidence. To be able to answer "can we throw this away and rewrite it in Go?" with "if the scenarios all pass, then yes" — that state is what Belay is for.

04 The idea

The climber moves freely. The belayer holds the rope.

src/ climbs, test/ belays, belay.config.ts is the anchor src/ on the left is the climber and is free to change language, framework and architecture. test/ on the right is the belayer and reaches src/ only through declared surfaces. belay.config.ts sits above both as the anchor the rope runs through. ANCHOR belay.config.ts CLIMBER src/ any language, any framework replaceable wholesale BELAYER test/ executable scenarios never imports src/ knocks only on declared surfaces
The rope runs from the belayer through the anchor to the climber — which is why the anchor is a file and not a convention. Move the anchor and both sides move with it; remove it and the rope holds nothing.

Climber = src/

Where the agent climbs. Language, framework and architecture are all free to change.

Belayer = test/

Holds the rope from outside. It does not know the internals and does not need to. It only catches the fall.

Anchor = the run contract

What to boot and how to knock on it, pinned in one file. Without it the rope holds nothing.

Three rules follow, and they are the whole of Belay.

  1. Black box. Tests reach the system only through interfaces the deployed artefact exposes. No importing source, no calling internals, no touching its database.
  2. Reproducible. Everything from booting infrastructure to cleaning up lives in one command. What works on your laptop has to be what works in CI.
  3. The spec is the test. A .feature file is both the specification a person reads and the check a machine runs. There is no gap for them to drift into.

05 Project layout

The top level is always just src and test.

# project root
Belay/
├── belay.config.ts        # the anchor: how to boot src/ and how to knock on it
├── docs/
│   └── index.html
│
├── src/                   # ── the system under test, exactly as its own repo would look ──
│   ├── package.json           # Node? Kotlin? Go? Rust? It makes no difference
│   ├── Dockerfile
│   └── ...                    # Belay never looks inside
│
└── test/                  # ── black-box scenarios ──
    ├── support/
    │   ├── world.ts           # per-scenario context (drivers, state)
    │   ├── hooks.ts           # Before / After, isolation between scenarios
    │   ├── steps/             # shared steps (status codes, JSONPath assertions)
    │   └── drivers/           # surface adapters — no domain logic
    │       ├── http.ts
    │       ├── graphql.ts
    │       ├── grpc.ts
    │       ├── ws.ts          # WebSocket / SSE and other streaming
    │       ├── browser.ts     # page control · capture · visual checks
    │       ├── cli.ts
    │       └── mocks.ts       # programming stubs · asserting on what they received
    │
    ├── contracts/             # protocol schemas — where drivers get their types
    │   ├── schema.graphql
    │   ├── order.proto
    │   └── pg.openapi.yaml    # the external payment stub's contract
    │
    ├── baselines/             # approved screenshots — changes go through review
    │   └── L2-checkout/
    │       ├── cart-empty.desktop.png
    │       └── cart-empty.mobile.png
    │
    └── features/
        ├── L0-smoke/
        │   └── health.feature
        ├── L1-account/
        │   ├── signup.feature      # HTTP
        │   ├── signup.steps.ts
        │   └── login.feature
        ├── L1-catalog/
        │   ├── search.feature      # GraphQL
        │   └── search.steps.ts
        ├── L2-order/
        │   ├── checkout.feature    # gRPC + WebSocket
        │   └── checkout.steps.ts
        ├── L2-ui-checkout/
        │   ├── cart.feature        # browser journey + visual checks
        │   └── cart.steps.ts
        └── L3-cross/
            └── signup-to-first-order.feature
PathRoleRule
src/The system under testNo constraints on its contents. Replaceable wholesale
test/features/Scenario layersDirectories are layers. Keep a .feature and its steps together
test/support/Drivers and contextSurface adapters only. No domain logic
test/contracts/Protocol schemasGraphQL SDL, .proto, OpenAPI. Where driver types come from
test/baselines/Approved screenshotsCommitted. Updated only through explicit, reviewed approval
belay.config.tsThe run contractInfra, boot, readiness, surfaces, reports, cleanup

06 The execution contract

One file at the root holds everything needed to put this project into a testable state.

This file is the only thing Belay knows about src/. How to build it, what it depends on, when to consider it ready, and which windows to knock on. Anything not written here is invisible to the tests — which is the point.

belay.config.ts
import { defineConfig } from '@beoks/belay';

export default defineConfig({
  // 1. Whatever src/ needs around it — containers, mock servers, anything
  infra: [
    { name: 'postgres', image: 'postgres:16', ports: ['5432'],
      env: { POSTGRES_PASSWORD: 'test' },
      ready: { tcp: '5432' } },

    { name: 'payment-gateway', // stand-in for the payment provider
      stub: { port: 4010 },
      // A programmable in-process stub: starts in milliseconds, needs no Docker },
  ],

  // 2. How to boot src/. One command is enough
  app: {
    build: 'docker build -t belay-sut ./src',
    start: 'docker run --rm -p 8080:8080 --env-file .belay/env belay-sut',
    env: {
      DATABASE_URL: 'postgres://postgres:test@localhost:5432/app',
      PAYMENT_BASE_URL: 'http://127.0.0.1:4010',
    },
    // Without a readiness probe every run races the boot
    ready: { http: 'http://localhost:8080/health', timeoutMs: 60_000 },
  },

  // 3. The only windows tests may knock on.
  //    What is declared here exists; reaching the SUT any other way breaks the black box.
  surfaces: {
    http:    { baseUrl: 'http://localhost:8080' },
    graphql: { endpoint: 'http://localhost:8080/graphql' },
    grpc:    { address: 'localhost:9090', proto: './test/contracts/order.proto' },
    ws:      { url: 'ws://localhost:8080/ws' },
    cli:     { command: 'docker run --rm belay-sut app-cli' },

    // The front-end surface — the screen a person actually sees
    browser: { baseUrl: 'http://localhost:3000',
               engine: 'chromium',
               viewports: { desktop: [1280, 800], mobile: [390, 844] },
               locale: 'en-US', timezone: 'UTC', colorScheme: 'light' },
  },

  // 4. Visual policy
  visual: {
    baselineDir: 'test/baselines',
    diff:      { threshold: 0.01 },
    interpret: { enabled: true, gate: false },  // never gates by default — see 10
    capture:   { onFailure: ['screenshot', 'dom', 'console'] },
  },

  // 5. Control — declared windows onto state you cannot create from outside
  control: {
    clock:  { via: 'http', endpoint: '/__test/clock' },   // a test-only contract the SUT serves
    seed:   { via: 'http', endpoint: '/__test/seed' },
    faults: { targets: ['postgres', 'payment-gateway'] },      // what may be slowed or cut
  },

  // 6. Where the scenarios are
  suite: {
    features: 'test/features/**/*.feature',
    steps:    'test/**/*.steps.ts',
    support:  'test/support/**/*.ts',
  },

  // 7. Isolation between scenarios — leaked state eats trust
  isolation: {
    between: 'scenario',
    // Either a shell command or a request to a declared window. Neither touches internals.
    reset:   { http: '/__test/seed', body: { data: { todos: [] } } },
    browser: true,    // a fresh context every time — cookies, storage, session
    mocks:   true,    // stub programming and recorded calls both cleared
  },

  // 8. Deadlines — so one stuck step cannot hold the run hostage
  timeouts: { step: 30_000, scenario: 120_000, hook: 30_000 },

  // 9. Results and cleanup
  report:   { formats: ['html', 'junit', 'json'], outDir: '.belay/reports', trace: true },
  teardown: { keepOnFailure: true },
});
Running it is always one line. belay run — typed by a person, by CI or by an agent, the same command, the same order, the same result. If part of the setup lives in someone's head, that suite is not something to trust.

07 Run lifecycle

What one belay run does — go up, stay protected, come back down.

The nine phases of one belay run, drawn as a climbing route Phases 1 to 4 climb: preflight, infra up, build and start, ready. Phases 5 to 7 run level under observation: scenarios, deadlines, evidence. Phases 8 and 9 descend: teardown and exit code. GO UP STAY PROTECTED COME DOWN 01 02 03 04 05 06 07 08 09 preflight infra up build · start ready? scenarios deadlines evidence teardown exit code
Every phase is a gate on the next one — nothing is built before the ground is checked, and no scenario runs before readiness is proven. The descent is not conditional: whatever breaks on the way up, 08 and 09 still happen.
01
Read the config, check the ground Load belay.config.ts and check port conflicts, required tools and leftovers from a previous run. If this fails, nothing is started.
02
Start the mock infrastructure Databases, brokers and external API stubs come up in dependency order, each waited on until its own ready condition holds.
03
Build and start the SUT app.buildapp.start. The environment it receives points at the infrastructure just started. The SUT has no idea it is under test.
04
Decide it is ready app.ready must pass before anything proceeds. On timeout the boot log is attached to the failure — telling "it was slow" apart from "it crashed" is stated, not guessed.
05
Execute scenarios Layer by layer, L0 → L3. Each scenario gets reset state, reset stubs and a fresh browser context, and every protocol exchange is recorded to the trace.
06
Watch the deadlines Steps, hooks and scenarios each carry a limit (30s · 30s · 120s by default). A step that never finishes fails by name and the run continues — one stuck step must never take the CI job with it.
07
Collect the evidence Step timeline, protocol transcripts (HTTP · GraphQL · gRPC · WS), screenshots and visual diffs, the DOM and console at the point of failure, and the process logs — all under .belay/reports.
08
Tear down In reverse order. With keepOnFailure on, a failed run's environment is left standing for a post-mortem.
09
Settle the exit code Any failure means non-zero. There is no ambiguous pass — skipped and undefined steps are stated in the result too.
Whatever breaks along the way, Belay always brings back down what it brought up, in reverse. Orphaned containers and held ports poison the next run, and a poisoned run is worse than no run because it looks like a result.

08 Writing scenarios

Written in Gherkin, so the specification a person reads and the check a machine runs are the same sentences — which is what makes the spec reviewable by someone who will never open src/.

test/features/L1-account/signup.feature
Feature: email signup

  An unused email creates an account and sends a verification mail.
  An email already in use is refused.

  Background:
    Given the mail stub is empty

  Scenario: signing up with a new email succeeds
    Given "ada@example.com" is not registered
    When I sign up as "ada@example.com"
    Then the status is 201
    And the response contains an account id
    And 1 verification mail is sent to "ada@example.com"

  Scenario: an email already in use is refused
    Given an account exists for "ada@example.com"
    When I sign up as "ada@example.com"
    Then the status is 409
    And no verification mail is sent

  Scenario Outline: malformed emails are refused
    When I sign up as "<input>"
    Then the status is 400

    Examples:
      | input           |
      | not-an-email    |
      | @example.com    |
      |                 |

What makes a scenario good

do

Say only what is observable

"the status is 409", "1 verification mail is sent" — facts checkable from outside.

don't

Describe the implementation

"UserService.create is called", "a row appears in the users table" — statements that break when the structure changes.

do

Make each scenario self-contained

Never lean on leftovers from the previous one. Shuffle the order and the result must not change.

watch

Write the failure paths too

A spec with only happy paths is the most dangerous kind when porting. Pin refusals, duplicates and timeouts as well.

09 Step definitions

Always TypeScript. Whatever language src/ is, the belayer speaks one.

Step implementations should be thin. Translate the sentence into a driver call, assert on the response, and stop there. Reimplementing domain rules here turns the test suite into a second implementation of the system.

test/features/L1-account/signup.steps.ts
import { Given, When, Then, expect } from '@beoks/belay';
import type { AppWorld } from '../../support/world';

Given('{string} is not registered', async function (this: AppWorld, email: string) {
  // Check through the public API only. Never go digging in the database.
  const res = await this.http.get(`/api/v1/accounts?email=${encodeURIComponent(email)}`);
  expect(res.status).toBe(404);
  this.ctx.email = email;
});

Given('an account exists for {string}', async function (this: AppWorld, email: string) {
  const res = await this.http.post('/api/v1/accounts', { email, password: 'P@ssw0rd!' });
  expect(res.status).toBe(201);
  this.ctx.email = email;
  await this.mock.mail.clear();   // side effects of setup are not under observation
});

When('I sign up as {string}', async function (this: AppWorld, email: string) {
  this.last = await this.http.post('/api/v1/accounts', { email, password: 'P@ssw0rd!' });
});

Then('the status is {int}', function (this: AppWorld, status: number) {
  expect(this.last.status).toBe(status);
});

Then('{int} verification mail is sent to {string}',
  async function (this: AppWorld, count: number, to: string) {
    // The mail stub is ours, so reading its inbox keeps the black box intact.
    const mails = await this.mock.mail.waitFor({ to, timeoutMs: 5_000 });
    expect(mails).toHaveLength(count);
});
test/support/world.ts
import { World, setWorldConstructor } from '@beoks/belay';

export class AppWorld extends World {
  // World already carries a driver per surface:
  //   this.http · this.gql · this.grpc · this.ws · this.cli · this.page
  //   this.mock.<name>  — the stubs and mail servers you declared
  //   this.control      — clock · seed · faults
  //   this.last         — the most recent Exchange, which shared steps assert on
  // Touching a surface that is not configured fails, telling you what to declare.

  // Keep only scenario-scoped state and thin helpers here.
  token?: string;

  async authenticate(email: string, password = 'P@ssw0rd!'): Promise<void> {
    const response = await this.http.post('/api/v1/sessions', { email, password });
    this.token = (response.body as { token?: string })?.token;
  }
}

setWorldConstructor(AppWorld);
Drivers never reach outside surfaces. The moment step code imports from src/ or execs into a container, it stops being a black box, and passing no longer says anything about a port to another stack.

10 Protocol drivers

HTTP · GraphQL · gRPC · WebSocket — different surfaces, the same shape of sentence.

Backends rarely speak one protocol. Reads over GraphQL, internal calls over gRPC, live updates over WebSocket, admin over REST. If each protocol brought its own way of asserting, the scenarios would turn into technical documentation and collapse along with the next rewrite of src/.

So every Belay driver returns the same Exchange. Status, body and metadata are normalised into one shape, and assertions about status codes or JSON paths are written once and reused across protocols.

Six protocols normalise into one Exchange, which shared steps assert on http, graphql, grpc, grpc stream, ws and cli all converge into a single Exchange shape carrying ok, status, body, errors, meta and elapsedMs. One set of shared steps then asserts against that shape regardless of protocol. http graphql grpc grpc stream ws cli NORMALISED Exchange ok · status · body errors[] · meta · elapsedMs WRITTEN ONCE shared steps the response succeeds the error code is …
The narrow waist in the middle is the whole point. Add a protocol and you add a driver, not a new vocabulary — and a scenario sentence stays true when the surface behind it changes.
ProtocolDriver callWhat the scenario observes
HTTPthis.http.post(path, body)Status, headers, body, elapsed time
GraphQLthis.gql.query(doc, vars)data shape, errors[].extensions.code, partial success
gRPCthis.grpc.call('svc/Method', msg)Status code, trailers, the response message
gRPC streamthis.grpc.stream(...)Message order, terminal status, what must not arrive
WebSocketthis.ws.connect()waitFor()Messages received, order, reconnection, close code
Queuethis.queue.publish(topic, msg) · consume(topic)waitFor()Messages published, keys and headers, order, what must not arrive (quietFor)
CLIthis.cli.run(args)Exit code, stdout, stderr
Status (1.0.0). The HTTP, GraphQL, WebSocket, CLI and gRPC drivers, and the Exchange normalisation, are implemented and exercised by the examples on every run (gRPC in examples/04-grpc — unary, status codes, server streaming). The gRPC surface needs optional peers, @grpc/grpc-js and @grpc/proto-loader; the queue surface needs kafkajs. The queue driver creates missing topics through the admin API, so it does not depend on broker auto-create. Its verification status is stated honestly in section 17 — field-verified against a live broker, not continuously verified by an example.

The normalised exchange

test/support/drivers/types.ts
export interface Exchange<T = unknown> {
  ok: boolean;
  status: number;          // HTTP status · gRPC code · WS close code, normalised
  body: T;                  // data for GraphQL, the message for gRPC
  errors?: { code: string; message: string; path?: string[] }[];
  meta: Record<string, string>;   // headers · trailers · frame metadata
  elapsedMs: number;
  // Every Exchange is traced automatically — no logging to add
}

Which means these steps are written once and used everywhere.

test/support/steps/common.steps.ts
Then('the response succeeds', function (this: AppWorld) {
  expect(this.last.ok).toBe(true);
});

Then('the error code is {string}', function (this: AppWorld, code: string) {
  expect(this.last.errors?.[0].code).toBe(code);   // GraphQL · gRPC · HTTP alike
});

Then('the response {string} is {string}',
  function (this: AppWorld, path: string, value: string) {
    expect(String(readPath(this.last.body, path))).toBe(value);
});

gRPC keeps both halves of the contract

A refusal has two answers: what the transport said, and what the domain meant. Porting has to preserve why something was refused, not just that it was — so Belay keeps the transport verdict in meta and lifts a domain code out of the details string into errors[0].code.

test/features/L1-inventory/reserve.feature
Scenario: reserving more than available is refused
  When I reserve 9 of "CAM-001"
  Then the gRPC status is "FAILED_PRECONDITION"
  And the error code is "OUT_OF_STOCK"

Streaming has a time axis

test/features/L2-order/checkout.steps.ts
Given('I am subscribed to the order channel', async function (this: AppWorld) {
  this.sub = await this.ws.subscribe({ topic: 'order', id: this.ctx.cartId });
});

Then('I receive order status {string} within {int} seconds',
  async function (this: AppWorld, status: string, sec: number) {
    // A condition with a deadline, not a sleep: fixed waits are how suites go flaky.
    const msg = await this.sub.waitFor(
      (m) => m.type === 'order.status' && m.status === status,
      { timeoutMs: sec * 1000 },
    );
    expect(msg.orderId).toBe(this.last.body.orderId);
});

Then('no other order notification arrives', async function (this: AppWorld) {
  await this.sub.quietFor({ ms: 1_000 });
});
Streaming assertions need both a condition and a deadline. waitFor also scans what already arrived, so a message landing between the action and the assertion is not missed. And "nothing should arrive" only means something once you have actually waited for it — that is what quietFor is.

11 Browser and visual checks

A front end's public interface is the screen. So Belay opens it, clicks it, captures it and reads it.

For a front-end project, "black box" means the browser. Belay drives a real engine the way a person would and checks the result at three levels. Each level has a different degree of determinism — and therefore a different right to fail your build.

gates

1 · Structural assertions

Roles, names and states read from the accessibility tree. Fully deterministic, so this is always the first basis for pass or fail.

gates

2 · Visual regression

Perceptual diff against an approved baseline. Thresholds are declared, so it is reproducible — on the machine that approved it.

does not gate

3 · Visual interpretation

A model reads the capture and judges claims like "the legend does not overlap". Catches what no baseline covers, but is not deterministic.

Levels 1 and 2 pass through the pass/fail gate; level 3 goes around it into the report Structural assertions and visual regression are deterministic and reach the gate, so they decide the exit code. Visual interpretation bypasses the gate and lands in the report as a warning for a person to review and promote. 1 · Structural assertions roles, names and states 2 · Visual regression diff against an approved baseline 3 · Visual interpretation a model reads the capture GATE pass / fail · exit code deterministic and reproducible this is what breaks the build bypasses the gate report · warning a person reviews it, then promotes it into level 1 or level 2
The bypass is deliberate. A judge that is right most of the time would make every green result mean probably — so level 3 may only produce work for a human, never a verdict. The normal path for a real finding is promotion into level 1 or 2, where it becomes deterministic and can gate.
Status (1.0.0). The browser driver is implemented and verified by examples/05-browser — semantic locators, capture and baseline comparison. The visual gate was checked by breaking it on purpose: changing one CSS colour produced a 3.33% difference and failed the run. It needs optional peers: npm i -D playwright pngjs pixelmatch && npx playwright install chromium
Visual interpretation does not fail your build by default. Wiring a non-deterministic judge straight into pass/fail would destroy the property that makes the whole suite worth trusting. Its findings land in the report as warnings; when a human confirms one is a real defect, the fix is to promote it into a structural assertion or a baseline. If you do want it to gate, turn on visual.interpret.gate — the model's reasoning is recorded alongside the verdict.

The scenario

test/features/L1-cart/cart.feature
Feature: the cart page

  Scenario: an empty cart explains itself and refuses checkout
    Given the cart is empty
    When I open the cart page
    Then I see "Your cart is empty"
    And the "Checkout" button is disabled

  Scenario: changing the quantity updates the total
    Given the cart holds 1 of "CAM-001"
    When I open the cart page
    And I set the quantity to 3
    Then the total reads "$387.00"
    And no console errors occurred

  @visual
  Scenario Outline: the checkout button stays reachable on every viewport
    Given the cart holds 2 of "CAM-001"
    When I resize the window to "<viewport>"
    And I open the cart page
    Then the "Checkout" button is enabled
    And the screen "cart-filled-<viewport>" matches its baseline

    Examples:
      | viewport |
      | desktop  |
      | mobile   |

Browser steps

test/features/L1-cart/cart.steps.ts
When('I set the quantity to {int}', async function (this: AppWorld, quantity: number) {
  // Found by label, never by CSS class: a class disappears with the framework.
  await this.page.byLabel('Quantity').first().select(String(quantity));
  await this.page.waitForStable();
});

Then('the {string} button is disabled', async function (this: AppWorld, name: string) {
  await this.page.byRole('button', { name }).expectDisabled();
});

// ── deterministic: this one gates ─────────────────────────────
Then('the screen {string} matches its baseline',
  async function (this: AppWorld, name: string) {
    await this.page.waitForStable();   // animations, fonts and images settled
    const shot = await this.page.capture(name);
    await shot.toMatchBaseline();       // missing baseline fails; approval is a separate act
});

Then('no console errors occurred', function (this: AppWorld) {
  expect(this.page.consoleErrors()).toEqual([]);
});
A missing baseline fails; it is not written silently. If the first run happened to render a broken page, generating the baseline automatically would enshrine that breakage as the standard. A human looking at the capture before approving it is the entire safety net.

Locator rules

On the front end, keeping the black box intact comes down to what you find elements by. Only criteria that survive a move from React to Svelte let the scenarios go on being the answer key.

How you locateVerdictWhy
byRole('button', { name })preferredMatches how a user perceives it, and forces accessibility to exist
byLabel · byPlaceholderpreferredThe semantic identifiers of form controls
byTestId('cart-total')fallbackOnly where a role cannot express it. It is a contract, so a port must keep it
.css .class > div:nth-child(2)neverBreaks on a style change; wiped out entirely by a framework swap
Component instances or internal storesneverA black-box violation, and a concept that does not exist outside the browser

Front-end-only projects

When src/ is only a front end, contract-based mocks stand in for the backend. Start stubs from the OpenAPI or GraphQL schema in test/contracts/, point the browser's network at them, and every loading, error, empty and slow state becomes reproducible without a server.

// Shaping the backend's answer to reach a screen state — the stub is ours, so this is allowed
Given('the product list is delayed by {int}ms', async function (this: AppWorld, ms: number) {
  await this.mock.api.onGet('/api/v1/products').delay(ms);
});

Given('the product list fails', async function (this: AppWorld) {
  await this.mock.api.onGet('/api/v1/products').reply(503);
});

// → lets you pin a screen contract like "then a retry button is shown", deterministically

12 Controlling the environment

Producing situations you cannot reach from outside — without reaching inside.

The things most worth checking usually cannot be reproduced with an ordinary request. A coupon that expires in thirty days. The moment the payment provider returns 502. What retries do when the database blinks. The common temptation here is "let the test write to the database directly" — and portability dies right there.

Belay uses control surfaces instead. There are two kinds of target, and they have different rules.

free rein

Mock infra — ours

External API stubs, brokers and test databases were started by Belay. Change their answers, kill them, and assert on what they received.

declared windows only

The SUT — not ours

Moving its clock or seeding it goes through a test-only contract declared under control. No exec into the container, no writing to its state.

Mock infra may be driven freely; the SUT only through declared windows A scenario drives mock infrastructure with no restrictions because Belay started it. Reaching the system under test goes through narrow declared windows such as slash underscore underscore test slash clock and slash underscore underscore test slash seed. No exec into the container and no direct writes to its state. SCENARIO this.mock.… this.control.… no restrictions MOCK INFRA — OURS reply · delay · fail · kill · received() Belay started it, so a scenario may program it at will declared windows only /__test/clock /__test/seed THE SUT — NOT OURS reachable only through what it declares no exec into the container, no writing to its state
Both rows produce situations you could not reach with an ordinary request — the difference is who owns the thing being manipulated. A window you declare becomes part of the contract, so a rewrite in another language has to serve it too, and the suite keeps working.
ControlCallWhat it lets you verify
Clockthis.control.clock.advance('P30D')Expiry, schedules, grace periods, settlement cutoffs
Seedthis.control.seed('catalog.yaml')Large-data preconditions, pagination
Stub responsethis.mock.pg.reply(502)Fallback, retry and user messaging under external failure
Stub latencythis.mock.pg.delay(5_000)Timeout handling, loading states, circuit breakers
Received callsthis.mock.pg.received()Idempotency, retry counts, the payload actually sent
Faultsthis.control.faults.partition('pg')Reconnection, health checks, data consistency
Networkthis.control.faults.latency('pg', 800)Behaviour and timeout boundaries under a slow dependency
test/features/L3-cross/payment-outage.feature
Feature: surviving a payment outage

  Scenario: a transient outage holds the order and charges exactly once
    Given items are in the cart
    And the payment provider answers 502 to the first 2 requests
    When I place the order
    Then the response succeeds
    And I receive order status "PENDING_PAYMENT" within 30 seconds
    And every charge that reached the provider carries the same idempotency key

  Scenario: an unpaid order is cancelled after three days
    Given an order in "PENDING_PAYMENT"
    When 3 days pass
    Then the order status is "CANCELLED"
    And the stock is restored
Given('the payment provider answers {int} to the first {int} requests',
  async function (this: AppWorld, status: number, times: number) {
    await this.mock.pg.onPost('/charges').replyTimes(times, status).reply(200);
});

When('{int} days pass', async function (this: AppWorld, days: number) {
  await this.control.clock.advance(`P${days}D`);
});

Then('every charge that reached the provider carries the same idempotency key',
  async function (this: AppWorld) {
    const keys = this.mock.pg.received('POST /charges')
      .map((call) => call.headers['idempotency-key']);
    expect(new Set(keys).size).toBe(1);   // however many retries, never a double charge
});
A control surface is part of the contract. If you decide on /__test/clock, a Go rewrite has to serve the same window. That sounds like a cost but it is a gain: "can time be moved forward?" is what decides whether a system is testable at all, and declaring it means the port cannot lose that property.

13 The black-box boundary

An explicit line about what may and may not be touched.

test/ reaches src/ only through declared surfaces; every other route is blocked test/ sits outside the deployed artefact. The only way in is the declared surfaces: http, graphql, grpc, ws, cli, browser and the test-only control endpoints. Everything inside the boundary — routes, domain services, the database schema, internal classes — is invisible to the suite, and any route that reaches it directly is blocked. OUTSIDE test/ the belayer SURFACES http graphql grpc ws cli browser /__test/* src/ — THE DEPLOYED ARTEFACT routes · handlers domain services database schema internal classes Belay never looks in here. This structure is free to change — nothing inside it is a contract. import src/ · read its database · exec into the container · match log strings allowed — through a surface the config declares never — anything that reaches past the boundary
The dashed rectangle is the only line that matters: what is inside it may be replaced wholesale, and what crosses it is the contract. Every "never" in the table below is the same rule — a route that pierces the boundary makes a passing suite say nothing about a port.
ActionVerdictReason
HTTP · GraphQL · gRPC · WS callsallowedThe public contract — what a rewrite must preserve
Running the CLI, asserting on exit code and stdoutallowedA window users actually use
Observing published events and messagesallowedSide effects visible from outside
Driving the browser, locating by role or labelallowedExactly how a person uses the screen
Capturing screenshots, comparing approved baselinesallowedThresholds are declared, so it is deterministic
Programming mock infra and asserting on its received callsallowedBelay started those; it is still the SUT's observable behaviour
Using declared control surfaces (/__test/*)allowedPart of the contract; a port implements it too
Reading or writing the database against its internal schemaneverBreaks on every structural change, and cannot survive a port at all
Importing src/ or calling internal functionsneverVoid the moment the language changes
Locating elements by CSS class or component instanceneverBreaks on a style change; wiped out by a framework swap
exec into the SUT container, or setting its clock directlyneverIf you need the control, declare it as a window
Asserting by matching log stringsneverLogs are not a contract. If you need to observe it, promote it to an interface
Letting model interpretation gate on its ownavoidA non-deterministic verdict undermines the premise that results are trustworthy
Fixed sleep to line up timingavoidThe main cause of flakiness. Wait on a condition instead
Sharing state between scenarios, cookies includedavoidOrder dependence destroys confidence in the result

There is one grey area worth naming: test-only interfaces. If a state genuinely cannot be produced from outside — moving time, seeding fixtures — do not drill inward. Have the SUT serve an explicit, documented window like /__test/clock. A port has to implement the same window, so it stays part of the contract rather than becoming an exception to it.

14 Scenario layers

Directories are layers, layers are execution order — and the grammar you read failures with.

L0 to L3: execution order runs bottom to top, and scope widens as you climb L0 smoke is the narrowest and runs first, then L1 capability, L2 flow and L3 cross and regression. A cut line shows that belay run --upto L1 stops after L1. L3 L2 L1 L0 RUN ORDER Cross & regression across domains and surfaces, plus past bugs pinned so they cannot return Flow a user journey across several calls and screens Capability one feature, one protocol or one screen state Smoke is it up, does it answer --upto L1 ✂ L1 green and L2 red ⇒ the capabilities hold; the wiring between them does not.
Because the layers run in order, the first failing layer is already a diagnosis — and that is the information you hand to an agent along with the failure list. Cutting the run at a layer is how you keep the inner loop fast without pretending the upper layers passed.
L0
SmokeIs it up, does it answer. If this breaks, nothing else runs; the cause is almost always the boot.
L1
CapabilityThe contract of a single feature: a request and response that completes within one protocol, or the state contract of one screen — empty, error, loading.
L2
FlowA user journey across several calls and screens. Cart → payment → order confirmed. What is under test is the consistency of the transitions.
L3
Cross & regressionScenarios that span domains and surfaces — order in the browser, notification over WS, survive a provider outage — plus bugs that actually happened, pinned so they cannot come back.

With layers, a failure is already a diagnosis. If L1 passes and L2 breaks, the individual capabilities are fine and the wiring is not — and when you hand it to an agent, that information goes with it. For fast feedback, cut the run: belay run --upto L1.

15 Porting and rewrites

The real reason Belay exists.

If the scenarios live outside src/ and speak only through public interfaces, then src/ is a replaceable part. Rewriting an Express prototype in Kotlin and Spring, swapping REST for GraphQL, moving a React screen to Svelte — the procedure is the same.

A port replaces src/ and two config keys; test/ is untouched Before, src/ is Express on Node and all layers are green. After, src/ is Spring on Kotlin and all layers are green again. Only build and start change in the config; surfaces, control and suite are unchanged, and test/ is identical in both. BEFORE — ALL LAYERS GREEN AFTER — ALL LAYERS GREEN src/ Express · Node 20 src/ Spring · Kotlin 21 REWRITTEN belay.config.ts build · start → npm surfaces · control · suite — unchanged belay.config.ts build · start → gradle surfaces · control · suite — unchanged test/ 42 scenarios · all green test/ the same 42 · all green PORT 2 keys identical
Read it as an equation: the two green columns are what makes the swap in the middle a port rather than a rewrite you hope is equivalent. If passing required editing the right-hand column, nothing was proved — the standard moved with the implementation.
  1. Take a baseline. Confirm every layer passes with the current src/ and keep the report. A spec that does not pass cannot be a standard.
  2. Swap the target. Put the new implementation in src/ and change only build and start. surfaces and test/ are not touched at all.
  3. Light the layers up in order. Get L0 green, then climb. Each layer passing is progress, and the remaining failures are the remaining work.
  4. Call it done. Every layer green means the port is complete. Not "I think I moved everything" — a confirmed fact.
If the only way to pass is to edit test/, it is usually one of two things: the scenario was secretly coupled to the old implementation (fix the scenario), or the contract genuinely changed (fix the spec, with agreement). Either way, never quietly lower a scenario to make it pass — that is the same as leaving slack in the rope.

What changes when the surface changes

Kind of changeWhat moves and what stays
Language or framework
Express → Spring
Only build/start. surfaces and test/ stay as they are. The purest form of a port.
Protocol
REST → GraphQL
The contract itself changes, so the driver calls in the steps change. But the .feature sentences must not — if the scenarios move, this is a feature change, not a port.
Front-end framework
React → Svelte
If roles, labels and data-testid survive, so do the steps. Needing to change a locator means the scenario was coupled to the framework.
Redesign Structural assertions hold; visual baselines are invalidated. Renewing a baseline is an explicit, reviewed approval — automatic renewal erases the safety net.

The same structure works when the work is handed to an agent. Give it write access to src/ and read-only access to test/, then let it loop on belay run. It cannot rewrite its own grading criteria, so the only thing that can converge is the implementation.

This paragraph has been measured, at the weakest link. A Haiku-class agent was handed a real adopted service (the RealWorld reference implementation, Express + Prisma + PostgreSQL, pinned under 50 scenarios) and told to rewrite it in Rust with test/ read-only. It converged — 50/50 in ~14 runs and ~25 minutes, three undocumented behaviors reproduced exactly because the failure list demanded them, grading criteria untouched. One caveat came out of the run: a small model stops early with failures remaining. The failure list supplies direction; keeping the loop running is the job of whoever holds the rope.

16 Results and trust

That it passed matters less than that the pass can be believed.

Evidence from every run

Step timeline, protocol transcripts, screenshots and visual diffs, process logs, and the DOM and console at the point of failure. You should not have to re-run anything to see what happened.

Non-deterministic verdicts kept apart

Visual interpretation is tallied separately as warnings, never mixed into pass/fail. The model's own words and the image behind them are kept, so a person can judge.

Flaky counts as failing

Retries are not used to manufacture green. An unstable scenario is quarantined and fixed — a safety net that wobbles is worse than none.

Undefined steps do not stay quiet

Steps with no definition and skipped scenarios are named in the result and subtracted from coverage. A silent pass is the most dangerous outcome there is.

Machine-readable output

junit.xml and json drop straight into CI gates and agent loops. The exit code alone should be enough to decide.

$ belay run

  ▸ infra     postgres ✓ 1.8s   kafka ✓ 4.1s   payment-gateway ✓ 0.6s
  ▸ build     belay-sut ✓ 24.1s
  ▸ ready     http://localhost:8080/health ✓ 3.2s
  ▸ surfaces  http · graphql · grpc · ws · browser(chromium) ✓

  L0-smoke          2 scenarios   2 passed
  L1-account       14 scenarios  14 passed
  L1-catalog        6 scenarios   6 passed            graphql
  L2-order          9 scenarios   8 passed   1 failed   grpc · ws
  L2-ui-checkout    7 scenarios   7 passed            browser · ⚠ 1 visual warning
  L3-cross          4 scenarios   4 passed

  ✗ L2-order/checkout.feature:31  reserving more than available is refused
      Then the gRPC status is "FAILED_PRECONDITION"
        expected FAILED_PRECONDITION, received INTERNAL
        → .belay/reports/2026-07-29T20-14/L2-order-31/

  ⚠ L2-ui-checkout/cart.feature:24  [visual interpretation · not a gate]
      claim: "no price is clipped or overlapping" — does not hold (confidence 0.71)
      "on the mobile viewport the total overflows its container"
      → cart-filled.mobile.png · review, then promote to an assertion or a baseline

  ▸ teardown  keepOnFailure=true — environment kept (belay down to clean up)

  41/42 passed · 1 warning · 2m 38s · exit 1

17 Where this is verified

A design document that runs ahead of the code betrays its first reader. Here is which example proves which claim.

examples/ is a regression suite, not a gallery. npm run examples runs all of them exactly as a user would, and CI treats a failure there as a broken build.

ExampleLanguageWhat it proves
01-http-todoEnglish In-process stubs without Docker · real SMTP capture · fault injection (500 · latency · partition) · clock control · waiting on async side effects
02-portingEnglish The same scenarios pass against both a Node and a Python implementation. The central claim of this document
03-graphql-wsEnglish GraphQL and WebSocket woven into one flow · waitFor/quietFor · refusing to call partial success a success
04-grpcEnglish Unary calls · transport code and domain code kept apart · idempotent replay · server streaming
05-browserEnglish Semantic locators · approved baseline comparison · per-viewport outlines · console error observation

What is not implemented yet

What does not work — or deliberately never will — is named here rather than left to be discovered.

  • The queue (Kafka) driver — implemented in 1.0 (surfaces.queue, optional peer kafkajs): publish with keys and headers, latest-offset subscriptions with the same waitFor/quietFor/mark replay contract as WebSocket, automatic topic creation via the admin API. Field-verified against a live single-node Kafka (apache/kafka 3.8, KRaft) — produce→consume roundtrip, key/header integrity, latest-offset semantics, silence assertions — and its buffering semantics are pinned by unit tests against the declared client seam. Like the Docker infra path, it is not continuously verified: the examples deliberately run without Docker, so no example exercises a live broker on every run.
  • The Docker infra path — the code to start containers via image: exists and has been field-verified once (a Postgres container carried the RealWorld showcase, macOS/colima), but every example still runs without Docker, so it is not continuously verified. Prefer command:, stub: and mail:. Since 1.0, ports: ['15432:5432'] publishes host ≠ container mappings, and a readiness command can reach the generated container by name — ready: { command: 'docker exec {container} pg_isready' } — which is the honest probe behind a VM port-forwarder, where TCP accepts before the service does.

Windows

Supported natively (it used to be refused). Teardown kills the whole process tree with taskkill /T where POSIX kills the process group, and belay down additionally hunts down anything still holding a declared port. Two platform notes: stop signals are a POSIX concept — on Windows every stop is forceful, so an app that needs a graceful-shutdown window will not get one there; and --link symlink falls back to a directory junction when symlinks would need Developer Mode. WSL2 remains fully supported.

This list emptying out is what progress means here. "Implemented" is a word reserved for things an example passes on every run. Anything else lets the documentation become quietly false, which is the exact condition this project exists to remove.

18 Glossary

The words used throughout.

TermMeaning
BelayIn climbing, holding the rope so a falling partner stops. The name of this framework and its job.
SUTSystem under test — what sits in src/. Belay knows nothing of its internals.
SurfaceA public window tests may knock on: HTTP, CLI, a queue. Only what is declared exists.
FeatureAn executable specification written in Gherkin: the spec a person reads and the check a machine runs.
Step definitionThe TypeScript that translates a Gherkin sentence into a driver call. Kept thin.
DriverAn adapter wrapping one surface. Carries no domain logic.
ExchangeThe normalised result a driver returns. The same shape across protocols, which is what makes shared steps possible.
Semantic locatorFinding an element by role, label or accessible name. Survives a framework change.
Visual baselineAn approved reference screenshot. Committed, and renewed only by explicit, reviewed approval.
Visual interpretationA model's judgement of a capture. Non-deterministic, so a warning rather than a gate by default.
Control planeDeclared surfaces for state you cannot create from outside: clock, seed, fault injection.
Mock infraTest doubles for what the SUT depends on. Belay started them, so scenarios may program them freely.
LayerScenario layers L0–L3. Execution order, and the grammar for reading failures.
Baseline (run)The all-layers-green record taken before a rewrite. The reference a port is measured against.