Self-test: the release smoke check

Since 0.6

--self-test <scenario> plays a scenario inside the engine and exits with its verdict.

--self-test <scenario> plays a scenario inside the engine and exits with its verdict. It is how a release build is smoke-tested on every platform: a release engine serves no query API (G2), and on Android, iOS and the web there is nothing to connect to anyway, so the check runs in-process and needs no server.

argus run <project | scene.json> --self-test <scenario> [--headless]
<exported game> --self-test <scenario> [--headless]

Exit code: 0 when every step and every check passed, 1 when anything failed, 2 when the scenario could not be found or read.
Report: self-test.json, written to $ARGUS_SELF_TEST_OUT if set, else beside $ARGUS_LOG_FILE (an exported game's logs/ folder in its unpack directory), else the platform's log folder (%LOCALAPPDATA%\Argus\logs, ~/Library/Logs/Argus, $XDG_STATE_HOME/argus/logs or ~/.local/state/argus/logs). Captures are saved beside it as self-test-<name>.png.
Stdout: one line, argus-self-test {"ok":…,"scenario":…,"frames":…,"report":"<path>"}.
Surface: never a window. Without --headless the engine renders offscreen, so the capture checks run; with --headless there is no GPU and every capture is skipped (reported as skipped, never faked). An exported game's launcher forwards --self-test and runs the engine offscreen unless --headless is given.
Serving: none. --self-test conflicts with --serve, --watch and the session flags.
Time: fixed 60 Hz ticks. Seconds and milliseconds in a scenario become ticks, so a scenario is deterministic in the critic build and never depends on how fast the machine is.

Finding the scenario

In this order:

a file path (absolute, or relative to the working directory; an exported game's launcher runs the engine in its unpack directory, so give it an absolute path);
<project root>/selftest/<name>.json. A project export packs the project's selftest/*.json at app/selftest/, so a shipped game runs its own scenarios by name;
smoke, the built-in scenario: 180 ticks, no script errors, and a frame that is not blank.

The format

One format for scripted play: the bench's step language (bench/README.md, "Scripts"; bench/lib/exportrun.mjs) with the same spellings, run against the world directly instead of over HTTP, plus an expect block.

{
  "selfTest": 1,
  "description": "What this scenario proves.",
  "maxFrames": 1800,
  "steps": [
    { "wait": "screen", "name": "main-menu", "top": true },
    { "press": "Start" },
    { "wait": "screen-gone", "name": "main-menu" },
    { "axis": "move", "value": [0, 1], "frames": 60 },
    { "capture": "playing" }
  ],
  "expect": { "goals": 1, "screen": "hud", "top": false, "scriptErrors": 0, "capture": "auto" }
}

selfTest is required, so a stray JSON file is never run as a scenario. Unknown keys are refused. maxFrames (default 3600) is the tick budget for the whole run; running out fails it.

Steps

step	does	differs from the bench
`{ "frames": 30 }`	run 30 ticks	self-test only
`{ "sleep": 500 }`	run 500 ms of ticks (30)	ticks, not wall-clock
`{ "wait": "screen", "name": "hud", "top": true }`	until the screen is on the stack (on top, with `top`)
`{ "wait": "screen-gone", "name": "main-menu" }`	until it is not
`{ "wait": "text", "match": "score" }`	until a visible node's text contains it (case-blind)	a plain substring: the engine links no regex engine, and a pattern with regex characters is refused
`{ "press": "Start" }`	activate the nearest focusable, enabled ancestor of the visible text `Start` (whole, case-blind), through the same code `POST /ui/activate` runs; `{ "press": { "id": "main-menu:start" } }` names the node	waits `settleMs` (default 1000) of ticks after start first, as the bench does; a label starting `^` is not a regex here
`{ "action": "jump", "frames": 6, "repeat": 3, "gapFrames": 18 }`	hold the action, through the same code `POST /input` places holds with; fails if nothing consumes it within `bindTimeout` seconds (default 3)	`gapMs` also accepted
`{ "axis": "move", "value": [0, 1], "frames": 30 }`	the same, for a two-axis action
`{ "touch": [0.2, 0.8], "to": [0.2, 0.5], "frames": 60 }`	put a finger down at a normalised screen position, walk it to `to` (default: where it landed) and lift it after `frames` ticks, through the same holds; fails if no action reads touch within `bindTimeout` (docs/INPUT.md, "Touch")	self-test only
`{ "capture": "menu" }`	render, save `self-test-menu.png`, fail if the frame is blank	skipped under `--headless`
`{ "expect": "goals", "min": 2 }`	until that many distinct goals have been reached (`timeout` seconds)	self-test only
`{ "repeatUntil": { "screen": "game-over" }, "max": 80, "do": [ … ] }`	the inner steps again and again until the screen is on the stack (`screenGone`: until it is not), checked before each round; fails after `max` rounds (default 30) or when an inner step fails	no `exited` condition

Every waiting step takes timeout in seconds (default 15). A failed step stops the run: every later step assumed the ones before. The bench steps that only make sense from outside the process — launch, restart, checkUi, track, read, and the expect kinds var, moved, saved and exit — are refused by name rather than skipped.

Checks (`expect`)

key	passes when	default
`goals`	at least this many distinct goals were reached (`goal-reached` events), or every goal the scene declares with `"all"`	not checked
`screen`, `top`	the screen is on the stack at the end (on top, with `top`)	not checked
`scriptErrors`	the run recorded at most this many script errors, load-time errors included	0
`capture`	the final frame is not blank: `true` requires it, `false` skips it, `"auto"` checks it when the run renders	`"auto"`

"Not blank" means at least 0.5% of pixels differ from the frame's first pixel by more than 8 in some channel: a clear-colour frame, a black frame or a failed draw fails it.

Where it runs

crates/app/tests/self_test.rs — the checked-in fixtures/selftest/starter-course-start.json on the starter course, and each way a scenario has to fail.
packages/export/test/native.test.ts — --self-test smoke on a release export.
scripts/release/matrix.sh — every platform the machine can build: the same scene export headless on Linux and Windows, and a project export offscreen on Linux, recorded in dist/release/<version>/MATRIX.json.

Known gap (not the self-test's): a scene-only export carries no font, and a scene that declares text refuses to load without one, so such an export can only run --headless. A project export packs a font under app/ui/fonts/ and renders. The matrix's capture check therefore runs on a project.