Execution worker recipes
Develop, deploy, diagnose, recover, and safely replace the Node/PostgreSQL Runtime worker.
These recipes assume the supported worker host:
import { config } from "@use-crux/core";
import { node } from "@use-crux/core/runtime";
import { postgres } from "@use-crux/postgres/runtime";
export default config({
runtime: node({ store: postgres() }),
});Local development
Run PostgreSQL, apply setup once, and keep the dev server and execution worker in separate terminals:
crux setup --apply
crux devcrux runtime workercrux dev refreshes generated artifacts after relevant saves. Restart the
execution worker after a target changes so its imported program matches the new
files. Use a separate database or namespace for each developer when sharing a
PostgreSQL server.
Supervise production shutdown
Build one image containing the application source, dependencies, Crux CLI, and generated Runtime files. Run the application and worker as separate service commands from that image.
Configure the supervisor to:
- send
SIGTERM, not an immediate kill; - allow more than 11 seconds before
SIGKILL; - restart the worker after an unexpected non-zero exit; and
- keep the worker replica count at one per namespace.
For a container, make crux runtime worker the signal-receiving process (for
example, use an exec-form command). Crux forwards shutdown to the full worker
process tree: a Unix process group receives SIGTERM; a Windows Job Object
receives a console break and is closed to terminate descendants after timeout.
Diagnose a duplicate worker
The actionable error says maintenance ownership is already held for the namespace in the durable store. Check for:
- an old container still terminating;
- a deployment strategy that overlaps replicas;
- two supervisors managing the same command; or
- two environments using the same database and namespace.
Keep the existing healthy owner running. Stop the unintended process and wait
for it to exit before retrying the replacement. PostgreSQL holds ownership with
a session-scoped advisory lock on a dedicated pooled connection. Shutdown
unlocks and releases that connection; closing the database session also
releases the lock. Configure the pool with max >= 2 so maintenance can use a
second connection while the ownership connection remains checked out.
Repair missing, stale, or incompatible artifacts
| Error | Meaning | Recovery |
|---|---|---|
SETUP_REQUIRED | program.ts or manifest.json is missing. | Run crux runtime generate in the deployed project root. |
ARTIFACTS_STALE | Program hash or target list does not match the manifest. | Regenerate the complete set and rebuild/redeploy it together. |
RUNTIME_ARTIFACT_MANIFEST_INVALID | JSON, generated module, config import, or exported program is invalid. | Fix the reported import or source error, then regenerate. |
RUNTIME_ARTIFACT_MANIFEST_INCOMPATIBLE | Manifest or program format is unsupported by this CLI. | Generate and run with the same installed Crux release. |
Never copy only one generated file between releases. Generation stages all changes and activates the manifest last so the worker sees one coherent set.
Set up PostgreSQL and verify restart recovery
Set DATABASE_URL, then inspect and apply the Crux-owned schema:
crux setup --check
crux setup --apply
crux setup --checkFor a restart drill:
- start the worker and submit work that suspends, waits on a timer, or remains queued;
- send
SIGTERMand wait for the command to exit; - start
crux runtime workeragain with the same database and namespace; and - verify the work resumes or completes once, from its last committed boundary.
Use application or observability evidence to verify the result. Do not infer recovery merely from a clean process exit.
Recover from a worker failure
Artifact and config failures happen before execution and leave durable data unchanged. Correct the error and restart. A fatal maintenance error closes the worker; correct database connectivity or the target failure and let the supervisor restart it. Work whose lease expired can then be reclaimed.
If shutdown reports that an active tick did not settle within 10 seconds, inspect the target or store operation. Future ticks are stopped, but Crux cannot physically cancel external work that already began. Make that operation idempotent before retrying.
Roll out or replace safely
Use stop-then-start deployment for a namespace:
- generate artifacts from the release source and build the image;
- confirm pending work does not reference a target removed by the release;
- send
SIGTERMto the old worker and wait for exit; - start exactly one new worker; and
- verify it acquires ownership and processes a known durable item.
Application processes may roll independently if they preserve every target needed by non-terminal work. When a target must be renamed or removed, first deploy code that stops creating old work, drain or migrate the old work, then deploy the generated program without that target.