Server and configuration
How to start the server, where it keeps its state, what it reads from the environment, how scheduled flows fire, and how to put a new version live without stopping anything.
Starting the server
$ nexus serve --port 9090NexusFabric server starting on 0.0.0.0:9090 db: /home/you/.nexus/registry.db ui db: /home/you/.nexus/nexus-ui.db audit db: /home/you/.nexus/nexus-audit.db log db: /home/you/.nexus/nexus-log.db queue dir: /home/you/.nexus/queues webhook signature verification: disabledOne process serves everything: the flow endpoints, the WSDL endpoint, the health check, the queue workers, the cron scheduler, and the management UI. Start the same binary again against the same state directory and you have a second engine — the two coordinate through file locks, with no orchestrator between them. See Running more than one engine.
| Flag | Default | Meaning |
|---|---|---|
--host <addr> |
0.0.0.0 |
Address to bind. |
--port <n>, -p |
9090 |
Port to bind. |
--db <path> |
~/.nexus/registry.db |
Registry database. Its parent directory becomes the state directory — every other file is placed relative to it. |
--grpc-port <n> |
off | Open the gRPC door on a separate port, h2c (plaintext). Without the flag nothing listens, and a flow deployed with inbound: leaves a WARN naming it. 0 asks the system for a port, announced as NexusFabric gRPC ingress listening on <addr>. See gRPC. |
--metrics-port <n> |
off | Serve metrics on a separate port that answers only GET /metrics, with no authentication; its body names your tenants, flows and certificates. 0 asks the system for a port, announced as NexusFabric metrics listening on <addr>. Bind it on a private interface or let only the collector reach it. See Metrics. |
--no-auth |
off | Serve without API key authentication. An unregistered or disabled tenant still answers 404, as with the gate closed. Local development only — the boot line reads DISABLED (--no-auth) and a warning goes to the log on every start. |
--trusted-proxy <ip-or-cidr> |
none | A peer allowed to assert a client certificate in the Client-Cert header: an IPv4 or IPv6 address or CIDR range, repeatable. A value that does not parse stops the start and is named. Without the flag every Client-Cert is refused with 401, and a tenant that accepts client_cert leaves a WARN naming it. The list is logged at start as an INFO line, after the listening lines. See Client certificates. |
--accept-new-state-key |
off | Start over flow state written under another NEXUS_SECRET_NEXUS_STATE_KEY and record the new key. Refused on any start that has nothing to accept. |
The shared secret for X-Nexus-Signature verification is not a flag: it is read from
NEXUS_WEBHOOK_SECRET in the service’s environment (EnvironmentFile, .env). When set, every
/run and /enqueue request must carry a valid signature; set and empty, or not valid UTF-8, the
server refuses to start. See Authentication.
--secret <key> is refused: nexus serve exits without starting, because a value on the command line is readable by every user of the host in the process list.
With a state key set, the boot output says what the start did with it, under the flow-state line:
state key: recorded on the first start with a key, and
state key: replaced (<n> deduplication rows and <m> correlations no longer match) when another
key’s check value is replaced. A start under the recorded key prints neither. A start under another
key while rows are stored is refused, unless --accept-new-state-key is given; that start then
writes a state.key_replaced audit entry on the nexus chain before it replaces the key (see
the platform’s own events). Neither the key nor a check value
is printed. See the state key at install time.
API keys are required unless you pass --no-auth, so the boot output ends with
API key authentication: enabled and a further warning if the registry holds no keys at all. See
Authentication.
There is no TLS option. Terminate TLS in front of the process. A tenant that identifies its callers by
client certificate gets them from the proxy, which verifies the certificate and forwards it in
Client-Cert; the engine honours that header only from a peer named by --trusted-proxy. The front
then has four obligations: every proxy listener that forwards to an engine sets or clears
Client-Cert; a trusted address shared with other processes makes each of them the proxy; the mTLS
listener verifies with ssl_verify_client on, never optional_no_ca; and revocation (CRL or OCSP)
is checked by the proxy, since nexus tenant client-cert revoke withdraws only the registration. See
The gateway boundary.
The message-size ceiling is 1 MiB by default and is resolved per access point, not per process:
max_message_bytes in system_config, overridden by a flow’s front matter, overridden by an
http_egress step for that call. With the gate on, an unauthenticated caller announcing an
oversized body is refused with 401 before the ceiling is even consulted. See Limits.
Where state lives
Everything is derived from the parent directory of --db. With the default, that is ~/.nexus.
| Path | Holds |
|---|---|
registry.db |
Compiled flow artifacts (as content-addressed blobs), the version registry and its refs, API keys, protobuf descriptors, platform settings. |
nexus-audit.db |
The hash-chained audit trail, one chain per tenant. |
nexus-log.db |
The structured log — one row per event. |
nexus-ui.db |
Management UI users, sessions and preferences. |
logs/nexus-log-YYYY-MM-DD.<host>-<pid>.jsonl |
The structured log again, as JSON Lines, one file per day and per engine process. |
queues/{tenant}/{flow}/ |
Write-ahead queue files for one flow: its segment.<16 hex digits>.log files (the number is the byte offset at which each starts), its checkpoint, its retry schedule, its message-id counter (segment.log.seq, which keeps counting across a drain so an id is never reissued), and dlq.log with its marks file. |
The databases are ordinary SQLite files. They are opened with synchronous=FULL and a rollback
journal rather than WAL mode, so the state directory can live on a shared filesystem. If two
server processes share one state directory, they coordinate through lock files: only one runs
the cron schedules, and only one runs the workers for any given queue.
Back up the state directory as a unit. The four databases reference each other only by tenant and flow name, but a partial restore will leave the audit chain and the log describing artifacts the registry no longer has.
Back it up with sqlite3 .backup, not with cp — a file copy taken while the server runs can
catch a database in the middle of a transaction and gives a file that looks intact and is not.
The script, the restore procedure and the checks that prove it worked are on
Backup and restore.
Environment variables
| Variable | Read by | Meaning |
|---|---|---|
NEXUS_SECRET_<KEY> |
The secret_read effect |
The value of the secret named <KEY>. See below. |
NEXUS_CONFIG_<KEY> |
serve, config check |
The value of the configuration key <KEY> declared by a flow’s config:. See below. |
RUST_LOG |
Every command | Filter for the operational log the process writes to stderr. Default info. |
NEXUS_UI_ADMIN_USER |
serve |
Username for the first management UI account. |
NEXUS_UI_ADMIN_PASS |
serve |
Password for that account, at least 15 characters, not an example value from this manual. A shorter one stops the start. |
HOME (or USERPROFILE on Windows) is used only to compute the default database path.
What a flow may read
A flow reads from the server environment only NEXUS_CONFIG_
A flow whose endpoint: names an environment variable directly in the text the author wrote is
refused by nexus validate and nexus deploy, and nothing is stored:
step 'send': endpoint: ${…} is not read from the environment; declare the key in config: and write {{ ctx.<KEY> }}, or read a credential with secret_readThe same refusal applies to bearer_token and the five oauth2_token keys, and to the connector
keys trigger_connection, trigger_login, trigger_path, trigger_done_path and
trigger_error_path, where the message has no step '…': prefix. A value a {{ }} produced at
run time is sent as it is and never reads the environment.
To give a flow a value, declare it in config: and set NEXUS_CONFIG_<KEY>; see
How configuration is named. To give it a credential, use
secret_read; see How secrets are named.
How secrets are named
A step with the secret_read effect names a secret with key:; if it has no key:, the step
name is used with - and . replaced by _. The variable name is that key, uppercased, with
NEXUS_SECRET_ in front.
## Step: load-api-tokeneffects: [secret_read]key: downstream_tokenThat reads NEXUS_SECRET_DOWNSTREAM_TOKEN and makes the value available to later steps as
ctx.downstream_token. A missing variable fails the step. The variable name never appears in
the response or in the log — only the key does.
How configuration is named
A flow declares the environment configuration it needs
(config:), and each name is fed by
NEXUS_CONFIG_<KEY>:
config: [BILLING_URL]
## Step: forwardeffects: [http_egress]endpoint: "{{ ctx.BILLING_URL }}/submissions"That reads NEXUS_CONFIG_BILLING_URL. Same source as secrets, different notion: an endpoint is not a
secret, so it is not masked in the log and it costs no step.
The differences from secrets that matter operationally:
-
A missing value stops the server, at start-up, naming the tenant, the flow, the key and the variable — not at the first message that needs it. Ask
nexus config checkfor the same verdict without starting anything. -
A registry that cannot be listed also stops the server. The check has nothing to check against, and starting anyway would mean starting on a gate that verified nothing. An individual artifact that cannot be read does not stop the boot: it is reported, at
ERROR, naming the flow — the server cannot say what that flow requires, so it does not pretend it checked. -
A flow published with the old
ctx.Xspelling on a credential key also stops the server. Since 2026-09-16bearer_token:and the fiveoauth2_tokenkeys are{{ ctx.X }}templates and the bare form is no longer read; an artifact that still carries it would send nothing where a credential was meant. The refusal names the tenant, the flow, the step and the key — never the value — and asks for the flow to be rewritten and republished.nexus config checkgives the same verdict. A literal published before keeps working. -
A flow published with an environment variable written in a step key also stops the server. Such an artifact would send the text as written. The refusal names the tenant, the flow, the step and the key, never the value:
these deployed flows hold a written ${…}, which is not read from the environment since ADR-0337, so the step would send it as written: acme/orders step 'send' key `endpoint`. Run `nexus config check` to see the whole list; rewrite each ${…} as a config: key read with {{ ctx.KEY }}, or as secret_read for a credential, and republish the flownexus config checklists each such key on a line marked!and exits non-zero. See What a flow may read. -
The values are captured once, at start-up, and checked then. A secret is read on every execution instead, but from the process environment, which does not change after start-up. Either way, a new value requires a restart.
Set-but-empty is not the same as absent. An empty prefix is a legitimate value: it starts the
server and is reported at INFO. An absent one is a refusal.
The first UI account
If nexus-ui.db has no users and both NEXUS_UI_ADMIN_USER and NEXUS_UI_ADMIN_PASS are set
at startup, that account is created with the sysadmin role. Once any user exists the two
variables are ignored, so remove them after the first boot. If they are not set and there are
no users, the UI has no way to log in and says so in the operational log.
The password must have at least 15 characters. With no users and a shorter
NEXUS_UI_ADMIN_PASS, nexus serve stops before it listens, with a non-zero exit and an error that
names the variable and the minimum, never the value. The same minimum holds for every UI password;
see Management UI sign-in.
An example password from this manual, such as the .env value below, is refused the same way:
nexus serve stops before it listens and names the variable, never the value.
.env
Every nexus command except nexus mcp loads a .env file before doing anything else. Only
.env in the working directory is read, never one in a parent directory.
The file is read only when all of these hold:
- it is a regular file, not a symbolic link, a directory or a pipe;
- it is owned by the account that runs
nexus, or by root; - neither the file nor the working directory can be written by group or others. A sticky bit on the directory does not change this.
Otherwise the command stops before doing anything, with exit status 1 and one line on stderr that
starts .env refused:, names the file and the condition, and prints no value from it. The same
happens when a line of the file cannot be parsed, where the message gives its line number, and
when a value holds a NUL byte, where it names the variable. The fix is
in the message: chmod go-w on the file or the directory, chown to the running account, or
removing the file. A missing .env is not an error.
A .env does not override variables already present in the environment — a real environment
variable always wins. It holds secrets, so give it mode 0600.
RUST_LOG=info,nexus_http=debugNEXUS_SECRET_DOWNSTREAM_TOKEN=s3cr3tNEXUS_CONFIG_BILLING_URL=https://billing.example.comNEXUS_UI_ADMIN_USER=adminNEXUS_UI_ADMIN_PASS=change-me-before-first-bootBecause .env is read before the log subscriber is installed, RUST_LOG set there takes
effect.
Capacity settings
These live in the system_config table of the registry database, not in the environment, because
they belong to the deployment rather than to the process that happens to be running. Most are read at startup, so changing one takes effect on the next restart; the ones marked below
as read per request, per cycle or per message are the exceptions.
| Key | Default | What it limits |
|---|---|---|
max_concurrent_flow_executions |
32 per core, capped at 256 | Flow executions running at once, across all tenants, detached fork branches included. Past it, /run answers 503 with Retry-After, and a fork branch is not started (branch_refused in the log) |
max_queue_workers |
4 per core, capped at 64 | Queue workers running at once, across all tenants |
max_queue_workers_per_tenant |
equal to max_queue_workers |
One tenant’s share of the above |
flow_max_duration_secs |
7200 |
Wall time of one execution, checked between steps |
flow_lease_ttl_secs |
equal to flow_max_duration_secs, minimum 1 |
How long one claim on a dedup_key: key lasts without being renewed. The delivery renews it between steps once a third of it has passed; a delivery that could not confirm its claim for two thirds of it stops before its next step and answers 503. A process killed mid-delivery holds its key this long; nexus state release frees it sooner. When NEXUS_SECRET_NEXUS_STATE_KEY is set, a value that is not a plain number, or below 1, is refused with a message naming the key and the value, by nexus serve at start and by nexus config check |
flow_state_retention_hours |
96, minimum 1 |
How long a dedup_key: receipt is kept by default, and how long a closed or timed-out correlation stays past its deadline before the purge deletes it — see The flow-state window. When NEXUS_SECRET_NEXUS_STATE_KEY is set, a value that is not a plain number, or below 1, is refused with a message naming the key and the value, by nexus serve at start and by nexus config check |
dedup_max_response_bytes |
65536 |
Largest response a dedup_key: flow keeps for replay. A larger one is not kept, and a duplicate of that delivery cannot be answered with it — see Deduplication. When NEXUS_SECRET_NEXUS_STATE_KEY is set, a value that is not a plain number is refused with a message naming the key and the value, by nexus serve at start and by nexus config check |
correlation_state_max_bytes |
65536 |
Largest serialised correlation_state: one open correlation may hold; an open over it is refused — see Correlation. When NEXUS_SECRET_NEXUS_STATE_KEY is set, a value that is not a plain number is refused with a message naming the key and the value, by nexus serve at start and by nexus config check |
fanout_max_parallel |
10, from 1 to 256 |
Branches of one split step running at once; a longer array runs in waves. A blank, non-numeric, zero or above-256 value is refused with a message naming the key, the value and the range, by nexus serve at start and by nexus config check. Not on the Settings page |
max_message_bytes |
1048576 |
Default ceiling on one message body. Read per request, not at startup, and overridable per flow and per egress step |
queue_segment_max_bytes |
134217728 (128 MiB), minimum 1048576 |
Size at which a queue’s active segment is sealed and the next one started. A blank, non-numeric or below-minimum value is refused with a message naming the key, the value and the minimum, by nexus serve at start, by nexus config check and by the Settings page; it is never replaced by the default. Read once, at start: a changed value applies at the next start |
queue_retention_hours |
none: absent or blank means no term; minimum 1 |
The longest a queued message may wait, in whole hours. A flow’s own queue_retention_hours: overrides it. 0 or a value that is not a whole number of hours is refused with a message naming the key, the value and the minimum, by nexus serve at start, by nexus config check and by the Settings page. Checked at start; the queue worker also reads it again for every message, so a change applies to the next message without a restart, and a refused value it finds there applies no term and is logged once. A message older than the term is not delivered: it moves to the DLQ with the reason expired, and stays there until it is replayed or discarded. |
log_retention_days |
30, minimum 1 |
How long log rows and the daily JSON Lines files are kept. Read at the start of each retention cycle |
oidc_jwks_ttl_secs |
300, minimum 30 |
How long an OIDC issuer’s signing keys are used before the platform asks the issuer again — see When the issuer is down. Below 30, or not a plain number, it is refused with a message naming the key and the value, by nexus serve at start and by nexus config check |
$ sqlite3 ~/.nexus/registry.db \ "INSERT INTO system_config (key, value) VALUES ('max_queue_workers_per_tenant', '8') ON CONFLICT(key) DO UPDATE SET value = excluded.value;"A key whose row says a value is refused stops nexus serve at start and fails
nexus config check. For every other key, a value that cannot be parsed falls back to the default
rather than removing the limit. Apart from queue_retention_hours, whose absence means no term,
there is no way to express “unlimited”: the defaults exist because an unbounded server accumulates
work instead of refusing it, and a caller waiting behind an invisible queue is worse off than one
told to come back.
A flow’s own limit
A flow that declares max_concurrent_executions: (see
The flow file) runs at most that many executions at once on one
engine, every door counted together, inside max_concurrent_flow_executions. The limit is per
engine: N engines run up to N times the number. Over it:
/runand a declared route answer503withRetry-After: 1and the body{"ok":false,"error":"flow concurrency limit reached"}, after the access gate and before the body is read. A caller without valid credentials still gets401.- The gRPC door answers
resource_exhaustedwith the messageflow concurrency limit reachedand the trailersretry-after: 1andgrpc-retry-pushback-ms: 1000. - A cron tick is not run.
- A queued message stays in its queue until a slot frees, and it takes the next slot that frees:
while it waits,
/run, declared routes and the gRPC door are refused one slot earlier, so the queue is not starved by a door that keeps the flow busy. No attempt is counted and nothing goes to the DLQ because of the limit; the worker logs oneWARNwhen the wait starts. A message that passes its retention term while it waits goes to the DLQ as expired, not to the flow.
Each refusal writes a handoff_refused event whose error is flow concurrency limit <n> reached.
A flow fed only by its queue already runs one execution at a time on an engine, since one worker
drains it.
One worker per queue, and it never stops
A queue worker drains one (tenant, flow) queue and holds its thread for the life of the
process. So max_queue_workers is not a rate — it is the number of distinct queues that can drain
on this server at all. Reach it, and a queue that does not already have a worker will not get one
until you restart. Its messages stay durable and unprocessed, and /enqueue keeps answering 202.
Two things follow, and both are worth planning for rather than discovering:
- Count your queues, not your traffic. A deployment with 50 queued flows needs
max_queue_workersabove 50, however quiet they are. max_queue_workers_per_tenantis inert until you set it. It defaults to the global ceiling, so out of the box one tenant can take every worker slot and leave nothing for the others — permanently, since a slot comes back only when its worker ends, and a healthy worker runs until the server stops. The default is deliberate: a fraction would have silently cost queues to every single-tenant install on upgrade. Startup logs a line saying the quota is inert, so it is not mistaken for a protection that is switched on.
A tenant that has reached its quota is refused even when the server has room. That is the point: whether your worker starts depends only on what your tenant has deployed, not on what the neighbours happen to be doing at that moment.
A worker that ends is started again. A flow that panics while it delivers a queued message does
not end the worker: the attempt fails with the error the flow panicked (the panic is in the server log), the message is retried and moves to the DLQ at its queue_max_attempts like any failed
message, and the messages behind it go on. The server logs the panic in an ERROR a flow panicked while delivering a queued message; …. A worker that cannot open its queue or its dead-letter queue,
or whose own code panics, ends and gives its slot back. The server logs an ERROR queue worker could not start; its messages stay queued and it is started again at most once every 5 s (or
queue worker panicked; …) with the tenant, the flow and the cause, once a minute while the fault
lasts. Its next pass starts the worker again, no sooner than 5 seconds after the last start. The
messages stay in the queue meanwhile. Until the worker runs again, /enqueue to that flow still
answers 202 and writes a handoff_refused event that names the cause, and a queue_publish step
to it fails. A worker whose own code panics 5 times in a row, less than a minute apart, is not
started again until the server restarts: a delivery not yet acknowledged would run again at every
start. The ERROR queue worker panicked 5 times in a row; … says so.
Scheduled flows
A flow with a schedule: key in its front matter runs on a timer.
---flowmarkdown_version: "0.1"flow: pull-meter-readingstenant: acmeschedule: "0 */15 * * * *"effects: [http_egress]---
## Step: fetcheffects: [http_egress]endpoint: https://meters.example.com/readingsmethod: GETThe cron format
Six fields, not the usual five. Seconds come first.
sec min hour day-of-month month day-of-week 0 30 * * * *| Expression | Fires |
|---|---|
"0 30 * * * *" |
Every hour at minute 30, second 0. |
"*/10 * * * * *" |
Every ten seconds. |
"0 */15 * * * *" |
Every fifteen minutes. |
"0 0 2 * * *" |
Every day at 02:00:00. |
A five-field expression is not valid here and the flow will not be scheduled. Quote the value —
it contains spaces and *.
What a tick passes in
The flow is invoked with a single-field object:
{ "triggered_at": "2026-08-06T09:15:00.123456789+00:00" }Read it as $.triggered_at. Nothing else is supplied: there is no request, no headers, and no
correlation ID from a caller.
A scheduled run has no HTTP response, so its output is discarded. Whatever the flow is for must happen in an effect — an HTTP call, a queued submission — not in the value the last step returns.
When changes take effect
The schedule list is read once, at startup, from the latest artifact of every flow. Each tick
resolves latest again before running, so changing a flow’s steps takes effect on the next
tick, but adding, changing or removing a schedule: takes effect only after a restart.
An unparseable cron expression is skipped with a warning in the operational log; the rest of the flows are scheduled normally.
Deploying a new version
nexus deploy compiles the file, stores the artifact, registers it under (tenant, flow, version), and moves that flow’s latest pointer to it. The tenant in the file’s tenant: must
already exist — register it once with nexus tenant create, since deploy refuses an unknown or
disabled one rather than creating it.
$ nexus deploy forward-order.flow.md --version 1.3.0deployed forward-order.flow.md flow: forward-order tenant: acme version: 1.3.0 hash: 9f2c... db: /home/you/.nexus/registry.dbYou can run this while the server is running. Every request resolves latest at call time, so
the next request after the deploy uses the new artifact. Requests already in flight finish
against the artifact they started with. No restart, no connection drop, no queue drain — with
one exception: a change to schedule: needs a restart, as above.
Point --db at the same database the server was started with. If you started the server with a
non-default --db, pass the same path to deploy.
A version is claimed once. Deploying a version number that already exists for that flow fails and changes nothing:
$ nexus deploy forward-order.flow.md --version 1.3.0error: registering acme/forward-order v1.3.0: artifact already exists: tenant=acme name=forward-order version=1.3.0Compile before you deploy if you want the errors without the side effects — nexus validate
runs fewer checks than nexus deploy does. See Command line.
Rolling back
Deploy the previous source again under a new version number. That moves latest back to
equivalent bytes:
$ nexus deploy forward-order-1.2.0.flow.md --version 1.4.0Undeploying a flow
nexus undeploy --tenant acme --flow forward-order takes a flow out of service. From the next
call, on every engine of the installation, every door answers as for a flow that was never
deployed. An execution already running finishes. The Registry page of the management UI offers
the same action to a sysadmin. See Command line for the flags and the output.
The cron. The next tick is skipped, not failed. One delivery_refused event with the reason
flow_not_published is written for the run of skipped ticks, not one per tick. Deploying the flow
again resumes the ticks without a restart.
The queue. Messages already queued for the flow, retries included, move to its dead-letter
queue with the reason missing_flow. To deliver them, deploy the flow again and replay them with
nexus dlq replay.
What stays. Nothing is deleted:
- the versions and their artifact bytes;
- the dead-letter queue;
- the dedup and correlation state;
- the logs and the audit chain;
- certificates and proto descriptors;
- environment variables;
- API key and client certificate allowlists that name the flow. They grant it again if it is deployed again.
The refusal. While another deployed flow of the same tenant names this one in queue: or
timeout_flow:, the command refuses and names each of them. Undeploy those first.
Connectors. A JMS or VFS connector feeding the flow gets 404 and keeps retrying. The message
stays at its source until the flow is deployed again or the connector is stopped.
Deploying again needs a new version number, as any deploy does: a published version is fixed.
How versions resolve
| What | Resolves to |
|---|---|
POST /flows/{tenant}/{flow}/run |
The latest artifact, looked up per request. |
POST /flows/{tenant}/{flow}/enqueue |
The queue stores only the message. The worker resolves latest when it picks the message up — which may be a newer version than was live at submission time. |
GET /flows/{tenant}/{flow} |
Metadata of the latest artifact: version, hash, deployment time. |
GET /flows/{tenant}/{flow}/wsdl |
Generated from the latest artifact. |
| A cron tick | The latest artifact, resolved at each tick. |
Older versions are never deleted. They stay in the registry with their artifact bytes intact,
and the Registry page of the management UI lists them. The HTTP endpoints always run latest;
there is no per-version URL.
Artifacts are addressed by the hash of their own bytes, and that hash is verified on every read. If a stored artifact no longer matches its key, the request fails with 500 rather than executing something that changed underneath you.
Log and queue housekeeping
Six background tasks run inside the server.
| Task | Interval | What it does |
|---|---|---|
| Log retention | Every 6 hours, and once at startup | Deletes flow_log rows from nexus-log.db and the daily logs/nexus-log-*.jsonl files older than the retention window. Default 30 days, minimum 1. |
| Queue compaction | Every hour, configurable | Deletes the queue files of any (tenant, flow) whose messages have all been acknowledged and which has no pending retries, no unhandled dead letters, and no running worker. The directory stays; the next submission recreates the files. A queue that does have a worker reclaims its own space instead, in place — see Queues and delivery. |
| Correlation reconciliation | Every 60 seconds, first pass immediately | Marks correlations past their deadline timed_out and, where one is declared, enqueues the compensating flow with the recorded state as its message. |
| Queue adoption | Every 5 seconds, first pass at startup | Starts the worker of any queue that holds undelivered messages, whether they were left by a restart, a stopped engine or a replay. The worker moves a message older than its term to the DLQ (see Queues). |
| Flow-state purge | Every 6 hours, and once at startup | Deletes flow_inbox rows past their own expires_at and closed / timed-out flow_correlations rows more than flow_state_retention_hours (default 96, minimum 1) past their deadline. See below. |
| Certificate and API key expiry sweep | Every cert_check_interval_hours, and once at startup |
Emits cert_expiring / cert_expired for registered certificates and api_key_expiring / api_key_expired for non-revoked API keys inside the warning window. cert_expiry_warning_days is the one window for both, and cert_check_enabled turns both off. |
Log retention and queue compaction are configured from the Settings page of the management UI, and both read their settings at the start of each cycle — a change applies at the next cycle, without a restart. That page also has buttons to run either immediately.
After a compaction run the page says how many queues were compacted and how many were skipped, because a worker or undelivered messages still use them or because they are already empty. When a queue, or a directory of the queue tree, could not be read or compacted, the page shows a warning with their number, and the system log names each one with its error. A queue whose deletion stopped part way keeps its messages: the next start delivers them again.
The flow-state window
flow_state_retention_hours (a system_config value, default 96) is read once, at startup —
unlike the two above — so a change to it takes effect only after a restart. The boot banner prints
the value in force: flow state (dedup + correlation): enabled — retention 96h.
The two tables it purges are cut differently:
flow_inbox(deduplication): each row carries its ownexpires_at, fixed at claim time from the window then in force — the flow’sdedup_window_hoursif it set one, otherwise the installation default. Loweringflow_state_retention_hoursand restarting does not shorten receipts already written; the new value applies to later claims only.flow_correlations: a closed, timed-out, or late-closed row is deleted once its deadline is more thanflow_state_retention_hoursin the past — cut retroactively, and measured from the deadline, not from when the row closed.
A correlation key is one-shot for that whole window. correlation_open: refuses to reuse a key
that still has a row — open, closed, or timed_out alike — because a late or duplicate reply
could still arrive on it. The key frees only after its row is purged: deadline + flow_state_retention_hours,
and then at the next purge pass. For a test loop, send a fresh key each time (a distinct
X-Correlation-ID, or none or an invalid one, and no valid X-Request-ID, so the platform
generates one). A stuck row can also be cleared by hand with nexus state release.
What log retention covers
One setting governs both outputs of the log. Until it did, the half that was never cleaned was
the half that holds plaintext payloads at log_level: full — the rows were pruned on schedule while
the files grew without limit.
The two edges differ deliberately. Rows are compared against an instant; files against a date, because a file covers a whole day, so the one dated exactly on the boundary is kept. Files therefore live up to a day longer than the equivalent rows. That is the safe direction.
Three rules protect the files you did not ask us to touch:
- The current day’s file is never removed. The writer holds it open, and on Unix unlinking it would send every subsequent line into an invisible inode.
- A file with a name that is not ours is left alone.
- A file that is ours but carries no parsable date is reported, not deleted.
Neither half can stop the other: if row deletion fails, the files are still swept, and the UI reports the failure as a warning instead of showing you a success built from the other half’s numbers.
Health
$ curl http://localhost:9090/health{"ok":true}Unauthenticated and free of database access — it answers as long as the process is listening.