Skip to content

Server and configuration

How to start the server, where it keeps its state, what it reads from the environment, how scheduled flows fire, and how to put a new version live without stopping anything.

Starting the server

Terminal window
$ nexus serve --port 9090
NexusFabric server starting on 0.0.0.0:9090
db: /home/you/.nexus/registry.db
ui db: /home/you/.nexus/nexus-ui.db
audit db: /home/you/.nexus/nexus-audit.db
log db: /home/you/.nexus/nexus-log.db
queue dir: /home/you/.nexus/queues
webhook signature verification: disabled

One process serves everything: the flow endpoints, the WSDL endpoint, the health check, the queue workers, the cron scheduler, and the management UI. Start the same binary again against the same state directory and you have a second engine — the two coordinate through file locks, with no orchestrator between them. See Running more than one engine.

Flag Default Meaning
--host <addr> 0.0.0.0 Address to bind.
--port <n>, -p 9090 Port to bind.
--db <path> ~/.nexus/registry.db Registry database. Its parent directory becomes the state directory — every other file is placed relative to it.
--grpc-port <n> off Open the gRPC door on a separate port, h2c (plaintext). Without the flag nothing listens, and a flow deployed with inbound: leaves a WARN naming it. 0 asks the system for a port, announced as NexusFabric gRPC ingress listening on <addr>. See gRPC.
--metrics-port <n> off Serve metrics on a separate port that answers only GET /metrics, with no authentication; its body names your tenants, flows and certificates. 0 asks the system for a port, announced as NexusFabric metrics listening on <addr>. Bind it on a private interface or let only the collector reach it. See Metrics.
--no-auth off Serve without API key authentication. An unregistered or disabled tenant still answers 404, as with the gate closed. Local development only — the boot line reads DISABLED (--no-auth) and a warning goes to the log on every start.
--trusted-proxy <ip-or-cidr> none A peer allowed to assert a client certificate in the Client-Cert header: an IPv4 or IPv6 address or CIDR range, repeatable. A value that does not parse stops the start and is named. Without the flag every Client-Cert is refused with 401, and a tenant that accepts client_cert leaves a WARN naming it. The list is logged at start as an INFO line, after the listening lines. See Client certificates.
--accept-new-state-key off Start over flow state written under another NEXUS_SECRET_NEXUS_STATE_KEY and record the new key. Refused on any start that has nothing to accept.

The shared secret for X-Nexus-Signature verification is not a flag: it is read from NEXUS_WEBHOOK_SECRET in the service’s environment (EnvironmentFile, .env). When set, every /run and /enqueue request must carry a valid signature; set and empty, or not valid UTF-8, the server refuses to start. See Authentication. --secret <key> is refused: nexus serve exits without starting, because a value on the command line is readable by every user of the host in the process list.

With a state key set, the boot output says what the start did with it, under the flow-state line: state key: recorded on the first start with a key, and state key: replaced (<n> deduplication rows and <m> correlations no longer match) when another key’s check value is replaced. A start under the recorded key prints neither. A start under another key while rows are stored is refused, unless --accept-new-state-key is given; that start then writes a state.key_replaced audit entry on the nexus chain before it replaces the key (see the platform’s own events). Neither the key nor a check value is printed. See the state key at install time.

API keys are required unless you pass --no-auth, so the boot output ends with API key authentication: enabled and a further warning if the registry holds no keys at all. See Authentication.

There is no TLS option. Terminate TLS in front of the process. A tenant that identifies its callers by client certificate gets them from the proxy, which verifies the certificate and forwards it in Client-Cert; the engine honours that header only from a peer named by --trusted-proxy. The front then has four obligations: every proxy listener that forwards to an engine sets or clears Client-Cert; a trusted address shared with other processes makes each of them the proxy; the mTLS listener verifies with ssl_verify_client on, never optional_no_ca; and revocation (CRL or OCSP) is checked by the proxy, since nexus tenant client-cert revoke withdraws only the registration. See The gateway boundary.

The message-size ceiling is 1 MiB by default and is resolved per access point, not per process: max_message_bytes in system_config, overridden by a flow’s front matter, overridden by an http_egress step for that call. With the gate on, an unauthenticated caller announcing an oversized body is refused with 401 before the ceiling is even consulted. See Limits.

Where state lives

Everything is derived from the parent directory of --db. With the default, that is ~/.nexus.

Path Holds
registry.db Compiled flow artifacts (as content-addressed blobs), the version registry and its refs, API keys, protobuf descriptors, platform settings.
nexus-audit.db The hash-chained audit trail, one chain per tenant.
nexus-log.db The structured log — one row per event.
nexus-ui.db Management UI users, sessions and preferences.
logs/nexus-log-YYYY-MM-DD.<host>-<pid>.jsonl The structured log again, as JSON Lines, one file per day and per engine process.
queues/{tenant}/{flow}/ Write-ahead queue files for one flow: its segment.<16 hex digits>.log files (the number is the byte offset at which each starts), its checkpoint, its retry schedule, its message-id counter (segment.log.seq, which keeps counting across a drain so an id is never reissued), and dlq.log with its marks file.

The databases are ordinary SQLite files. They are opened with synchronous=FULL and a rollback journal rather than WAL mode, so the state directory can live on a shared filesystem. If two server processes share one state directory, they coordinate through lock files: only one runs the cron schedules, and only one runs the workers for any given queue.

Back up the state directory as a unit. The four databases reference each other only by tenant and flow name, but a partial restore will leave the audit chain and the log describing artifacts the registry no longer has.

Back it up with sqlite3 .backup, not with cp — a file copy taken while the server runs can catch a database in the middle of a transaction and gives a file that looks intact and is not. The script, the restore procedure and the checks that prove it worked are on Backup and restore.

Environment variables

Variable Read by Meaning
NEXUS_SECRET_<KEY> The secret_read effect The value of the secret named <KEY>. See below.
NEXUS_CONFIG_<KEY> serve, config check The value of the configuration key <KEY> declared by a flow’s config:. See below.
RUST_LOG Every command Filter for the operational log the process writes to stderr. Default info.
NEXUS_UI_ADMIN_USER serve Username for the first management UI account.
NEXUS_UI_ADMIN_PASS serve Password for that account, at least 15 characters, not an example value from this manual. A shorter one stops the start.

HOME (or USERPROFILE on Windows) is used only to compute the default database path.

What a flow may read

A flow reads from the server environment only NEXUS_CONFIG_ through config: and NEXUS_SECRET_ through secret_read; any other variable is not read, and a ${…} in a flow is refused at publication.

A flow whose endpoint: names an environment variable directly in the text the author wrote is refused by nexus validate and nexus deploy, and nothing is stored:

step 'send': endpoint: ${…} is not read from the environment; declare the key in config: and write {{ ctx.<KEY> }}, or read a credential with secret_read

The same refusal applies to bearer_token and the five oauth2_token keys, and to the connector keys trigger_connection, trigger_login, trigger_path, trigger_done_path and trigger_error_path, where the message has no step '…': prefix. A value a {{ }} produced at run time is sent as it is and never reads the environment.

To give a flow a value, declare it in config: and set NEXUS_CONFIG_<KEY>; see How configuration is named. To give it a credential, use secret_read; see How secrets are named.

How secrets are named

A step with the secret_read effect names a secret with key:; if it has no key:, the step name is used with - and . replaced by _. The variable name is that key, uppercased, with NEXUS_SECRET_ in front.

## Step: load-api-token
effects: [secret_read]
key: downstream_token

That reads NEXUS_SECRET_DOWNSTREAM_TOKEN and makes the value available to later steps as ctx.downstream_token. A missing variable fails the step. The variable name never appears in the response or in the log — only the key does.

How configuration is named

A flow declares the environment configuration it needs (config:), and each name is fed by NEXUS_CONFIG_<KEY>:

config: [BILLING_URL]
## Step: forward
effects: [http_egress]
endpoint: "{{ ctx.BILLING_URL }}/submissions"

That reads NEXUS_CONFIG_BILLING_URL. Same source as secrets, different notion: an endpoint is not a secret, so it is not masked in the log and it costs no step.

The differences from secrets that matter operationally:

  • A missing value stops the server, at start-up, naming the tenant, the flow, the key and the variable — not at the first message that needs it. Ask nexus config check for the same verdict without starting anything.

  • A registry that cannot be listed also stops the server. The check has nothing to check against, and starting anyway would mean starting on a gate that verified nothing. An individual artifact that cannot be read does not stop the boot: it is reported, at ERROR, naming the flow — the server cannot say what that flow requires, so it does not pretend it checked.

  • A flow published with the old ctx.X spelling on a credential key also stops the server. Since 2026-09-16 bearer_token: and the five oauth2_token keys are {{ ctx.X }} templates and the bare form is no longer read; an artifact that still carries it would send nothing where a credential was meant. The refusal names the tenant, the flow, the step and the key — never the value — and asks for the flow to be rewritten and republished. nexus config check gives the same verdict. A literal published before keeps working.

  • A flow published with an environment variable written in a step key also stops the server. Such an artifact would send the text as written. The refusal names the tenant, the flow, the step and the key, never the value:

    these deployed flows hold a written ${…}, which is not read from the environment since ADR-0337, so the step would send it as written: acme/orders step 'send' key `endpoint`. Run `nexus config check` to see the whole list; rewrite each ${…} as a config: key read with {{ ctx.KEY }}, or as secret_read for a credential, and republish the flow

    nexus config check lists each such key on a line marked ! and exits non-zero. See What a flow may read.

  • The values are captured once, at start-up, and checked then. A secret is read on every execution instead, but from the process environment, which does not change after start-up. Either way, a new value requires a restart.

Set-but-empty is not the same as absent. An empty prefix is a legitimate value: it starts the server and is reported at INFO. An absent one is a refusal.

The first UI account

If nexus-ui.db has no users and both NEXUS_UI_ADMIN_USER and NEXUS_UI_ADMIN_PASS are set at startup, that account is created with the sysadmin role. Once any user exists the two variables are ignored, so remove them after the first boot. If they are not set and there are no users, the UI has no way to log in and says so in the operational log.

The password must have at least 15 characters. With no users and a shorter NEXUS_UI_ADMIN_PASS, nexus serve stops before it listens, with a non-zero exit and an error that names the variable and the minimum, never the value. The same minimum holds for every UI password; see Management UI sign-in. An example password from this manual, such as the .env value below, is refused the same way: nexus serve stops before it listens and names the variable, never the value.

.env

Every nexus command except nexus mcp loads a .env file before doing anything else. Only .env in the working directory is read, never one in a parent directory.

The file is read only when all of these hold:

  • it is a regular file, not a symbolic link, a directory or a pipe;
  • it is owned by the account that runs nexus, or by root;
  • neither the file nor the working directory can be written by group or others. A sticky bit on the directory does not change this.

Otherwise the command stops before doing anything, with exit status 1 and one line on stderr that starts .env refused:, names the file and the condition, and prints no value from it. The same happens when a line of the file cannot be parsed, where the message gives its line number, and when a value holds a NUL byte, where it names the variable. The fix is in the message: chmod go-w on the file or the directory, chown to the running account, or removing the file. A missing .env is not an error.

A .env does not override variables already present in the environment — a real environment variable always wins. It holds secrets, so give it mode 0600.

RUST_LOG=info,nexus_http=debug
NEXUS_SECRET_DOWNSTREAM_TOKEN=s3cr3t
NEXUS_CONFIG_BILLING_URL=https://billing.example.com
NEXUS_UI_ADMIN_USER=admin
NEXUS_UI_ADMIN_PASS=change-me-before-first-boot

Because .env is read before the log subscriber is installed, RUST_LOG set there takes effect.

Capacity settings

These live in the system_config table of the registry database, not in the environment, because they belong to the deployment rather than to the process that happens to be running. Most are read at startup, so changing one takes effect on the next restart; the ones marked below as read per request, per cycle or per message are the exceptions.

Key Default What it limits
max_concurrent_flow_executions 32 per core, capped at 256 Flow executions running at once, across all tenants, detached fork branches included. Past it, /run answers 503 with Retry-After, and a fork branch is not started (branch_refused in the log)
max_queue_workers 4 per core, capped at 64 Queue workers running at once, across all tenants
max_queue_workers_per_tenant equal to max_queue_workers One tenant’s share of the above
flow_max_duration_secs 7200 Wall time of one execution, checked between steps
flow_lease_ttl_secs equal to flow_max_duration_secs, minimum 1 How long one claim on a dedup_key: key lasts without being renewed. The delivery renews it between steps once a third of it has passed; a delivery that could not confirm its claim for two thirds of it stops before its next step and answers 503. A process killed mid-delivery holds its key this long; nexus state release frees it sooner. When NEXUS_SECRET_NEXUS_STATE_KEY is set, a value that is not a plain number, or below 1, is refused with a message naming the key and the value, by nexus serve at start and by nexus config check
flow_state_retention_hours 96, minimum 1 How long a dedup_key: receipt is kept by default, and how long a closed or timed-out correlation stays past its deadline before the purge deletes it — see The flow-state window. When NEXUS_SECRET_NEXUS_STATE_KEY is set, a value that is not a plain number, or below 1, is refused with a message naming the key and the value, by nexus serve at start and by nexus config check
dedup_max_response_bytes 65536 Largest response a dedup_key: flow keeps for replay. A larger one is not kept, and a duplicate of that delivery cannot be answered with it — see Deduplication. When NEXUS_SECRET_NEXUS_STATE_KEY is set, a value that is not a plain number is refused with a message naming the key and the value, by nexus serve at start and by nexus config check
correlation_state_max_bytes 65536 Largest serialised correlation_state: one open correlation may hold; an open over it is refused — see Correlation. When NEXUS_SECRET_NEXUS_STATE_KEY is set, a value that is not a plain number is refused with a message naming the key and the value, by nexus serve at start and by nexus config check
fanout_max_parallel 10, from 1 to 256 Branches of one split step running at once; a longer array runs in waves. A blank, non-numeric, zero or above-256 value is refused with a message naming the key, the value and the range, by nexus serve at start and by nexus config check. Not on the Settings page
max_message_bytes 1048576 Default ceiling on one message body. Read per request, not at startup, and overridable per flow and per egress step
queue_segment_max_bytes 134217728 (128 MiB), minimum 1048576 Size at which a queue’s active segment is sealed and the next one started. A blank, non-numeric or below-minimum value is refused with a message naming the key, the value and the minimum, by nexus serve at start, by nexus config check and by the Settings page; it is never replaced by the default. Read once, at start: a changed value applies at the next start
queue_retention_hours none: absent or blank means no term; minimum 1 The longest a queued message may wait, in whole hours. A flow’s own queue_retention_hours: overrides it. 0 or a value that is not a whole number of hours is refused with a message naming the key, the value and the minimum, by nexus serve at start, by nexus config check and by the Settings page. Checked at start; the queue worker also reads it again for every message, so a change applies to the next message without a restart, and a refused value it finds there applies no term and is logged once. A message older than the term is not delivered: it moves to the DLQ with the reason expired, and stays there until it is replayed or discarded.
log_retention_days 30, minimum 1 How long log rows and the daily JSON Lines files are kept. Read at the start of each retention cycle
oidc_jwks_ttl_secs 300, minimum 30 How long an OIDC issuer’s signing keys are used before the platform asks the issuer again — see When the issuer is down. Below 30, or not a plain number, it is refused with a message naming the key and the value, by nexus serve at start and by nexus config check
Terminal window
$ sqlite3 ~/.nexus/registry.db \
"INSERT INTO system_config (key, value) VALUES ('max_queue_workers_per_tenant', '8')
ON CONFLICT(key) DO UPDATE SET value = excluded.value;"

A key whose row says a value is refused stops nexus serve at start and fails nexus config check. For every other key, a value that cannot be parsed falls back to the default rather than removing the limit. Apart from queue_retention_hours, whose absence means no term, there is no way to express “unlimited”: the defaults exist because an unbounded server accumulates work instead of refusing it, and a caller waiting behind an invisible queue is worse off than one told to come back.

A flow’s own limit

A flow that declares max_concurrent_executions: (see The flow file) runs at most that many executions at once on one engine, every door counted together, inside max_concurrent_flow_executions. The limit is per engine: N engines run up to N times the number. Over it:

  • /run and a declared route answer 503 with Retry-After: 1 and the body {"ok":false,"error":"flow concurrency limit reached"}, after the access gate and before the body is read. A caller without valid credentials still gets 401.
  • The gRPC door answers resource_exhausted with the message flow concurrency limit reached and the trailers retry-after: 1 and grpc-retry-pushback-ms: 1000.
  • A cron tick is not run.
  • A queued message stays in its queue until a slot frees, and it takes the next slot that frees: while it waits, /run, declared routes and the gRPC door are refused one slot earlier, so the queue is not starved by a door that keeps the flow busy. No attempt is counted and nothing goes to the DLQ because of the limit; the worker logs one WARN when the wait starts. A message that passes its retention term while it waits goes to the DLQ as expired, not to the flow.

Each refusal writes a handoff_refused event whose error is flow concurrency limit <n> reached. A flow fed only by its queue already runs one execution at a time on an engine, since one worker drains it.

One worker per queue, and it never stops

A queue worker drains one (tenant, flow) queue and holds its thread for the life of the process. So max_queue_workers is not a rate — it is the number of distinct queues that can drain on this server at all. Reach it, and a queue that does not already have a worker will not get one until you restart. Its messages stay durable and unprocessed, and /enqueue keeps answering 202.

Two things follow, and both are worth planning for rather than discovering:

  • Count your queues, not your traffic. A deployment with 50 queued flows needs max_queue_workers above 50, however quiet they are.
  • max_queue_workers_per_tenant is inert until you set it. It defaults to the global ceiling, so out of the box one tenant can take every worker slot and leave nothing for the others — permanently, since a slot comes back only when its worker ends, and a healthy worker runs until the server stops. The default is deliberate: a fraction would have silently cost queues to every single-tenant install on upgrade. Startup logs a line saying the quota is inert, so it is not mistaken for a protection that is switched on.

A tenant that has reached its quota is refused even when the server has room. That is the point: whether your worker starts depends only on what your tenant has deployed, not on what the neighbours happen to be doing at that moment.

A worker that ends is started again. A flow that panics while it delivers a queued message does not end the worker: the attempt fails with the error the flow panicked (the panic is in the server log), the message is retried and moves to the DLQ at its queue_max_attempts like any failed message, and the messages behind it go on. The server logs the panic in an ERROR a flow panicked while delivering a queued message; …. A worker that cannot open its queue or its dead-letter queue, or whose own code panics, ends and gives its slot back. The server logs an ERROR queue worker could not start; its messages stay queued and it is started again at most once every 5 s (or queue worker panicked; …) with the tenant, the flow and the cause, once a minute while the fault lasts. Its next pass starts the worker again, no sooner than 5 seconds after the last start. The messages stay in the queue meanwhile. Until the worker runs again, /enqueue to that flow still answers 202 and writes a handoff_refused event that names the cause, and a queue_publish step to it fails. A worker whose own code panics 5 times in a row, less than a minute apart, is not started again until the server restarts: a delivery not yet acknowledged would run again at every start. The ERROR queue worker panicked 5 times in a row; … says so.

Scheduled flows

A flow with a schedule: key in its front matter runs on a timer.

---
flowmarkdown_version: "0.1"
flow: pull-meter-readings
tenant: acme
schedule: "0 */15 * * * *"
effects: [http_egress]
---
## Step: fetch
effects: [http_egress]
endpoint: https://meters.example.com/readings
method: GET

The cron format

Six fields, not the usual five. Seconds come first.

sec min hour day-of-month month day-of-week
0 30 * * * *
Expression Fires
"0 30 * * * *" Every hour at minute 30, second 0.
"*/10 * * * * *" Every ten seconds.
"0 */15 * * * *" Every fifteen minutes.
"0 0 2 * * *" Every day at 02:00:00.

A five-field expression is not valid here and the flow will not be scheduled. Quote the value — it contains spaces and *.

What a tick passes in

The flow is invoked with a single-field object:

{ "triggered_at": "2026-08-06T09:15:00.123456789+00:00" }

Read it as $.triggered_at. Nothing else is supplied: there is no request, no headers, and no correlation ID from a caller.

A scheduled run has no HTTP response, so its output is discarded. Whatever the flow is for must happen in an effect — an HTTP call, a queued submission — not in the value the last step returns.

When changes take effect

The schedule list is read once, at startup, from the latest artifact of every flow. Each tick resolves latest again before running, so changing a flow’s steps takes effect on the next tick, but adding, changing or removing a schedule: takes effect only after a restart.

An unparseable cron expression is skipped with a warning in the operational log; the rest of the flows are scheduled normally.

Deploying a new version

nexus deploy compiles the file, stores the artifact, registers it under (tenant, flow, version), and moves that flow’s latest pointer to it. The tenant in the file’s tenant: must already exist — register it once with nexus tenant create, since deploy refuses an unknown or disabled one rather than creating it.

Terminal window
$ nexus deploy forward-order.flow.md --version 1.3.0
deployed forward-order.flow.md
flow: forward-order
tenant: acme
version: 1.3.0
hash: 9f2c...
db: /home/you/.nexus/registry.db

You can run this while the server is running. Every request resolves latest at call time, so the next request after the deploy uses the new artifact. Requests already in flight finish against the artifact they started with. No restart, no connection drop, no queue drain — with one exception: a change to schedule: needs a restart, as above.

Point --db at the same database the server was started with. If you started the server with a non-default --db, pass the same path to deploy.

A version is claimed once. Deploying a version number that already exists for that flow fails and changes nothing:

Terminal window
$ nexus deploy forward-order.flow.md --version 1.3.0
error: registering acme/forward-order v1.3.0: artifact already exists: tenant=acme name=forward-order version=1.3.0

Compile before you deploy if you want the errors without the side effects — nexus validate runs fewer checks than nexus deploy does. See Command line.

Rolling back

Deploy the previous source again under a new version number. That moves latest back to equivalent bytes:

Terminal window
$ nexus deploy forward-order-1.2.0.flow.md --version 1.4.0

Undeploying a flow

nexus undeploy --tenant acme --flow forward-order takes a flow out of service. From the next call, on every engine of the installation, every door answers as for a flow that was never deployed. An execution already running finishes. The Registry page of the management UI offers the same action to a sysadmin. See Command line for the flags and the output.

The cron. The next tick is skipped, not failed. One delivery_refused event with the reason flow_not_published is written for the run of skipped ticks, not one per tick. Deploying the flow again resumes the ticks without a restart.

The queue. Messages already queued for the flow, retries included, move to its dead-letter queue with the reason missing_flow. To deliver them, deploy the flow again and replay them with nexus dlq replay.

What stays. Nothing is deleted:

  • the versions and their artifact bytes;
  • the dead-letter queue;
  • the dedup and correlation state;
  • the logs and the audit chain;
  • certificates and proto descriptors;
  • environment variables;
  • API key and client certificate allowlists that name the flow. They grant it again if it is deployed again.

The refusal. While another deployed flow of the same tenant names this one in queue: or timeout_flow:, the command refuses and names each of them. Undeploy those first.

Connectors. A JMS or VFS connector feeding the flow gets 404 and keeps retrying. The message stays at its source until the flow is deployed again or the connector is stopped.

Deploying again needs a new version number, as any deploy does: a published version is fixed.

How versions resolve

What Resolves to
POST /flows/{tenant}/{flow}/run The latest artifact, looked up per request.
POST /flows/{tenant}/{flow}/enqueue The queue stores only the message. The worker resolves latest when it picks the message up — which may be a newer version than was live at submission time.
GET /flows/{tenant}/{flow} Metadata of the latest artifact: version, hash, deployment time.
GET /flows/{tenant}/{flow}/wsdl Generated from the latest artifact.
A cron tick The latest artifact, resolved at each tick.

Older versions are never deleted. They stay in the registry with their artifact bytes intact, and the Registry page of the management UI lists them. The HTTP endpoints always run latest; there is no per-version URL.

Artifacts are addressed by the hash of their own bytes, and that hash is verified on every read. If a stored artifact no longer matches its key, the request fails with 500 rather than executing something that changed underneath you.

Log and queue housekeeping

Six background tasks run inside the server.

Task Interval What it does
Log retention Every 6 hours, and once at startup Deletes flow_log rows from nexus-log.db and the daily logs/nexus-log-*.jsonl files older than the retention window. Default 30 days, minimum 1.
Queue compaction Every hour, configurable Deletes the queue files of any (tenant, flow) whose messages have all been acknowledged and which has no pending retries, no unhandled dead letters, and no running worker. The directory stays; the next submission recreates the files. A queue that does have a worker reclaims its own space instead, in place — see Queues and delivery.
Correlation reconciliation Every 60 seconds, first pass immediately Marks correlations past their deadline timed_out and, where one is declared, enqueues the compensating flow with the recorded state as its message.
Queue adoption Every 5 seconds, first pass at startup Starts the worker of any queue that holds undelivered messages, whether they were left by a restart, a stopped engine or a replay. The worker moves a message older than its term to the DLQ (see Queues).
Flow-state purge Every 6 hours, and once at startup Deletes flow_inbox rows past their own expires_at and closed / timed-out flow_correlations rows more than flow_state_retention_hours (default 96, minimum 1) past their deadline. See below.
Certificate and API key expiry sweep Every cert_check_interval_hours, and once at startup Emits cert_expiring / cert_expired for registered certificates and api_key_expiring / api_key_expired for non-revoked API keys inside the warning window. cert_expiry_warning_days is the one window for both, and cert_check_enabled turns both off.

Log retention and queue compaction are configured from the Settings page of the management UI, and both read their settings at the start of each cycle — a change applies at the next cycle, without a restart. That page also has buttons to run either immediately.

After a compaction run the page says how many queues were compacted and how many were skipped, because a worker or undelivered messages still use them or because they are already empty. When a queue, or a directory of the queue tree, could not be read or compacted, the page shows a warning with their number, and the system log names each one with its error. A queue whose deletion stopped part way keeps its messages: the next start delivers them again.

The flow-state window

flow_state_retention_hours (a system_config value, default 96) is read once, at startup — unlike the two above — so a change to it takes effect only after a restart. The boot banner prints the value in force: flow state (dedup + correlation): enabled — retention 96h.

The two tables it purges are cut differently:

  • flow_inbox (deduplication): each row carries its own expires_at, fixed at claim time from the window then in force — the flow’s dedup_window_hours if it set one, otherwise the installation default. Lowering flow_state_retention_hours and restarting does not shorten receipts already written; the new value applies to later claims only.
  • flow_correlations: a closed, timed-out, or late-closed row is deleted once its deadline is more than flow_state_retention_hours in the past — cut retroactively, and measured from the deadline, not from when the row closed.

A correlation key is one-shot for that whole window. correlation_open: refuses to reuse a key that still has a row — open, closed, or timed_out alike — because a late or duplicate reply could still arrive on it. The key frees only after its row is purged: deadline + flow_state_retention_hours, and then at the next purge pass. For a test loop, send a fresh key each time (a distinct X-Correlation-ID, or none or an invalid one, and no valid X-Request-ID, so the platform generates one). A stuck row can also be cleared by hand with nexus state release.

What log retention covers

One setting governs both outputs of the log. Until it did, the half that was never cleaned was the half that holds plaintext payloads at log_level: full — the rows were pruned on schedule while the files grew without limit.

The two edges differ deliberately. Rows are compared against an instant; files against a date, because a file covers a whole day, so the one dated exactly on the boundary is kept. Files therefore live up to a day longer than the equivalent rows. That is the safe direction.

Three rules protect the files you did not ask us to touch:

  • The current day’s file is never removed. The writer holds it open, and on Unix unlinking it would send every subsequent line into an invisible inode.
  • A file with a name that is not ours is left alone.
  • A file that is ours but carries no parsable date is reported, not deleted.

Neither half can stop the other: if row deletion fails, the files are still swept, and the UI reports the failure as a warning instead of showing you a success built from the other half’s numbers.

Health

Terminal window
$ curl http://localhost:9090/health
{"ok":true}

Unauthenticated and free of database access — it answers as long as the process is listening.