Skip to content
NexusFabric

Core concepts

The execution model, in one page: what a flow is, what travels through it, and what happens between the file on your disk and a response on the wire. Read this once and the rest of the documentation is reference.

A flow

A flow is one .flow.md file. It has front matter — a block of key: value lines between two --- delimiters — and a sequence of ## Step: sections. Nothing else is required.

---
flowmarkdown_version: "0.1"
flow: forward-order
tenant: acme
---


## Step: shape
```ntd
{ "id": "{{ $.orderId }}", "at": "{{ date_format(now()) }}" }
```

A flow is identified by the pair (tenant, flow). That pair is its address everywhere: in the registry, in the log, in the audit trail, and in its URL — POST /flows/acme/forward-order/run.

Tenants

Every flow belongs to exactly one tenant, declared in the file and not overridable at deploy time. The tenant is the isolation boundary:

  • Two tenants may each have a flow named forward-order; they are unrelated artifacts.
  • An API key is issued for one tenant and cannot call another tenant’s flows.
  • Log entries, audit entries and queue files are partitioned by tenant.

A tenant is registered with nexus tenant create before any of its flows is deployed; nexus deploy refuses a flow whose tenant is not registered.

The message

One value travels through the flow. It is called ctx.input, and in expressions you write it as $. It starts as the request body and ends as the response body.

It is a value, not bytes. A JSON request becomes an object; an XML request becomes an XML value that keeps elements, attributes and order. See JSON and XML.

Each step may replace it. That is the central rule — a step does not “add to” the message, it returns a new one.

Step Effect on the message
ntd or xslt fence Replaced by the value the template produced
http_egress Replaced by the response body of the call
grpc_egress Replaced by the response message
validate, condition, route Unchanged — these decide, they do not transform
No fence, no effect Unchanged
restore_body: on the step Replaced by a copy saved earlier

So a step that calls a downstream system loses the request you sent it. If you need the original afterwards, save a copy first — see save_body.

Variables

Alongside the message there is a set of named variables, ctx.vars. You read one as ctx.<name> in any expression. A variable that was never set reads as null rather than failing.

Variables are not part of the message. They survive every step, including steps that replace the message, and they are not returned to the caller.

Some are set by the platform on every synchronous run:

Variable Contents
ctx.flow The flow name
ctx.correlation_id The value of the X-Correlation-ID or X-Request-ID request header if it is valid, or a generated one. The same value is returned to the caller in the X-Correlation-ID response header of every answer (where the caller finds it)
ctx.headers The request headers, as an object with lowercase names

ctx.headers is filled only on the synchronous path — a queued, scheduled or connector-driven run has no request to take headers from, and there the variable is an empty object, so reading a header off it gives null. Nothing downstream treats that as a blank: a headers: value built on it omits the header with a warning, and a dedup_key built on it is refused on every message. ctx.tenant is set on every path, but it is the same value for all of a flow’s traffic.

Others are written by steps as they run — save_body: writes the message into a variable of your choosing, a secret_read step writes the secret, an http_egress step writes the response status into ctx.HTTP_SC. Each is documented with the feature that produces it.

Compile, deploy, execute

Three separate acts.

Terminal window
$ nexus tenant create --id acme --display-name "Acme Ltd"   # once per tenant
$ nexus validate forward-order.flow.md
$ nexus compile  forward-order.flow.md --pretty
$ nexus deploy   forward-order.flow.md --version 1.0.0

Compiling parses the file and lowers it into an intermediate representation — a JSON document holding the steps, their declared effects, and every expression already turned into a tree. All the errors that can be found without running anything are found here: an unknown effect, a malformed expression, a duplicate step name, a timeout of zero, a route pointing at a step that does not exist.

validate runs the parse and the semantic checks but stops before lowering, so a handful of errors — a bad timeout value, a malformed template in an endpoint: — only surface under compile. If you want certainty before you deploy, run compile.

Deploying compiles, then stores the result and registers it under a version you choose. It refuses a tenant: that has not been registered with nexus tenant create, and refuses one that has been disabled. Creating the tenant implicitly would mean a typo in tenant: silently becomes a real tenant, with its own keys, its own audit chain, and no way to tell it from a new customer.

Executing loads the stored artifact and runs its steps in order. No parsing happens at request time; the source file is not consulted and does not need to exist on the server.

What stops a started execution

What can end an execution early acts between steps: it keeps the next step from starting. A step that has started runs to its end, bounded only by its own time budgets (below) and by the life of the process that runs it. In a foreach or a split, that holds for each step of the body, not for the whole fence.

Apart from a step that fails, two things end an execution there.

Stop When Answer on JSON
The execution ceiling The execution has run longer than flow_max_duration_secs, 2 h by default 500 with {"ok":false,"error":"flow exceeded its maximum duration","step":"...","limit_secs":N}, where step is the step that did not start. A flow with a ## Fault: section runs it instead, as for a failed step, with a ceiling of its own (Deadlines and timeouts)
A lost claim Only on a flow with dedup_key:. An operator released the key with nexus state release, or the platform could not confirm the claim for two thirds of flow_lease_ttl_secs 503 with Retry-After: 1 and {"ok":false,"error":"flow execution stopped before it finished; some of its steps may have run","step":"..."}. The ## Fault: section does not run, or stops before its next step if it was running, and nothing is recorded for the key (A lost claim skips the handler)

On SOAP and gRPC, see Without a handler.

Both stops are judged before each step, the steps of a foreach body and of a ## Fault: section included. A split branch judges the ceiling before each of its steps but holds no claim, so a lost claim does not reach a wave of branches that has started, and stops the split before its next wave. A branch started by fork has a ceiling of its own and holds no claim: neither stop of the flow that started it reaches it.

The caller is not one of them. A caller that closes the connection, a caller whose own timeout expires, a proxy or gateway that cuts the request short — none of that reaches the flow. It keeps running and produces the effect of every step it runs: each outbound HTTP or gRPC call, each message handed to another system, and whatever those systems do as a result.

So a timeout is not a “did not happen”. A caller that gave up knows one thing only: no answer arrived in time. It does not know whether the work never started, stopped halfway, or finished correctly a moment later.

Three things follow.

Consequence What it means
A retry may be a duplicate Treat the abandoned attempt as having succeeded, not failed, and make the retry safe to repeat. See Retrying safely.
The response is lost, the record is not A synchronous run returns its output to the caller and there is no route to read it afterwards, so a closed connection loses it for good. That the execution ran, and how it ended, is still recorded under its correlation ID — see Logging and audit.
Nothing is rolled back A flow is not a transaction. Steps that already ran keep their effects, and there is no compensation unless you wrote one yourself as further steps.

What actually bounds an execution

Most time budgets are per step. One is not.

Bound Default Where
HTTP egress, establishing the connection 10 s connect_timeout_ms on the step
HTTP egress, the whole request 30 s read_timeout_ms on the step
gRPC, establishing the connection 10 s connect_timeout_ms on the step
gRPC, the whole call including a collected stream 30 s deadline_ms on the step
The whole execution 2 h flow_max_duration_secs, set server-side

The execution ceiling is one of the two stops above: judged between steps, it limits how many further steps begin, not how long one step runs.

timeout_ms on a split fence is checked once the branches have finished rather than interrupting them, so a fan-out can overshoot its own budget and report it afterwards.

Retrying safely

The server deduplicates only what a flow asks it to. Without dedup_key: in the front matter, two identical requests are two executions, on /run and on /enqueue alike. Declare it and the platform claims the key before the flow runs — on /run, on a queued message when the worker picks it up, on the gRPC door and on a cron tick alike, because the claim sits in the delivery bracket all four share. While the key is remembered, a second delivery of it does not run once an earlier execution of the key finished without failing: /run and the gRPC door answer from the record, the worker acknowledges the message, and the cron tick is skipped. While that execution still holds its claim on the key, the second delivery is held back instead. After a failure it runs again, unless the flow sets dedup_on_failure: absorb. The keys, the verdicts and the replay header are in The flow file.

That is the platform’s half. It bounds repeats of the same delivery; it says nothing about the same business operation arriving twice by different routes, and it does not make the downstream system idempotent. Outside the server, the JMS connector suppresses a broker redelivery of a message-id it has already forwarded. Beyond that, suppressing a duplicate is the job of the system that owns the data, and the flow’s job is to give it something to suppress on.

As a caller. Pick a key that identifies the business operation — an order number, an invoice id, a UUID you minted before the first attempt — and send the same key on every retry. A fresh key per attempt is how one timeout becomes two orders. Sending it as X-Correlation-ID also puts it on every log entry for the execution, which is how you find out afterwards what the first attempt did. That holds for a key of 1 to 128 characters of A-Z a-z 0-9 . _ : -. A key outside that rule, one with a space or a / for example, is replaced by a generated id, in the log and in ctx.correlation_id. A flow called through /run still reads the header in ctx.headers['x-correlation-id'], unless the value holds a non-ASCII byte: such a header is left out of ctx.headers.

In the flow. Forward that key to the system that has to absorb the repeat.

---
flowmarkdown_version: "0.1"
flow: order-intake
tenant: acme
effects: [http_egress]
---


## Step: check
```validate
$.orderId != null                      | "orderId is required"         | syntactic
ctx.headers["idempotency-key"] != null | "Idempotency-Key is required" | syntactic
```


## Step: forward
effects: [http_egress]
endpoint: https://orders.example.com/orders
method: POST
read_timeout_ms: 5000
headers: {"Idempotency-Key": "{{ ctx.headers['idempotency-key'] }}", "X-Correlation-ID": "{{ ctx.correlation_id }}"}

The validate fence is there because a headers template whose value is absent omits the header rather than sending it empty — with a warning in the server log, but nothing the caller sees: a caller who forgot the key would otherwise get no protection and no error. Make the key part of the contract and say so at the door.

Two rules keep such a key usable:

  • Derive it from the message, never from the clock or a random value. uuid() and now() produce something new on every attempt and defeat the deduplication they were meant to enable; $.orderId, or a header the caller repeats, does not.
  • Prefer effects that are safe to repeat. A PUT that sets a record to a known state survives a second attempt; a POST that appends a row does not.

On /enqueue a timeout tells you even less. The route answers 202 as soon as the message is written and the flow runs afterwards, so a request that failed on the way back may already have a message being processed; submitting again is a second message. Without dedup_key: it is a second execution. With it, the worker claims the key when it reaches that message: while the key is remembered, the message is acknowledged without running once an earlier execution of the key finished without failing, and held back, without spending an attempt, while that execution still holds its claim on the key; after a failure it runs the flow again, unless the flow sets dedup_on_failure: absorb, in which case it is acknowledged without running. Queued messages need the same discipline for a second reason — delivery is at least once. See Queues, retries and the DLQ.

Versions and immutability

The compiled artifact is stored under the hash of its own bytes, and that hash is re-checked every time it is read. An artifact cannot be edited in place, and a deployed version cannot change under you.

Deploying the same (tenant, flow, version) twice is an error:

Terminal window
$ nexus deploy forward-order.flow.md --version 1.0.0
error: registering acme/forward-order v1.0.0: artifact already exists: tenant=acme name=forward-order version=1.0.0

Each deployment also moves the flow’s latest pointer to the new artifact. latest is what the HTTP endpoints serve, so deploying a new version is how you release one. Older versions stay in the registry, byte-for-byte as you deployed them.

Version strings are yours to choose; the platform treats them as opaque labels and does not order or compare them.

Effects are declared

Anything a flow does beyond transforming its own message is an effect: calling an HTTP endpoint, opening a gRPC channel, fetching an OAuth2 token, reading a secret. Effects are listed in an effects: key, and they are listed twice — once on the flow, once on the step.

---
flowmarkdown_version: "0.1"
flow: forward-order
tenant: acme
effects: [http_egress]
---


## Step: forward
effects: [http_egress]
endpoint: https://orders.example.com/ingest
method: POST

The two declarations do different jobs.

On the flow it is a whitelist. A step may not declare an effect the flow has not declared; compilation fails if it does. This is what makes the header of the file an honest summary of the flow’s reach — you do not have to read every step to know whether it touches the network.

On the step it is the permission that is actually enforced. Before a step runs, the executor looks up a handler for each effect the step declares and invokes it. A step that does not declare http_egress has no way to make an HTTP call, whatever else it says: endpoint: and method: on a step with no effects are inert text.

Declaring an effect on the flow alone does nothing at runtime and costs nothing. Declaring one on a step commits you to it: if the server has no handler for that effect, the step fails. See The effect catalog for what is available.

Where to go next