Performance

Throughput and latency belong to an installation, not to the software: they are decided by the flows you publish, the backends they call, the hardware underneath and a handful of settings. This page says what those are and how to measure them, so the numbers you quote are yours and reproducible.

What one delivery costs

A synchronous delivery through /run does this, in order:

1. The gate. Two indexed lookups — the tenant row by primary key, then the key hash — and the size ceiling for this access point, all inside one database step, before a byte of the body is read.
2. The body. Read up to the ceiling, then parsed into the canonical value — JSON, or XML when the content type says so.
3. The steps. Executed in order, on one thread. A split fans out to threads and joins; a fork detaches. Everything else is sequential.
4. The waits. Each http_egress or grpc_egress step is a round trip to somebody else. On any flow that calls out, this is the number.
5. The record. One received and one verdict event onto a bounded channel, written by the logging thread to SQLite and to the day’s JSON Lines file. Plus a dedup or correlation row if the flow declares one.

Two consequences follow. Latency is mostly not ours: on a forwarding flow, the platform’s share is parse, transform and record, and the rest is the backend. And concurrency is across deliveries, not inside one: one delivery uses one thread until it finishes or waits, so the number that matters for throughput is how many deliveries may run at once.

The settings that bound it

Setting	Where	What it bounds
max_concurrent_flow_executions	system_config	Deliveries running at once, across all tenants. Past it, /run answers 503 with Retry-After
max_queue_workers, max_queue_workers_per_tenant	system_config	How many distinct queues can drain on this server — a count of queues, not a rate
flow_max_duration_secs	system_config	Wall time of one execution, checked between steps
max_message_bytes	system_config, frontmatter, egress step	The largest body accepted or sent at one access point
rate_limit_rps	per API key	The caller’s own ceiling, as a token bucket with twice that as burst
log_level, log_payload_max_bytes	flow frontmatter	Whether payloads are written alongside events, and how much of them

All of these are described in Running NexusFabric and Limits. Most are read at startup, so a change takes effect on the next restart; max_message_bytes is read per request.

The two doors measure different things

/run measures the whole integration: the caller waits, the backend’s latency is inside the number, and saturation is visible as 503 with Retry-After.

/enqueue splits it in two. Acceptance is a bounded read plus an append to the flow’s write-ahead log, answered 202; the work happens later, in the worker that drains that queue. So an installation on the async door has two numbers — an acceptance rate and a drain rate — and measuring only the first tells you nothing about whether the queue is keeping up. Read the drain rate from the queue_processed events, or watch the depth on the Queue page.

Measuring an installation

Measure the flows you are going to run. A flow that forwards to a stub answering instantly measures the engine; the same flow against your real backend measures the integration. Both are useful, and they are different numbers — say which one you took.

Fix and record everything that moves the result. A figure without these is not comparable with the next one:

• the product version and the artifact hash of each flow under test;
• cores and memory of the machine, and whether the registry, queue and log databases are on local disk or on a network filesystem;
• the message size and its shape — a 2 KB JSON body and a 2 KB SOAP envelope do not cost the same;
• the concurrency the driver holds, and whether it is open or closed loop;
• the backend’s own latency during the run;
• log_level for each flow, and whether payload logging was on;
• whether the gate was on, and whether the key under test had a rate limit.

Drive it with a real HTTP load tool, giving it the key in the Authorization header. Discard a warm-up: the first calls to a flow pay the artifact read and the connection setup.

Read the result from three places, and expect them to differ:

• the driver, for client-side latency and error rate — this is the number a caller experiences;
• the duration_ms field on each completed event in the structured log, for the execution alone, without transport or queueing at the socket;
• the Stats page of the management UI, for volume, errors, latency and the busiest flows over the whole run.

One rule about errors. A run with transport errors in it is not a slow run, it is a failed run; fix the cause and measure again rather than reporting a rate next to a footnote.

Tuning, in the order worth trying

1. Turn payload logging down. log_level: event_only is the default for a reason: at payload_trimmed or full every delivery writes its body twice, to SQLite and to the JSON Lines file, and the masking pass runs over it first.
2. Raise the concurrency ceiling if 503 appears while the CPU is idle and the backend has room. Leave it alone if the backend is the thing that is full — refusing early is better than queueing inside a caller’s socket.
3. Count your queues. max_queue_workers must exceed the number of distinct (tenant, flow) queues you expect to drain, however quiet each one is, and max_queue_workers_per_tenant is inert until you set it.
4. Check the key’s rate limit. A per-key ceiling caps what you can measure. Remove it for the measurement, and put it back afterwards.
5. Move the wait. A flow whose backend is slow but whose caller does not need the answer belongs on /enqueue, with retry and a dead-letter queue, rather than holding a connection open.
6. Add engines. Several engines over one shared state directory, behind the proxy that already terminates TLS — see High availability.