The filesystem connector
The filesystem connector watches a directory and submits each matching file to a flow. It is a
separate process from the platform, configured from the same .flow.md files, so a flow’s trigger
lives next to the flow it triggers.
The connector submits each file to POST /flows/{tenant}/{flow}/enqueue. A flow fed by it that
declares routes: must also declare serve_by_name: true, or it is refused at publication. See
Served by name.
$ nexus-vfs-connector --server http://localhost:9090 order-intake.flow.mdConfiguring a trigger
A flow declares its trigger in front matter. The connector reads the file, finds the configuration
and starts a worker. A flow is ignored when none of its keys starts with trigger or, before its
first _, is within two letters of it, or when its trigger names another connector. The cases
that stop the connector instead are listed under
What stops the connector at start.
The connector reads its keys as the platform reads the front matter, comments included:
trigger_path: /in # the inbox is the directory /in (see
Comments).
---flowmarkdown_version: "0.1"flow: order-intaketenant: acmetrigger: vfstrigger_path: /var/spool/orders/intrigger_done_path: /var/spool/orders/donetrigger_error_path: /var/spool/orders/errortrigger_pattern: '.*\.xml$'trigger_poll_interval_ms: 5000max_message_bytes: 1048576trigger_move_timestamp: true---| Key | Meaning |
|---|---|
trigger |
Must be vfs, in lower case and with nothing after it, for this connector to pick the flow up |
trigger_path |
Directory to watch |
trigger_done_path |
Where a file goes after it has been submitted. Defaults to done inside trigger_path |
trigger_error_path |
Where a file goes on failure. Optional |
trigger_pattern |
Regular expression matched against the file name |
trigger_extensions |
The endings a file name must have, as a list: [xml, json]. Optional; without it, every name the pattern matches is accepted |
trigger_poll_interval_ms |
How often the directory is scanned, in milliseconds. At least 100; 5000 when absent |
max_message_bytes |
Largest file accepted. trigger_max_body_bytes is a deprecated alias, at least 1 |
trigger_move_timestamp |
true to append a timestamp to the file name when moving it; false when absent |
trigger_pattern is a regular expression, not a shell glob. .*\.xml$ matches XML files; *.xml
does not do what you expect. To select by ending, trigger_extensions is simpler; see
Which files are picked up.
trigger_path, trigger_done_path and trigger_error_path hold literal text and
{{ ctx.KEY }} of a key the flow declares in
config:. The connector reads each such
key from NEXUS_CONFIG_<KEY> when it starts, once, and puts the value in its place:
config: [INBOX]trigger_path: "{{ ctx.INBOX }}/orders"nexus validate and nexus deploy refuse, naming the key, any other {{ }} in these keys
(trigger_path: only {{ ctx.<KEY> }} of a key declared in config: may be written in a connector key)
and a key config: does not declare
(trigger_path: {{ ctx.INBOX }} names a key that config: does not declare). An environment variable
written directly in one of these keys is refused at publication too; see
What a flow may read.
Three cases stop the connector before it scans anything, each naming the key and the variable,
never a value: `trigger_path` cannot be resolved: NEXUS_CONFIG_INBOX is not set, … is empty,
and … holds {{, which cannot be substituted. A value read from the variable is not read
again. nexus serve checks the same variable at its own start, as for every config: key.
The connector reads the file with the platform’s own parser, so it strips quotes around a value the
way the platform does. It refuses to start on a file the platform cannot parse when one of its
trigger keys names vfs, in any case, and on a file whose trigger is vfs and whose tenant or
flow breaks
the rule for a name that becomes a path.
Any other file the platform cannot parse is ignored, like a flow whose trigger names another
connector. A file the platform parses but nexus validate refuses for another reason, such as a
step that writes ctx.correlation_id, does not stop it: the connector starts its worker like any
other.
A key written more than once is read in file order: every occurrence must be valid, so a
malformed one stops the connector, naming the key, whichever side of a valid one it is on; when all
are valid, the last stands.
Rows an earlier release stored in the local store under such a tenant or flow are not
submitted at start, and every start logs one ERROR per pair, whose remedy says what to do: a
file still in the watch directory is sent by the scan under the names the flow file declares, and
deleting those rows from pending_files ends the line.
What stops the connector at start
The connector reads its flow files once, when it starts, and stops before it scans anything at the first key it cannot read as written. The error names the file’s key:
- A key that starts with
trigger, or that is within two letters of it (Trigger_path,triger_pattern), and is not one of the keys above: “trigger_paternis not a key of the vfs connector; its keys aretrigger, …”, followed by “did you meantrigger_pattern?” when one is that close. The value is not repeated. - Such keys without a
trigger:key: “the file hasTrigger,trigger_pathbut notrigger:key”. triggerempty, in another case (VFS), with a space or tab and more aftervfs(vfs extra), or quoted with blanks around it (" vfs"): “triggeris empty”, or “triggermust be exactlyvfs, in lower case, with nothing after it”. The first word oftriggerends at the first space or tab; a first word other thanvfsin any case,vfs2included, names another connector, and the file is ignored with a line atdebug, which shows only whenRUST_LOGasks for it (RUST_LOG=debug); withoutRUST_LOG, or with it empty, the connector’s own lines show frominfoup.- A value that is not of its kind. A number must be a whole number at or above its minimum:
trigger_poll_interval_ms: 5sgives “trigger_poll_interval_msis “5s”: it must be an integer of milliseconds, at least 100”.trigger_move_timestampmust betrueorfalse, and a text key must not be empty. Nothing is rounded up to the minimum, and there is no maximum. - A path with a
..component, such as/data/../in(a name with two dots inside it, such as/in/a..b, is accepted), or a destination written astrigger_pathitself: “trigger_done_pathcontains..; write the path without it”, or “trigger_done_pathis the watched directory itself; a destination must be another directory”. - A
trigger_extensionsit cannot read; see Which files are picked up.
One file stopped stops the whole process, with every flow it serves, other tenants’ included.
nexus validate and nexus deploy refuse a flow whose trigger is not exactly jms or vfs:
trigger: vsf gives “trigger names no connector: it must be one of jms, vfs; did you
mean vfs?”, and the hint comes only when a name is within two letters of the value. A typo
in the name of another trigger key passes nexus deploy and shows at the connector’s next
restart. Restart it after changing a trigger key, and read its first lines.
Connector options
$ nexus-vfs-connector \ --server http://localhost:9090 \ --store /var/lib/nexus/vfs-connector.db \ order-intake.flow.md invoice-intake.flow.md| Flag | Default | Environment variable |
|---|---|---|
--server |
http://localhost:9090 |
NEXUS_SERVER_URL |
--store |
nexus-vfs-connector.db |
— |
--api-key |
none | NEXUS_API_KEY |
| (positional) | required | one or more .flow.md paths |
Each flow file gets its own worker, so one process can serve several directories.
The store is a local SQLite database used for crash recovery. Give it a stable path — a store in
the working directory means recovery state depends on where you launched the process from.
If the connector cannot read the files its store holds in flight at start, it logs an ERROR,
submits none of them, and starts: files still in the watch directory are picked up by the scan,
and the records stay for the next start.
How a file is processed
On each poll, the connector lists the directory, keeps the regular files whose names it accepts (see Which files are picked up), and sorts them by modification time. Files are then processed oldest first. A listing that fails part of the way is one failed scan: the connector says so once, with the directory and the error, and forgets nothing it knew.
For each file:
- Compute its identity and check whether it has already been processed.
- If it is over the size limit, record it as refused, move it to
trigger_error_pathif one is configured, and skip it. The file is never read into memory, and it is never filed underdone. - Record the submission in the store.
- Report a
connector_receivedevent to the platform. - Submit the content to the flow’s queued endpoint.
- Move the file to
trigger_done_path, under a name not taken there (see Where a file is moved). - Mark it done in the store.
The content type is chosen from the file extension, whatever its case (ORDER.XML is sent as XML), and the platform reads application/xml as
XML, text/csv as text and every other content type as JSON:
| Extension | Content type sent | How the platform reads the file |
|---|---|---|
.xml |
application/xml |
As XML |
.json |
application/json |
As JSON |
.csv |
text/csv |
As text: the file reaches the flow as a string, as sent, a byte-order mark included. A file that is not UTF-8 is refused with 400 |
| anything else | application/octet-stream |
As JSON, so anything that is not JSON is refused with 400 |
Point trigger_pattern at .xml, .json or .csv files. A file refused with 400 is set aside, not
retried; see Permanent refusals below.
Submission goes to the queued endpoint, and an XML file arrives at the flow whole: the queue keeps
the document’s text and parses it at delivery, so the flow reads the same $ it would on /run,
root element, attributes and repeated siblings included. See
JSON and XML.
Which files are picked up
The connector decides on each name in the listing before it looks at the file, in this order:
- A name that starts with
.is hidden and is never picked up, with no line in the log. Neithertrigger_patternnortrigger_extensionsbrings it back, and there is no setting that does. The one hidden name that gets a line is the connector’s own.nexus-<uuid>.del, the uuid in lower case; see Where a file is moved. - With
trigger_extensions, the name must end with.and one of the entries, compared without regard to case, and must be longer than that ending:a.XMLmatchesxml;a.tar.gzmatchesgzandtar.gz;a.xml.partdoes not matchxml;READMEanda.match nothing. - The name must match
trigger_pattern. - A name that is not valid UTF-8, and that items 2 and 3 accept (its invalid bytes read as
�), is not sent and not moved. It is named once aterror, with “rename it to UTF-8 to have it sent”, and left in place; renamed, it is sent. Any other such name is passed over without a line.
When both keys are present, both must accept the name. An ending selects by name and proves
nothing about the content: evil.exe.xml ends in .xml. Thumbs.db and desktop.ini are not
hidden; leave them out with trigger_extensions or the pattern.
trigger_extensions is written [xml, json] on one line, or xml, json without brackets.
Entries are read in lower case, and quoted entries go inside brackets: ["xml", "json"]. The
connector stops at start, naming the key, on a list it cannot read:
| Written | Error |
|---|---|
[] |
an empty list accepts nothing; remove the key to accept every extension |
[.xml] |
.xml: write xml |
[*.xml] |
*.xml: an extension is not a pattern; use trigger_pattern |
An entry is ASCII letters, digits, _, + and -, with single dots between them (tar.gz);
any other entry is refused the same way, with the allowed form.
The management UI does not edit trigger_extensions. In the VS Code Visual extension, Flow
Settings shows it under Other keys as a text field that holds the value as the file writes it
and saves it back as typed; the VS Code LSP extension offers no completion for it. Neither
extension checks the value: the connector does, when it starts (see
What stops the connector at start and
the VS Code extensions).
What is never sent
Only a regular file is sent. When a name passes the filter above and the entry is a symbolic
link (to a file, to a directory, dangling or a loop), a FIFO, a socket or a device, it is not
followed, not sent, not moved and not removed. The connector names it once at error, with its
path, its kind and, for a link, the link’s text (cut at 256 bytes, with its full length), and says
“left in place”. It stays until you remove it. A subdirectory is skipped without a line.
A regular file with a second hard link is not sent while it has one, since another name could
still change it. After 12 consecutive scans an error names it, saying that the file waits until
the second name is removed; once it is back to one name, the next scan sends it. A publisher that writes a file, links it under its final name and removes the
first name keeps working. A setup that hard-links incoming files into an archive does not: those
files are no longer sent.
A file the connector cannot read is named once at warn, with the kind of error, and is sent
when it becomes readable. A file that changes between the listing and the read is taken at the
next scan; one that still looks changed after 12 consecutive scans is named once at warn. On a
network mount with an attribute cache (NFS), or a cifs mount with noserverino, a file can look
changed for good: it is never sent, and that line names it.
“Once” means once per process: a restart names every such entry again. Each scan writes at most 20 lines about entries; the rest are counted in one line, “and N more entries over this scan’s 20 lines, not named one by one”.
trigger_path itself may be a link to a directory: it is followed, and the rule applies to the
entries inside it. Two things only you can secure: every directory above the watched directory
must be under your control, not a partner’s, and the connector must run under its own user, never
under a partner’s account. Without both, none of the protections on this page holds.
Where a file is moved
A move never replaces a file. When the name is taken in trigger_done_path or
trigger_error_path, the file gets the next free name, with a counter before the last extension:
| Taken | Next names |
|---|---|
a.json |
a-1.json, a-2.json, … |
a_20260929T101112.json (with trigger_move_timestamp) |
a_20260929T101112-1.json |
a.tar.gz |
a.tar-1.gz |
README |
README-1 |
A name is kept within 255 bytes by shortening the part taken from the original name. After
9,999, nothing more is moved under that name and one error says “… already holds 10,000 files
named …; clean it or turn on trigger_move_timestamp”.
When the destination is on another filesystem, the file is copied into a temporary
.nexus-<id>.tmp in the destination and then given its name. The copy keeps the original’s read
and write permissions, at most rw-r--r--, less the connector’s umask: never execute, setuid or
write for others. A crash during the copy leaves a .nexus-*.tmp behind; it is safe to delete.
A crash in the middle of a move can also leave the file under both names: the one in the watch
directory then has a second hard link and is not sent again; remove it.
The destinations, trigger_done_path and trigger_error_path, must be writable only by the
connector’s user and by operators you trust. Someone who can write there can already delete any file
in it, and the connector’s few remaining deletes by name in a destination, made after a comparison,
rely on that.
When the move puts the file in the destination by a hard link or the copy above, the one left in the
watch directory is first renamed to a hidden .nexus-<uuid>.del, checked to be the file that was
sent, and only then deleted. If that file cannot be checked or renamed aside, the name made in the
destination is removed only while the watch directory still holds that same file, and the move is
tried again every hour. If another file has taken its name, or it is gone, the file in the
destination is kept and the move is done. If the connector cannot check which file the name holds,
the file in the destination is kept as well, and the error names it: “… it is kept in … as …”.
Each hourly retry that still finds the file in the watch directory then puts it in the destination
under one more name, with the same bytes. After a hard link, the file kept in the destination is the
file itself, not a copy: while the watch directory still holds it, the two names are one
file, and a change written into the one in the watch directory shows in the destination. After the
copy, it is a separate file. A file another program puts at the same name at that moment is never
deleted: on a filesystem with a rename that refuses to replace a name, it is put back under its name
and taken at the next scan. If its name has been taken again by then, or putting it back fails, it
stays under the hidden name, and one error says “… it was not sent; rename it to a name the flow
picks up to have it sent”. On a filesystem without such a rename, it is never put back: it stays
under the hidden name, and one error says “… it was not sent, and it is not put back, because no
rename that refuses to replace a name is available here; rename it to a name the flow picks up to
have it sent”. A stop between the rename and the delete also leaves a .nexus-<uuid>.del behind,
and so does a delete that fails, which one error names at once: “… is the file moved from … into
…, where it is …; its name in the watched directory could not be removed; delete …”, and a hidden
name the connector cannot check, which one error also names at once: “… was set aside from …
while the file moved from there into …, where it is …, was being removed, and whether it is that
file cannot be checked; before deleting anything, compare its bytes with … in …: if they are the
same, delete …; otherwise rename … to a name the flow picks up to have it sent”. The connector
never sends, moves or deletes such a file; the first scan that lists it names it once at error,
unless the failed delete or check already did: “… was left while a moved file was being removed;
before deleting anything, compare its bytes with the files in …: if one of them has the same bytes,
delete … and keep that one; otherwise rename … to a name the flow picks up to have it sent”, naming
the hidden file and the destinations you configured.
A destination on a filesystem that offers neither hard links nor a rename that refuses to replace
(some FUSE and network mounts) is halted: the file is not moved, and nothing more is delivered for
that flow while the connector runs. One error says “put the destination on a filesystem with
hard links or rename-noreplace; a destination on a local disk is reached through the copy”, again
every hour. Every scan tries the destination again, and “delivery resumed” at info says when it
works. The halt is kept in memory only: the first scan after a restart delivers at most one file
before the halt is found again.
Other cases the connector names:
- A file replaced in the watch directory after it was sent and before the move: “sent, then replaced before it could be moved; the entry now at … was not moved and is taken at the next scan”.
- A link put in the destination in place of the moved file is removed, with an
error. A directory put there is moved back, or left with “a directory put at … after the post was moved into …; it is not the connector’s; remove it”. - A file that reaches the destination but is not the one sent, because another file took its name during the move, is left there with “moved to done/error but not the file delivered; it was not sent; move it back into the watched directory to send it”.
- A watched directory with the sticky bit, where neither the directory nor the file belongs to the connector’s user: “the watched directory is sticky and neither it nor the file is the connector’s: it may not remove the file”. The move is tried again every hour.
- A destination with a directory on its path that cannot be checked for a reason other than not
existing, such as a directory below one the connector may not search: “the destination cannot be
used, so nothing is delivered at this scan”, with “part of its path, … cannot be checked: …”.
Nothing is delivered for that flow until it can be checked; the line comes again every hour, and
“delivery resumed” at
infosays when it works. - A destination whose path does not reach the flow’s own watched directory before any other
flow’s, while the watched directory of another flow the same process serves cannot be checked
for a reason other than not existing. This holds for a destination outside every watched
directory, and for one whose path reaches the watched directory of another flow first, whether
it ends there or goes on below it. The directory that cannot be checked could lie on the
destination’s path under another path, where it would not be recognised as watched, so a link
put in it would be followed rather than refused. The destination is not used, and nothing is
delivered for that flow: “the destination cannot be used, so nothing is delivered at this scan”,
with “the watched directory of another flow, …, cannot be checked: …”, naming that directory and
its error, or, when several cannot be checked, “the watched directories of other flows cannot be
checked: …: …; …: …”, naming each with its error. The line comes again every hour and as soon as
the directories that cannot be checked change, and “delivery resumed” at
infosays when it works. A destination whose path reaches the flow’s own watched directory before any other flow’s is used as usual. A watched directory that does not exist stops no other flow.
A destination inside the watched directory
The default trigger_done_path, done inside trigger_path, is inside the watched directory, and
so is any destination that lies below it, or below the watched directory of another flow the same
process serves, however its path is written. A destination written through a link that leads into
a watched directory, this flow’s or another flow’s, or anywhere below one, is inside that
directory: the connector reads the links on the path itself and sees every directory it crosses.
The one exception is a directory below a watched directory that a bind mount, or a second mount of
the same share, also shows at another path: written through that other path, it is not recognised
as inside, so write such a destination through the watched directory. A destination may be hidden
(trigger_done_path: /data/in/.done), which keeps it out of the listing.
The connector opens such a destination from the watched directory it lies in, one directory at a
time, never following a link, and creates a missing one with mode 0755. Each directory on the way
must be a directory, not a link, owned by the connector’s user or by root, and not writable by
others. Otherwise one error names the key, the path and the reason, and nothing is delivered for
that flow; the line comes again every hour, and “delivery resumed” at info says when it opens.
When such a directory is replaced between two scans, one warn says “the directory at … is not the
one used at the last scan”, and delivery goes on.
One more destination is refused in the same way, with its reason: .. in a link’s text, once the
path has reached a watched directory: “part of its path below the watched directory … is ..,
which is not walked”.
Three refusals apply to every destination, inside a watched directory or not, and name their
reason in one error the same way:
- a path through more than 40 links: “its path crosses more than 40 links”;
- a relative destination while the connector’s working directory lies below a watched directory: “it is relative, and the working directory lies below the watched directory …”;
- a relative destination while a directory above the connector’s working directory cannot be checked for a reason other than not existing: “an ancestor of the working directory, …, cannot be checked: …”, naming that directory by its full path, or, when the working directory’s own path cannot be read, by its place from it, with that error.
On a filesystem without real ownership, such as cifs without unix extensions, these checks mean nothing: keep the destinations outside the watched directory there.
File identity, and what it means for duplicates
A file is identified by path:mtime:size, the mtime with its nanoseconds. All three participate,
which has consequences worth knowing:
- The same file re-created with different content is a different identity, so it is processed again. That is usually what you want for a spool directory.
- A file touched without a content change gets a new mtime, so it is a new identity and is processed again.
- Two files with the same name in different directories are distinct.
- Two different files that happen to share a path, mtime and size cannot occur, since the path is unique at a point in time.
- At start, a file recorded as in flight is submitted again only if the file at its path still
has the recorded identity. If it is missing, the record is dropped and nothing is sent; should
the same file appear later, for example once a network share is mounted, the scan submits it.
If something else is now at the path, the record is dropped with a
warnnaming both identities, and the scan takes what is there: a regular file is submitted once; a link, FIFO, socket or device is refused, as under What is never sent. If its identity cannot be read, or the file now has a second hard link or changes while it is read, nothing is sent and the record is kept for the next start.
The mtime is taken with its nanoseconds where the filesystem keeps them, and in whole seconds where it does not. On such a filesystem a file replaced within the same second, at the same path and the same size, is not distinguishable from the original and will be skipped. Files recorded by an earlier release, whose identity has seconds only, are still recognised by it, with the same limit, until they leave the watch directory.
The delivery guarantee
At least once. A message can be delivered twice, and there is a specific window where it will be.
Look at the order above: the file is submitted (step 5), then moved (step 6), then marked done
(step 7). A crash between 5 and 6 leaves the store’s record pending with the file still in place,
so on restart replay_pending submits it again. A crash between 6 and 7 leaves the record pending
with the file already moved: replay finds nothing at its path, drops the record and sends nothing.
| Crash point | Result on restart |
|---|---|
| Before step 3 | Nothing recorded; the file is still in the watch directory and is processed normally |
| Between 3 and 5 | Recorded as pending; resubmitted. May be a first delivery or a duplicate — the connector cannot tell whether the submission reached the platform |
| Between 5 and 6 | Recorded as pending, file still in place; resubmitted — a duplicate |
| Between 6 and 7 | Recorded as pending, file already moved; the record is dropped and nothing is sent. Put the file back in the watch directory and it is submitted again — a duplicate |
| After 7 | Nothing to do |
Two cases produce a duplicate without a crash. A file sent but not moved, for example while its
destination is halted (see Where a file is moved), loses its record at
the first restart more than 24 hours after it was sent, and is sent once more. And the platform
accepted the file, but the local store could not record it: if the file is no longer in the watch directory, it is not sent again. If it
is, the next scan sends it again; the scan stops at that file, so it costs one duplicate per poll
interval until the store or the done directory is fixed. Either way an error line says which.
So flows fed by this connector must tolerate the same file arriving twice. Derive any downstream
idempotency key from the file’s content or from a field inside it — not from uuid(), which
produces a new value on every attempt.
This is the same guarantee the queue itself gives, so a flow written to be safe under queue redelivery is already safe here.
Permanent refusals
The connector retries a failed submission — but only where retrying can work. Network failures,
429 and 5xx are retried; 400, 401, 403 and 413 are final answers and end the attempt.
| Status | What it means | What happens to the file |
|---|---|---|
401, 403 |
The key is wrong, revoked, expired, or not allowed on this flow or tenant | The file is left exactly where it is and the worker stops. Fix the key, restart, and the file is submitted then. |
400 |
The platform reads a file named .xml as XML, a .csv file as text and every other file as JSON, so a file of another extension that does not hold JSON is refused, and so is a .csv file that is not UTF-8. Otherwise the JSON or XML is malformed, the XML is nested more than 100 elements deep, or the JSON is nested too deeply to be queued. Point trigger_pattern at .xml, .json or .csv files |
Moved to trigger_error_path if configured, and recorded as refused either way. The worker carries on with the next file. |
413 |
The file is over the platform’s ceiling for this flow | Moved to trigger_error_path if configured, and recorded as refused either way. The worker carries on with the next file. |
429 |
Rate limited | Waits for Retry-After when the server sends one (seconds form, capped at 5 minutes), otherwise backs off, then retries |
5xx, network |
The platform is down or restarting | Backs off and retries |
Stopping the retry loop is only half of it. A refused file stays in the watch directory, so the next
poll would pick it up again and start over. That is why a refusal is also recorded: with no error
directory configured, the record in the connector’s store is the only thing that keeps the file out
of later scans. Every permanent refusal is logged at error with the status, tenant, flow and what
to do about it — a file that can be neither sent nor moved must not pass silently. When a refused
file can be neither moved nor recorded, every scan that still finds it at its path picks it up and
refuses it again, and each error line says so; if the error directory is halted, that starts once
it resumes. A file that a hard link also put in the error directory is the exception: while it has
both names, no scan picks it up, and its error says so. When the file is no longer at its path,
the error says so, and no scan picks it up again.
When trigger_error_path is configured but the move into it fails and the file is not put there,
the refused file is recorded and is not sent again either; a later scan that finds it in the watch
directory moves it there. The error says “it was not moved into …; the refusal is recorded and it
is not sent again”, then when the move is tried again, by what stopped it:
- “its move is tried again in an hour”, for a failure repeated every hour, such as a failed copy;
- “its move is tried again when the destination resumes”, for a halted destination;
- “its move is tried again once … changes”, for a destination that already holds 10,000 files of that name;
- “its move is tried again at each scan that finds it at …”, for any other failure;
- “it is no longer at …, so its move is not tried again”, when another file, or none, holds its name.
Each line carries only its own ending.
When the refused file was put in the error directory but its name in the watch directory could not
be removed, nor checked, the error says where it is: “it was put in … as …, but it could not be
removed from … and whether it is still there cannot be checked; the refusal is recorded and it is
not sent again; its move is tried again in an hour, and a retry that finds it still at … puts it in
… under one more name”, or, when the refusal could not be recorded either, “… and the local store
could not record the refusal — every scan picks it up and refuses it again until the store or the
error directory is fixed” after the copy, and “… and the local store could not record the refusal —
while it is at …, … and … are two names of one file, which no scan sends or moves; if … has the
same bytes as …, delete …” after a hard link.
To send a refused file again, correct it (or, after a 413, raise the ceiling) and put it back in
the watch directory. The refusal is recorded against the file’s identity, path:mtime:size, so a
file moved back unchanged under its original name, with mv, keeps its mtime and is skipped for
good. touch it after moving it back, or copy it back with cp, which gives it a new mtime; either
way it is a new identity and is sent again.
A refused file never goes to trigger_done_path. That directory is what an operator reads as
“delivered”.
404 is still retried indefinitely, on purpose: a flow that is not deployed yet, or is disabled,
and a suspended tenant all answer it, and each can pass without anyone touching the file.
A flow withdrawn with nexus undeploy answers 404 too, so the connector keeps retrying and the
file stays at its source until the flow is deployed again or the connector is stopped.
Reporting
The connector reports two event kinds to the platform, both carrying a correlation ID so a file can be followed from arrival to execution:
| Event | Meaning |
|---|---|
connector_received |
A file was accepted and is being submitted |
connector_oversized |
A file exceeded max_message_bytes and was skipped |
Follow one file through the structured log by its correlation ID.
In the connector’s own log, every value that comes from the files it reads (a file’s path, a
link’s text, the name a file was moved to) is written in quotes, with a line feed as \n, a
carriage return as \r and a byte that is not UTF-8 as \xNN, so one name stays on one line.
The paths and names you configured are written as they are.
The connector writes its log to stderr, never to stdout. It adds colour codes only when stderr is
a terminal and NO_COLOR is unset or empty, so a redirected log stays plain text. A RUST_LOG
that is not valid Unicode is read as unset: the connector’s own lines show from info up, and
nothing says so.
A complete setup
$ mkdir -p /var/spool/orders/{in,done,error}order-intake.flow.md:
---flowmarkdown_version: "0.1"flow: order-intaketenant: acmeeffects: [http_egress]trigger: vfstrigger_path: /var/spool/orders/intrigger_done_path: /var/spool/orders/donetrigger_error_path: /var/spool/orders/errortrigger_pattern: '.*\.xml$'trigger_poll_interval_ms: 5000max_message_bytes: 1048576trigger_move_timestamp: truequeue_max_attempts: 5---
Picks up order files from the spool directory and forwards them.
## Step: check```validate$.orderId != null | "orderId is required" | syntactic```
## Step: forwardeffects: [http_egress]endpoint: https://orders.example.com/ingestmethod: POSTcontent_type: application/json$ nexus tenant create --id acme --display-name "Acme Ltd"$ nexus deploy order-intake.flow.md --version 1.0.0$ nexus keys create --tenant acme --label vfs-connector \ --scopes '*' --flows order-intakecreated API key id=1 key: nxk_4f19c0b27ad3489e93f5c1e6a70d2b84
$ nexus serve --port 9090 &$ export NEXUS_API_KEY=nxk_4f19c0b27ad3489e93f5c1e6a70d2b84$ nexus-vfs-connector --server http://localhost:9090 \ --store /var/lib/nexus/vfs.db \ order-intake.flow.md &$ cp order-A1.xml /var/spool/orders/in/The connector authenticates like any other caller — there is no connector-shaped exemption. Give it
its own key, scoped with --flows to the flow it submits to. Without one it gets 401 on its first
submission and stops rather than retrying: a credential refusal is terminal, and the file stays where
it is.
Within one poll interval the file is submitted, moved to done with a timestamp appended, and the
flow runs.
Not supported
The connector runs on Linux, macOS and other unix systems only. On any other system it exits at start with an error saying so.
There is no watch based on filesystem notifications — the directory is polled. Subdirectories are not traversed. A file that fails processing is not retried by the connector; retries are the queue’s job, and the dead-letter queue is where an exhausted message ends up.
Writing files: the connector reads the directory, moves each accepted file to
trigger_done_path, and moves each file refused with 400 or 413 to trigger_error_path when
one is configured; without one, the refused file stays where it is, recorded as refused. Nothing in
the platform writes a file on a flow’s behalf, and no step puts a file anywhere.