Flow Reference

Flow Reference

A flow is the core automation unit in Xferity. It defines what files to transfer, which partner or endpoint to use, how to protect the payload, when to run, and how to handle failures, retries, and cleanup.

This page is a field-by-field reference. For conceptual guidance, see Automation and Configuration.

Who this page is for

• implementers writing or reviewing flow files
• operators validating flow configuration before rollout
• security reviewers auditing transfer behavior

File structure

Each flow is defined in a YAML file under the directory referenced by flows_path in the global configuration. The YAML root key is flows, containing a map of flow names to flow definitions.

flows:
  my-flow-name:
    direction: upload
    enabled: true
    # ... fields

The flow file name is not required to match the flow name, but keeping them aligned is strongly recommended.

Top-level flow fields

Field	Type	Required	Default	Description
name	string	yes	—	Unique flow identifier. Used by CLI and UI.
direction	string	yes	—	upload or download
enabled	bool	no	true	Whether the flow participates in run-all and scheduled execution.
type	string	no	—	Legacy incoming/outgoing alias. Use direction instead.
schedule_cron	string	no	—	Six-field cron expression (with seconds) for scheduled execution.
max_failures	int	no	0	Number of per-file errors tolerated before a run fails. 0 means fail immediately.
add_timestamp_to_filename	bool	no	false	Append a timestamp to destination file names.
timestamp_format	string	no	—	Go time layout string for the timestamp suffix.

Source and destination

A flow has a source and a target (for upload) or uses inline partner/endpoint references (for download). The Endpoint fields are:

Field	Type	Description
partner	string	Partner ID from the partners directory.
path	string	Remote path or S3 prefix override.
host	string	Inline host override (rarely needed; prefer partner config).
user	string	Inline user override.
password	string	Inline password or secret reference.
local_folder	string	Local folder for this endpoint in the flow.

For most flows, use partner to reference a partner definition and set path if the flow needs a different remote path than the partner default.

Local configuration

The local block sets paths for local staging, landing, and dead-letter locations.

local:
  path: ./storage/nav/incoming
  dead_letter_path: ./storage/nav/dead-letter

Field	Type	Description
path	string	Local base path for the flow.
dead_letter_path	string	Where to place files that failed processing.

Direction semantics

• upload: source is local, target is partner remote.
• download: source is partner remote, destination is local.

Direction also sets PGP role expectations through FlowRoleSpecs():

• upload flows require partner_encrypt and optionally use our_sign
• download flows require our_decrypt and optionally use partner_verify

SFTP configuration (inline)

Flows can reference an SFTP partner by partner name, or include inline SFTP settings. Inline settings override partner values when both are present.

sftp:
  host: sftp.partner.example
  port: 22
  user: transfer-user
  password: env:PARTNER_PASSWORD
  key_path: file:/etc/xferity/keys/partner_id_rsa
  key_passphrase: env:PARTNER_KEY_PASS
  remote_dir: /incoming
  known_hosts_path: /etc/xferity/known_hosts
  host_key_fingerprint_sha256: "SHA256:abc..."
  allow_insecure_host_key: false
  timeout: 30
  op_timeout_seconds: 120
  require_stable_remote: false
  stability_check_seconds: 2
  stability_checks: 3
  retry:
    max_attempts: 3
    initial_delay_ms: 1000
    max_delay_ms: 30000

Field	Type	Default	Description
host	string	—	SFTP server hostname or IP.
port	int	22	SFTP server port.
user	string	—	SSH login username.
password	string	—	Password or secret reference. Omit when using key auth.
key_path	string	—	Path or secret reference to the SSH private key.
key_passphrase	string	—	Passphrase or secret reference for the private key.
remote_dir	string	—	Remote directory to upload to or download from.
known_hosts_path	string	—	Path to an SSH known_hosts file for host verification.
host_key_fingerprint_sha256	string	—	SHA-256 fingerprint to pin. Must start with SHA256:.
allow_insecure_host_key	bool	false	Skip host key verification. Not recommended; rejected by hardened mode.
timeout	int	—	Connection timeout in seconds.
op_timeout_seconds	int	—	Per-operation timeout in seconds.
require_stable_remote	bool	false	Wait for remote file size stability before processing.
stability_check_seconds	int	2	Interval between stability size checks.
stability_checks	int	3	Number of matching size checks required before proceeding.
retry.max_attempts	int	—	Maximum transfer retry attempts.
retry.initial_delay_ms	int	—	Initial retry backoff in milliseconds.
retry.max_delay_ms	int	—	Maximum retry backoff cap in milliseconds.

File patterns

The files block defines which files to match. If omitted, all files in the source path are included.

files:
  - pattern: "*.xml"
    required: false
    rename_to: ""
    delete_after: false

Field	Type	Default	Description
pattern	string	—	Glob pattern to match file names.
required	bool	false	Fail the run if no files match this pattern.
rename_to	string	—	Rename matched files at the destination.
delete_after	bool	false	Delete the source file after successful transfer.

PGP configuration

The pgp block controls OpenPGP payload protection.

pgp:
  provider: auto
  gnupg_binary: /usr/bin/gpg
  encrypt: true
  decrypt: false
  sign: false
  verify: false
  public_key_path: file:/etc/xferity/keys/partner-public.asc
  private_key_path: file:/etc/xferity/keys/our-private.asc
  passphrase: env:PGP_PASSPHRASE

Field	Type	Default	Description
provider	string	gopenpgp	OpenPGP execution mode: gopenpgp, gnupg, or auto.
gnupg_binary	string	gpg	Path to the GnuPG binary. Required when provider=gnupg or provider=auto.
encrypt	bool	—	Encrypt output files using the partner public key.
decrypt	bool	—	Decrypt downloaded files using our private key.
sign	bool	—	Sign output files using our private key.
verify	bool	—	Verify signatures on downloaded files using the partner public key.
public_key_path	string	—	Path or secret reference to the partner or recipient public key.
private_key_path	string	—	Path or secret reference to our signing or decryption private key.
passphrase	string	—	Passphrase or secret reference for the private key.

Provider modes

Mode	Behavior
gopenpgp	Native Go OpenPGP only. No fallback.
gnupg	GnuPG binary only. Requires gnupg_binary to be available.
auto	Tries native path first. Falls back to GnuPG once only when the native error is classified as compat_enterprise_key_structure. Not a general retry mechanism.

When provider=gnupg or provider=auto, startup validation requires GnuPG to be reachable.

Idempotency

Idempotency prevents a flow from reprocessing files it already handled.

Field	Type	Default	Description
idempotency_mode	string	hash	hash (persist file hashes), none (disabled), or tracking (by filename).

In Postgres-backed mode, idempotency records are durable and survive restarts. In file-backed mode, they depend on local state files.

Locking

Flow locking prevents overlapping runs of the same flow.

Field	Type	Default	Description
lock_wait	bool	false	Wait for the lock instead of failing immediately when already locked.
lock_stale_after_seconds	int	—	After how many seconds to consider a lock stale and claim it.
max_lock_wait_seconds	int	—	Maximum time to wait for a lock release when lock_wait=true.

Cleanup behavior

Field	Type	Default	Description
delete_after_upload	bool	false	Delete local source file after successful upload.
delete_source_after_upload	bool	false	Alias for delete_after_upload.
delete_encrypted_after_upload	bool	false	Delete the encrypted intermediate file after upload.
delete_remote_after_decrypt	bool	false	Delete the remote file after successful download and decryption.
delete_encrypted_after_decrypt	bool	false	Delete the downloaded encrypted file after decryption.
compress_before_encrypt	bool	false	Compress the file before encrypting.

Performance overrides

Field	Type	Default	Description
performance.max_parallel_files	int	—	Maximum files processed in parallel for this flow.

Notifications

The notifications block controls per-flow notification behavior. It overrides or supplements global notification configuration.

notifications:
  on_start: false
  on_success: true
  on_failure: true
  failure_policy: fail_open
  email_to: ops@example.com
  slack_webhook_url: ""
  slack_channel: ""
  webhook_url: ""
  ntfy_topic: ""
  ntfy_priority: ""
  gotify_priority: 0
  pushover_sound: ""

Field	Type	Default	Description
on_start	bool	false	Send notification when a run starts.
on_success	bool	false	Send notification when a run succeeds.
on_failure	bool	true	Send notification when a run fails.
failure_policy	string	fail_open	fail_open: ignore notification errors. fail_closed: fail the run when notification delivery fails.
email_to	string	—	Override recipient email address for this flow.
slack_webhook_url	string	—	Override Slack webhook URL for this flow.
slack_channel	string	—	Override Slack channel for this flow.
webhook_url	string	—	Override generic webhook URL for this flow.
webhook_headers	map	—	Additional HTTP headers for the webhook request.
ntfy_topic	string	—	Override ntfy.sh topic for this flow.
ntfy_priority	string	—	Override ntfy.sh priority: min, low, default, high, urgent.
gotify_priority	int	—	Override Gotify message priority (1–10) for this flow.
pushover_sound	string	—	Override Pushover notification sound for this flow.
pushover_priority	int	—	Override Pushover priority (-2 to 2) for this flow.

Scheduling

Flows can be run on a cron schedule by setting schedule_cron. Xferity uses a six-field cron expression with seconds as the first field.

schedule_cron: "0 */10 * * * *"

Fields (left to right): seconds minutes hours day-of-month month day-of-week.

Alternatively, use xferity run-service <flow> --interval-seconds 300 for polling without a cron expression.

Complete minimal example: SFTP upload with PGP encryption

flows:
  payroll-upload:
    direction: upload
    enabled: true
    target:
      partner: payroll-processor
      path: /incoming
    local:
      path: ./storage/payroll/outgoing
    pgp:
      provider: auto
      encrypt: true
      public_key_path: file:./keys/payroll/processor-public.asc
      gnupg_binary: /usr/bin/gpg
    idempotency_mode: hash
    delete_after_upload: true
    schedule_cron: "0 0 6 * * *"
    notifications:
      on_failure: true
      email_to: ops@example.com

Complete minimal example: SFTP download with PGP decryption

flows:
  bank-statements-inbound:
    direction: download
    enabled: true
    source:
      partner: bank-sftp
      path: /outgoing/statements
    local:
      path: ./storage/bank/statements
    files:
      - pattern: "*.pgp"
    pgp:
      provider: auto
      decrypt: true
      private_key_path: file:./keys/bank/our-private.asc
      passphrase: env:BANK_PGP_PASSPHRASE
      gnupg_binary: /usr/bin/gpg
    idempotency_mode: hash
    delete_remote_after_decrypt: false
    delete_encrypted_after_decrypt: true
    schedule_cron: "0 */15 6-18 * * *"

Flow Crypto Requirements

Flow pages in the UI and the API endpoint GET /api/flows/{name}/crypto/requirements expose the canonical Flow Crypto Requirements derived from FlowRoleSpecs().

This is the same function used by:

• config validation
• runtime preflight
• runtime crypto resolution

Review Flow Crypto Requirements before enabling a flow in production.

Validation rules

• Flow YAML uses strict field parsing. Unknown fields are rejected.
• direction must be upload or download.
• schedule_cron must be a valid six-field cron expression when set.
• idempotency_mode must be hash, none, or tracking when set.
• sftp.host_key_fingerprint_sha256 must start with SHA256: when set.
• sftp.allow_insecure_host_key=true is rejected in hardened mode.
• PGP provider=gnupg or provider=auto requires GnuPG to be present at startup.

Related pages

• Configuration
• Automation
• Partner Reference
• Config Reference
• Protocols
• Security