Skip to content
Xferity

Tutorial: Automating SFTP Pickups

Tutorial: Automating SFTP Pickups

Section titled “Tutorial: Automating SFTP Pickups”

This tutorial builds on Your First SFTP Transfer and adds automated scheduling, idempotency controls, and failure alerting to turn a manual one-time run into a reliable recurring pickup.

Prerequisites: a working SFTP transfer (or complete the first tutorial first)

What you will add

Section titled “What you will add”
  • a six-field cron schedule to run every 15 minutes during business hours
  • idempotency to prevent duplicate processing on reruns
  • failure notifications via email or Slack
  • a run-service mode for long-running polling

Step 1: Add a schedule to your flow

Section titled “Step 1: Add a schedule to your flow”

Update flows/supplier-inbound.yaml:

flows:
  supplier-inbound:
    direction: download
    enabled: true
    source:
      partner: supplier-sftp
      path: /outgoing
    local:
      path: ./storage/inbox
    files:
      - pattern: "*.xml"
    idempotency_mode: hash
    schedule_cron: "0 */15 8-18 * * 1-5"
    notifications:
      on_failure: true
      email_to: ops@example.com

Schedule breakdown — 0 */15 8-18 * * 1-5:

  • 0 — at second 0
  • */15 — every 15 minutes
  • 8-18 — between 8am and 6pm
  • * * * — any day, month
  • 1-5 — Monday through Friday

Cron expressions use six fields with seconds as the first field.

Step 2: Add a notification channel

Section titled “Step 2: Add a notification channel”

If you want Slack instead of email, add a global notifications config:

# config/config.yaml additions
notifications:
  slack:
    enabled: true
    default_webhook_url: env:SLACK_WEBHOOK_URL
    default_channel: "#ops-alerts"

And in the flow, replace email_to with channel override:

notifications:
  on_failure: true
  slack_channel: "#ops-sftp-failures"

Step 3: Run in service mode

Section titled “Step 3: Run in service mode”

Instead of running once with xferity run, start a long-running polling loop:

Terminal window
xferity run-service supplier-inbound --interval-seconds 900

Or let the flow’s schedule_cron drive the schedule:

Terminal window
xferity run-service supplier-inbound

When schedule_cron is set and no --interval-seconds is provided, the service uses the cron expression to determine when to next run.

Step 4: Confirm idempotency is working

Section titled “Step 4: Confirm idempotency is working”

Run the flow twice manually on the same set of files:

Terminal window
xferity run supplier-inbound
xferity run supplier-inbound

On the second run, you should see in the logs that files were skipped as already-processed. The audit trace will show outcome: skipped for those files.

Step 5: Test failure notification

Section titled “Step 5: Test failure notification”

To test notifications without causing a real failure:

  1. point the partner to an unreachable host temporarily
  2. run once
  3. confirm the failure notification arrives
  4. restore the correct partner config

Understanding the operational model

Section titled “Understanding the operational model”

When running as a service:

  • Xferity holds a persistent process that wakes at each scheduled interval
  • each wakeup creates a fresh run context
  • idempotency records persist across runs so files are not reprocessed
  • flow locking prevents concurrent runs if the previous run is still active when the next trigger fires
Section titled “Related pages”