SFTP Source

The SFTP source connects to a remote SFTP server and polls a directory for new files at a configurable interval. Each file discovered is read into the pipeline, then moved to an archive directory or an error directory depending on the processing outcome.

Configuration

source:
  type: sftp
  host: sftp.lab-partner.com
  port: 22
  poll_interval: 1m
  directory: /outbound/results
  file_pattern: "*.hl7"
  move_to: /outbound/archive
  error_dir: /outbound/error
  auth:
    type: private_key
    username: intuware
    private_key_file: ./keys/id_ed25519
    passphrase: ${SFTP_KEY_PASSPHRASE}
  sort_by: date

Properties

Property	Type	Required	Default	Description
`host`	`string`	Yes	—	Hostname or IP of the SFTP server.
`port`	`int`	No	`22`	SSH port.
`poll_interval`	`string`	No	`1m`	How often to check for new files. Supports `s`, `m`, `h` suffixes.
`directory`	`string`	Yes	—	Remote directory to poll.
`file_pattern`	`string`	No	`*`	Glob pattern to match filenames.
`move_to`	`string`	No	—	Remote directory to move files to after successful processing.
`error_dir`	`string`	No	—	Remote directory to move files to when processing fails.
`auth`	`object`	Yes	—	Authentication credentials (see below).
`sort_by`	`string`	No	`name`	Order in which files are processed: `name`, `date`, or `size`.

Authentication

Password

auth:
  type: password
  username: intuware
  password: ${SFTP_PASSWORD}

Private Key

auth:
  type: private_key
  username: intuware
  private_key_file: ./keys/id_ed25519
  passphrase: ${SFTP_KEY_PASSPHRASE}

The passphrase field is only required if the private key is encrypted.

Complete Example

Poll lab results from an external SFTP server and route them into the hospital LIS:

channel:
  name: lab-sftp-pickup
  description: Retrieve lab results from reference lab SFTP

source:
  type: sftp
  host: sftp.lab-partner.com
  port: 22
  poll_interval: 1m
  directory: /outbound/results
  file_pattern: "*.hl7"
  move_to: /outbound/archive
  error_dir: /outbound/error
  auth:
    type: private_key
    username: intuware
    private_key_file: ./keys/id_ed25519
    passphrase: ${SFTP_KEY_PASSPHRASE}
  sort_by: date

transformer:
  type: typescript
  file: ./transforms/normalize-lab.ts

destination:
  type: tcp
  host: lis.hospital.local
  port: 2575
  mode: mllp