Configuration file (config.json)

Minimal configuration file

A minimal legal configuration file is as follows:

{
  "version": 1
}

With this configuration, the pipeline is triggered and yields a results.json file, but no operation is carried out by the pipeline.

Configuration file format

The configuration file has multiple sections, each for configuring different sections of the pipeline. The format of the file is as follows:

{
  "version": 1,
  "drop_zone_files": [
    ...
  ],
  "source_connectors": [
    ...
  ],
  "destination_adapters": [
    ...
  ],
  "data_validations": [
    ...
  ],
  "transformations": [
    ...
  ]
}

version is a mandatory field but all other sections are optional.

Detailed documentation on each section of the configuration file is available in the following sections:

• drop_zone_files: Drop Zone Files
• source_connectors: Source Connectors
• destination_adapters: Destination Adapters
• data_validations: Data Validations
• transformations: Transformations