Destination Adapter: PayAnalytics Dataset

Description

The PayAnalytics Dataset destination adapter allows you to push data to PayAnalytics (leverages the PayAnalytics REST API). It allows for the same level of configuration as the API provides, i.e. field mapping, gender mapping, currency configuration, etc.

Authentication

You need to provide your PayAnalytics API key as a secret so the DataBridge Engine Pipeline can authenticate against PayAnalytics. You can retrieve the key through the PayAnalytics web interface.

Configuration

Required Parameters

To use this adapter, set the adapter configuration attribute to payanalytics_dataset.

instance_url (string)

Your PayAnalytics instance URL.

data_source (string)

Name of the dataframe containing employee data.

api_token_secret_name (string)

Name of the secret containing your PayAnalytics API token.

field_mapping (object)

Configuration for mapping source fields to PayAnalytics fields.

Required Field Mappings

employee_id_field (string)

A field in the data_source dataset containing employee identifiers.

main_employee_characterization_field (string)

A field in the data_source dataset containing primary employee classification field.

first_demographic_variable (string)

A field in the data_source dataset containing the first demographic value (e.g., Gender).

salary_field (string)

A field in the data_source dataset containing base salary value.

Optional Field Mappings

gender_field (string or null)

A field in the data_source dataset containing gender information (often the same as first_demographic_variable)

second_demographic_variable (string or null)

A field in the data_source dataset containing a second demographic value (e.g., Ethnicity).

gender_mapping (object or none)

Optional mapping for gender values (required if gender_field is non-null). With the following gender mapping configuration, the value for men in the data_source dataset is "M", the indicator for women is "F", for non-binary people it is "NB" and "NR" indicates unknown/not reported.

{
  "male": "M",
  "female": "F",
  "nonBinary": "NB",
  "notReported": "NR"
}

A different example follows:

{
  "male": "1",
  "female": "2",
  "nonBinary": "9",
  "notReported": "0"
}

Optional Parameters

identifier_prefix (string or null)

Prefix for datasets. If omitted (or a null value provided) the name of the dataset will be the current date/time (UTC timezone), see date_format for format details. If a prefix is provided, it is prepended to the date and separated by a whitespace, e.g., where "identifier_prefix":"MyData":

MyData 2025-04-01 15:02:17

date_format (string or null)

Optional format string used when generating dataset identifiers (when auto_date_label is enabled). Defaults to %Y-%m-%d %H:%M:%S which produces a date formatted as 2024-12-31 15:02:17.

See python strftime() documentation for formatting options.

compensation_components (array of strings or null)

List of compensation component field names. All components have to exist as field names in the data_source Data Frame.

compensation_component_aggregates (array of objects or null)

Configuration for aggregated compensation components. Each component has the following structure where name indicates the name of the new aggregate, components is a list of fields that are to be summed and scaled_components is a fields of list that will be scaled according to the part-time ratio and added to the component sum. Example:

{
  "name": "Gross Pay",
  "components": [
    "bonus"
  ],
  "scaled_components": [
    "salary"
  ]
}

part_time_ratio_field (string or null)

Field containing part-time ratios. All values of the field shall be in the range [0=0%,1=100%]

part_time_scaled_compensation_components (array of strings or null)

A list of the compensation components that are scaled by part-time ratio.

exchange_rate_currency_code_field (string or null)

Field containing currency codes, e.g. "Currency". All values of this field need to contain a currency code, e.g. "USD", "EUR", etc.

exchange_rate_id (string or null)

An indicator as to which PayAnalytics currency table series to use for this dataset. Corresponds to the "Primary currency" field in the Currencies configuration of PayAnalytics.

Example value: "USD"

WARNING

There needs to be an existing currency table in PayAnalytics for use of the exchange_rate_id value.

partition_field (string)

A field which is used to partition the dataset. If this option is specified, multiple datasets will be created (one for each distinct value of the partition_field). A common example would be to specify the "Country" field - this would create one dataset for each country.

A corresponding label is applied to each created dataset (e.g. the "France" label is applied to the "France" dataset). Labels are created on demand if needed.

apply_labels (list)

A list of labels to apply to the newly created dataset.

Example value:

[
  "DataPipeline",
  "Production"
]

Creates new label on-demand if needed.

auto_date_label (boolean)

If true, adds a label with the current date (e.g. '2013-10-21'). This is useful when the same pipeline configuration is run periodically and each dataset (or set of datasets from partitions) need to have specific labeling.

Example value:

true

Creates a new label on-demand if needed

Configuration Examples

Basic Configuration

{
  "version": 1,
  "destination_adapters": [
    {
      "adapter": "payanalytics_dataset",
      "configuration": {
        "instance_url": "https://your-instance.payanalytics.com",
        "data_source": "employees",
        "api_token_secret_name": "pa_api_token",
        "field_mapping": {
          "employee_id_field": "empid",
          "main_employee_characterization_field": "jobrole",
          "gender_field": "gender",
          "first_demographic_variable": "gender",
          "salary_field": "salary"
        },
        "gender_mapping": {
          "male": "male",
          "female": "female",
          "nonBinary": null,
          "notReported": null
        }
      }
    }
  ]
}

The following data file (named employees.json and copied to the Drop Zone Folder matches the configuration above:

[
  {
    "empid": 1,
    "gender": "male",
    "jobrole": "engineering",
    "salary": 1234
  },
  {
    "empid": 2,
    "gender": "female",
    "jobrole": "legal",
    "salary": 12350
  }
]

Advanced Configuration with Compensation Components, partitioned by country and applies labels.

{
  "version": 1,
  "destination_adapters": [
    {
      "adapter": "payanalytics_dataset",
      "configuration": {
        "instance_url": "https://your-instance.payanalytics.com",
        "data_source": "employees",
        "api_token_secret_name": "pa_api_token",
        "field_mapping": {
          "employee_id_field": "employee_id",
          "main_employee_characterization_field": "job_title",
          "gender_field": "gender",
          "first_demographic_variable": "gender",
          "second_demographic_variable": "ethnicity",
          "salary_field": "base_salary"
        },
        "gender_mapping": {
          "male": "M",
          "female": "F",
          "nonBinary": "NB",
          "notReported": "NR"
        },
        "compensation_components": [
          "bonus",
          "stock_options",
          "benefits"
        ],
        "compensation_component_aggregates": [
          {
            "name": "total_compensation",
            "components": [
              "base_salary",
              "bonus",
              "stock_options"
            ],
            "scaled_components": [
              "base_salary"
            ]
          }
        ],
        "part_time_ratio_field": "fte_ratio",
        "part_time_scaled_compensation_components": [
          "base_salary",
          "bonus"
        ],
        "partition_field": "Country",
        "apply_labels": [
          "DataPipeline",
          "Production"
        ],
        "identifier_prefix": "FinalData",
        "date_format": "%Y-%m-%d %H:%M:%S",
        "auto_date_label": true
      }
    }
  ]
}