Source Connector: Star Wars API

The Star Wars API source connector is a demonstration connector that fetches data from the public Star Wars API (https://swapi.info). It's designed for testing, development, and learning purposes, providing a simple way to test DataBridge pipelines without requiring access to production systems.

WARNING

Not for production use. This demo connector is intended for learning/testing only and uses a fixed base URL (swapi.info) without environment overrides.

Overview

This connector serves as an excellent starting point for learning DataBridge and testing pipeline configurations. It retrieves character data from the Star Wars universe and demonstrates key concepts data transformation, and API integration.

Configuration

Required Parameters

Set the connector attribute to star_wars_api.

dataset_prefix (string)

Prefix for the output dataset names

Configuration Example

{
  "version": 1,
  "source_connectors": [
    {
      "connector": "star_wars_api",
      "configuration": {
        "dataset_prefix": "example_run"
      }
    }
  ]
}

Data Structure

Character Data

The connector retrieves comprehensive character information from the Star Wars universe and delivers in the following format:

{
  "name": "Luke Skywalker",
  "height": "172",
  "mass": "77",
  "hair_color": "blond",
  "skin_color": "fair",
  "eye_color": "blue",
  "birth_year": "19BBY",
  "gender": "male"
}

Dataset Naming

The connector creates datasets with names based on the configured prefix:

• Output Dataset: {dataset_prefix}_star_wars_people
• Example: With prefix "example_run", the dataset will be named "example_run_star_wars_people"

Integration Examples

Basic Pipeline

{
  "version": 1,
  "source_connectors": [
    {
      "connector": "star_wars_api",
      "configuration": {
        "dataset_prefix": "sw_demo"
      }
    }
  ],
  "transformations": [
    {
      "name": "Filter valid height",
      "input_data_frames": [
        "sw_demo_star_wars_people"
      ],
      "output_data_frame": "characters_valid_height",
      "type": "sql_inline",
      "configuration": {
        "statement": "SELECT *, CAST(height AS INTEGER) as n_height FROM sw_demo_star_wars_people WHERE TRY_CAST(height AS INTEGER) IS NOT NULL"
      }
    },
    {
      "name": "Filter only tall characters",
      "input_data_frames": [
        "characters_valid_height"
      ],
      "output_data_frame": "tall_characters",
      "type": "sql_inline",
      "configuration": {
        "statement": "SELECT * FROM characters_valid_height WHERE n_height > 200"
      }
    }
  ]
}