Built-in Settings and Capabilities

The Singer SDK library provides a number of built-in settings and capabilities.

singer_sdk.helpers.capabilities.ACTIVATE_VERSION

Support the ACTIVATE_VERSION extension.

Example:

{
    "activate_version": true
}

singer_sdk.helpers.capabilities.ADD_RECORD_METADATA

Add metadata to records.

Example:

{
    "add_record_metadata": true
}

singer_sdk.helpers.capabilities.BATCH

For taps, support emitting BATCH messages. For targets, support consuming BATCH messages.

Example:

{
    "batch_config": {
        "encoding": {
            "format": "jsonl",
            "compression": "gzip"
        },
        "storage": {
            "type": "root",
            "root": "file:///path/to/batch/files",
            "prefix": "batch-"
        }
    }
}

singer_sdk.helpers.capabilities.FLATTENING

Support schema flattening, aka de-nesting of complex properties.

Example:

{
    "flattening_enabled": true,
    "flattening_max_depth": 3
}

singer_sdk.helpers.capabilities.STREAM_MAPS

Support inline stream map transforms.

Example:

{
  "stream_maps": {
    "users": {
      "id": "id",
      "fields": "[f for f in fields if f['key'] != 'age']"
    }
  }
}

singer_sdk.helpers.capabilities.TARGET_BATCH_SIZE_ROWS

Target batch size in rows.

Example:

{
    "batch_size_rows": 10000
}

singer_sdk.helpers.capabilities.TARGET_HARD_DELETE

Support hard delete capability.

Example:

{
    "hard_delete": true
}

singer_sdk.helpers.capabilities.TARGET_LOAD_METHOD

Target load method.

Example:

{
    "load_method": "upsert"
}

singer_sdk.helpers.capabilities.TARGET_SCHEMA

Allow setting the target schema.

Example:

{
    "default_target_schema": "my_schema"
}

singer_sdk.helpers.capabilities.TARGET_VALIDATE_RECORDS

Validate incoming records against their declared schema.

Example:

{
    "validate_records": true
}