Batch Export Integration

This integration exports customer profile data from Treasure AI directly to Batch using the Batch Profile Mass Update API. Batch is a customer engagement platform (CEP) for omnichannel campaigns across mobile push, web push, in-app messaging, email, and SMS.

The integration maps each query result row to a Batch profile operation. Identity, native attributes, custom attributes, and events are all declared through a column-name prefix convention in your query, so no per-column configuration is needed. You can run the integration in Attributes mode (one row updates one profile) or Events mode (one row records one event, grouped by identifier).

• Basic knowledge of Treasure AI.
• Basic knowledge of Batch profiles, custom IDs, installation identities, native attributes, and events.
• A Batch REST API key.
• A Batch project key.
• A query that exposes a profile identity through the identifier__ column-name prefix (identifier__custom_id, or identifier__apikey together with identifier__installation_id).

If your security policy requires IP whitelisting, you must add Treasure Data's IP addresses to your allowlist to ensure a successful connection.

Please find the complete list of static IP addresses, organized by region, at the following document

• All array-valued attributes are replace-only. string_array__ custom attributes and $topic_preferences are always sent as the complete array. Incremental $add/$remove operations are not supported in this version, even though the Batch API supports them. Your query must emit the full desired array.
• There is no opt-out for the batch__ prefix. A column named batch__region always maps to the native $region attribute. To send a custom attribute instead, alias the column to a name without the prefix.
• The Profile Mass Update API is synchronous. There is no async indexing token or polling. (The Batch Audience API has async tokens; do not conflate the two.)
• Throughput (10,000 profile updates per second) is not actively throttled client-side. Only 429 responses trigger a back-off.
• Column names are normalized to lower case automatically, so uppercase or mixed-case names are accepted. Names that still do not match ^[a-z0-9_]{1,30}$ after lower-casing (for example, names containing hyphens or dots, or longer than 30 characters) must be aliased in the query. No runtime rename beyond case-folding is supported.
• Rate-limit (429) and server errors (5xx, network timeouts) are retried with exponential back-off. 400 and 401 responses fail fast because they indicate a payload or authentication problem.

In Treasure AI, you must create and configure the data connection before running your query. As part of the data connection, you provide authentication to access Batch.

1. Open TD Console.
2. Navigate to Integrations Hub > Catalog.
3. Click the search icon on the far right of the Catalog screen and enter Batch.
4. Hover over the Batch connector and select Create Authentication.

The New Authentication dialog opens.

1. Enter your Batch REST API Key.
2. Enter your Batch Project Key.
3. Enter a name for your connection.
4. Select Continue.

To configure the integration:

1. Open TD Console.
2. Navigate to Data Workbench > Queries.
3. Click New Query to create a query, or open a saved query that you want to export.
4. At the top of the query editor, select Export Results.
5. Select the Batch authentication that you created.
6. Configure the export parameters.

1. Column prefix detection: The integration inspects each query result column name and classifies it as an identifier (identifier__), a native attribute (batch__), a type-prefixed custom attribute (date__, url__, string_array__, object__, array__), or a plain custom attribute (no prefix, type inferred from the Embulk column type).
2. Identity extraction: For each row, the integration reads the identifier__ value and builds the identifiers block, using either custom_id or the installation form (apikey + installation_id).
3. Attribute or event mapping: In Attributes mode, native columns are emitted under their reserved $… wire keys and custom columns under wrapped or plain keys. In Events mode, each row builds one event, and events are grouped by identifier into the events[] array.
4. Per-row validation: The integration validates key formats, value limits, native attribute types, and the 25 kB serialized attributes size before sending.
5. Batching: Operations are buffered and flushed in batches (up to batch_size, maximum 10,000 operations per request).
6. Retry and partial errors: 429 and server errors are retried with back-off. Batch's SUCCESS_WITH_PARTIAL_ERRORS response is handled according to ignore_partial_errors.

Identity and attribute type are declared by column-name prefixes in the query result schema. No column configuration is needed. Columns fall into three groups.

Column names are matched and serialized case-insensitively. The integration lower-cases each column name before detecting its prefix and deriving the Batch wire key, so IDENTIFIER__Custom_Id, BATCH__Email_Address, and a plain LoyaltyTier are treated as identifier__custom_id, batch__email_address, and loyaltytier respectively. Custom attribute keys are therefore always sent to Batch in lower case (Batch requires them to match ^[a-z0-9_]{1,30}$). In Events mode the batch__name value is also lower-cased before it is validated and sent, so Purchase_Completed becomes purchase_completed.

In Attributes mode, the supported batch__ native attribute names are email_address, email_marketing, phone_number, sms_marketing, email_open_tracking_consent, language, timezone, region, and topic_preferences. In Events mode, the batch__ prefix instead carries the reserved event fields name (required), time, label, and tags. The native attribute set is gated by profile_sync_type: profile-native names are rejected in Events mode and event-reserved names are rejected in Attributes mode.

The connector infers Batch types for plain (no-prefix) custom attributes from the Treasure AI column type.

This example exports customer profiles with native attributes (email, language, region) and custom attributes (a signup date, a profile URL, and a loyalty tier).

SELECT
  customer_id          AS identifier__custom_id,
  email                AS batch__email_address,
  marketing_consent    AS batch__email_marketing,
  phone                AS batch__phone_number,
  preferred_language   AS batch__language,
  timezone             AS batch__timezone,
  country_code         AS batch__region,
  CAST(signup_ts AS VARCHAR) AS date__signup_date,
  profile_url          AS url__profile_url,
  loyalty_tier,
  interests            AS string_array__interests,
  preferences_json     AS object__preferences
FROM customer_profiles
WHERE TD_TIME_RANGE(time, TD_DATE_TRUNC('day', TD_SCHEDULER_TIME() - 86400, 'UTC'), NULL)

Sample query result

This produces one POST /2.11/profiles/mass-update request. Each row becomes one operation with an identifiers.custom_id block and an attributes object using the $email_address, $language, $region, date(signup_date), url(profile_url), and loyalty_tier wire keys. The batch__region value fr is normalized to FR.

Required configuration

This example records one purchase event per row in Events mode. The batch__name column is required; events are grouped under each identifier.

SELECT
  customer_id          AS identifier__custom_id,
  'purchase'           AS batch__name,
  CAST(purchased_at AS VARCHAR) AS batch__time,
  product_category     AS category,
  order_amount         AS amount
FROM purchases
WHERE TD_TIME_RANGE(time, TD_DATE_TRUNC('day', TD_SCHEDULER_TIME() - 86400, 'UTC'), NULL)

Sample query result

Both rows share custom_id abc123, so they are grouped into a single operation with two entries in the events[] array. The category and amount columns become event attributes under events[].attributes.

Required configuration

You can use Scheduled Jobs with Result Export to periodically write the output result to Batch.

Treasure AI's scheduler feature supports periodic query execution to achieve high availability.

You can use Scheduled Jobs with Result Export to periodically write the output result to a target destination that you specify.

Treasure Data's scheduler feature supports periodic query execution to achieve high availability.

When two specifications provide conflicting schedule specifications, the specification requesting to execute more often is followed while the other schedule specification is ignored.

For example, if the cron schedule is '0 0 1 * 1', then the 'day of month' specification and 'day of week' are discordant because the former specification requires it to run every first day of each month at midnight (00:00), while the latter specification requires it to run every Monday at midnight (00:00). The latter specification is followed.

1. Navigate to Data Workbench > Queries

2. Create a new query or select an existing query.

3. Next to Schedule, select None.

4. In the drop-down, select one of the following schedule options:

   Drop-down Value	Description
   Custom cron...	Review Custom cron... details.
   @daily (midnight)	Run once a day at midnight (00:00 am) in the specified time zone.
   @hourly (:00)	Run every hour at 00 minutes.
   None	No schedule.

 *    *    *    *    *
 -    -    -    -    -
 |    |    |    |    |
 |    |    |    |    +----- day of week (0 - 6) (Sunday=0)
 |    |    |    +---------- month (1 - 12)
 |    |    +--------------- day of month (1 - 31)
 |    +-------------------- hour (0 - 23)
 +------------------------- min (0 - 59)

The following named entries can be used:

• Day of Week: sun, mon, tue, wed, thu, fri, sat.
• Month: jan, feb, mar, apr, may, jun, jul, aug, sep, oct, nov, dec.

A single space is required between each field. The values for each field can be composed of:

1. (Optional) You can delay the start time of a query by enabling the Delay execution.

Save the query with a name and run, or just run the query. Upon successful completion of the query, the query result is automatically exported to the specified destination.

(Optional) You can delay the start time of a query by enabling the Delay execution.

You can also send segment data to the target platform by creating an activation in the Audience Studio.

1. Navigate to Audience Studio.
2. Select a parent segment.
3. Open the target segment, right-mouse click, and then select Create Activation.
4. In the Details panel, enter an Activation name and configure the activation according to the previous section on Configuration Parameters.
5. Customize the activation output in the Output Mapping panel.

• Attribute Columns
  • Select Export All Columns to export all columns without making any changes.
  • Select + Add Columns to add specific columns for the export. The Output Column Name pre-populates with the same Source column name. You can update the Output Column Name. Continue to select + Add Columnsto add new columns for your activation output.
• String Builder
  • + Add string to create strings for export. Select from the following values:
    • String: Choose any value; use text to create a custom value.
    • Timestamp: The date and time of the export.
    • Segment Id: The segment ID number.
    • Segment Name: The segment name.
    • Audience Id: The parent segment number.

6. Set a Schedule.

• Select the values to define your schedule and optionally include email notifications.

7. Select Create.

Within Treasure Workflow, you can specify the use of a data connector to export data.

Learn more at Using Workflows to Export Data with the TD Toolbelt.

Sample Workflow Configuration

timezone: UTC

_export:
  td:
    database: sample_datasets

+td-result-into-batch:
  td>: queries/batch_export.sql
  result_connection: batch
  result_settings:
    target: Profile
    profile_sync_type: Attributes
    batch_size: 10000
    skip_invalid_records: true
    ignore_partial_errors: true
    omit_null_fields: true

• Batch Profile Mass Update API documentation
• Batch Profile Attributes specification
• Batch Integration Overview