Realtime Secure Signed Webhook Streaming Export Integration

Secure Signed Webhook (SS Webhook) Streaming Export Integration enables real-time data delivery to external webhook endpoints with cryptographic signature verification as part of Treasure Data's activation workflows.

With this integration, you can securely send customer data to your webhook endpoints in real-time, with signatures to verify the authenticity and integrity of each request.

• Basic knowledge of Treasure Data
• Basic knowledge of webhook architecture and HTTP/HTTPS protocols
• Understanding of HMAC-SHA256 signature verification
• A publicly accessible HTTPS endpoint that can receive HTTP requests (Supported HTTP Methods: GET, POST, PUT, DELETE, and PATCH)
• Ability to implement signature verification logic on your webhook receiver

The primary use case for this streaming integration is to incorporate it into a real-time activation workflow to:

• Send customer profile data to external systems in real-time
• Trigger downstream processes based on customer behavior
• Synchronize customer data with third-party applications
• Enable real-time personalization in external platforms

Here is an overview of how to configure that workflow to trigger a delivery:

1. Set up your webhook endpoint to receive HTTP requests with JSON payloads (POST is the default method, but other HTTP methods can be configured).
2. Implement signature verification logic using HMAC-SHA256 on your webhook receiver.
3. Create an authentication for this integration with your webhook URL and signing secret.
4. In a Real-time Journey, select this authentication for a new triggered activation.

• The webhook endpoint must be accessible via HTTPS (TLS 1.2 or higher recommended).
• The endpoint must respond within 30 seconds to avoid timeout errors.
• Maximum payload size is 1MB per request.
• Rate limits depend on your infrastructure capacity.
• The signature is calculated using HMAC-SHA256 and sent in the X-TD-Signature header.
• Your webhook must return HTTP status codes 200-299 for successful processing.

You must create and configure the data connection in Treasure Data. As part of the data connection, you provide authentication credentials to access the integration.

Perform the following steps to create a new authentication with a set of credentials.

1. Select Integrations Hub.
2. Select Catalog.
3. Search for your Integration in the Catalog; hover your mouse over the icon and select Create Authentication.

4. Ensure that the Credentials tab is selected and then enter credential information for the integration.

Authentication Type Details:

• Basic Auth: HTTP Basic Authentication using username and password

  • User Name: The username for authentication
  • Password: The password for authentication

• Bearer Token: OAuth 2.0 Bearer Token authentication

  • Bearer Token: The authentication token value

• API Key: Custom API key authentication

  • API Key Name: The name of the API key header or parameter
  • API Key: The API key value

• Treasure API Key: Treasure API key authentication

  • Uses your Treasure API key for authentication

• Custom Auth: Custom authentication headers

  • Configure custom authentication headers as needed

• No Auth: No authentication required

  • Use only for testing or publicly accessible endpoints

5. Select Continue.
6. Enter a name for your connection.
7. Select Done.

After configuring the SS Webhook Streaming integration, it is now available to be incorporated into real-time customer journey activations. See Creating a Real-time Triggered Activation.

To create or update a triggered activation:

1. Select the triggered activation you want to configure .

2. In the triggered activation window, select an activation from the Copy triggered activation from drop-down menu.

3. Enter a Triggered activation name.

4. Enter an Optional description of the activation.

5. From the Authentication drop-down menu, select the authentication the activation should use.

Depending on the type of authentication you choose, the details you will need to provide will vary. Refer to the integration-specific table below for more information.

6. Verify or provide the information required for the fields on the Details page.
7. After the Details are configured, select Create or Update.

The SS Webhook integration supports dynamic runtime expressions and cryptographic functions that can be used in custom headers and request body.

Use these expressions to dynamically insert values at request time:

Example usage in Custom Headers:

{
  "X-Timestamp": "${unix_timestamp}",
  "X-Method": "${http_method}"
}

Use these cryptographic functions to generate secure signatures for request authentication:

Example usage for signature generation:

{
  "X-Signature": "${hmac_sha256_hex(my_secret_key, ${unix_timestamp}.${request_body})}",
  "X-Timestamp": "${unix_timestamp}"
}

This allows you to implement custom signature verification on your webhook endpoint similar to the Secure Signed Webhook pattern.

• The connector will retry to send the request when it gets the status code 429, 500, 502, 503, or 504.
• If the request is sent successfully before the Retry Limit is reached, the logging will show successfully.
• If the request still fails after reaching the Retry Limit, the connector will stop the job.
• We will see the error message and the status code in the client's activation table whenever the request sent fails.
• The Retry Limit can be configured in the Activation Config.

• Idempotency: Design your webhook receiver to handle duplicate requests safely, as retries may occur.
• Timeout Handling: Ensure your webhook endpoint responds within 30 seconds. For long-running processes, acknowledge receipt immediately and process asynchronously.
• Security: Always verify the signature before processing the payload to prevent unauthorized requests.
• Monitoring: Implement logging and monitoring on your webhook endpoint to track successful deliveries and failures.
• Error Handling: Return appropriate HTTP status codes (200-299 for success, 4xx for client errors, 5xx for server errors).