This feature is not enabled by default. Contact your Customer Success representative or Treasure AI Support to have it enabled for your account.
Performing zero-copy from the Snowflake Datawarehouse leverages the Treasure Data Snowflake integration to perform a federated query. The data flow looks like this:
At a high level, the steps you take to implement zero-copy with Snowflake are:
- In the TD Integrations Hub, create an authentication using the Snowflake integration.
- Create a New Zero-Copy source, specifying the Role Name, Warehouse, and Database as the "Zero-Copy Config."
- In the TD Data Workbench, create a New Query. Preface the table name you want to query with the name of the zero-copy source you just created. For example, if you had a Zero-Copy (Federated Query) config named snowflake_conn which was configured like this:
Then you could specify a query of table1 in the YAN database like this:
select * from snowflake_conn.test_schema.table1 The test_schema is the Snowflake schema name.
- After running your query, process the results to update or augment your existing audience profiles.
After Treasure Data finishes processing the query results, the query results are deleted in seven days. However, you can use the INSERT INTO option to store the query results in Treasure Data.
Zero-copy with Snowflake has the following limitations:
- Catalog name length cannot exceed 64 characters.
- Catalog name prefix must start with snowflake_.
- You can create up to 200 "Zero-Copy (Federated Query) Configs" per account across all supported data warehouses combined. Each Config is a connection configuration for one data warehouse and corresponds to one Trino catalog; this limit applies to Configs, not tables.
- The federated query is only available on Trino and does not support the Array type.
- The 10GB input limit applies to the amount of data read from the external data warehouse per query. It does not apply to query results or data read from Treasure AI storage. If enabled and a query exceeds 10GB during execution, the query is aborted. This limit is not currently enabled by default in production, but is planned to be enabled by default in the near future. Customers cannot change this limit themselves, but Treasure AI Support can adjust the threshold on request.
- Catalog reflection time: Max 5 minutes(It takes up to 5 minutes to reflect a newly created zero-copy catalog or a change in a catalog.)