The hash filter plugin for Treasure Data's Data Connectors allows you to specify the hashing algorithm on certain columns.
This plugin contains the following option:
| Option | Description |
|---|---|
| column_options | a key-value pairs. Key is a column name and Value is for column options. |
| Child Elements | Description |
|---|---|
| algorithm | The algorithm to digest. Can be: MD5, SHA-1, or SHA-256 (string, required) |
| salt | The salt value. When present, "salt value + original string" is digested (string, optional) |
| salt_column | The column that stores the salt value. When present, "salt column's value + original string" is digested (string, optional) |
| rename_to | Used to change the column name (string, optional) |
| charset_to_encode_strings | The character set encoding name. Used to encode strings in the column that your Java Runtime supports (string, default: UTF-8). |
Add the filter section, including the type, to your load.yml. In the following example, the type is digest:
in:
...
filters:
- type: digest
column_options:
id:
algorithm: MD5
name:
algorithm: SHA-1
rename_to: name_digest
email:
algorithm: SHA-256
salt: salt_value
phone:
algorithm: SHA-256
salt_column: col1
out:
...