The encrypt filter plugin for Treasure Data integrations allows you to encrypt columns using an encryption algorithm such as AES.
The encrypt filter plugin contains the following options:
| Option | Description |
|---|---|
algorithm | Encryption algorithm (string, required) Available algorithms are:
|
| column_names | Names of string columns to encrypt (array, required) |
| key_type | Encryption type, can be either "inline" or "s3" (string, optional, default: inline) |
| key_hex | Encryption key (string, required if key_type is inline) |
| iv_hex | Encryption initialization vector (string, required if mode of the algorithm is CBC and key_type is inline) |
| output_encoding | The encoding of encrypted value, can be either "base64" or "hex" (string, optional, default: base64) |
| aws_params | AWS/S3 parameters (a hash, required if key_type is s3) |
| Child Elements (aws_params) | Description |
|---|---|
| region | a valid AWS region (string, required) |
| access_key | a valid AWS access key (string, required) |
| secret_key | a valid AWS secret key (string, required) |
| bucket | a valid S3 bucket (string, required) |
| path | a valid S3 key, S3 file path (string, required) |
Add the filter section, including the type, to your load.yml. In the following example, the type is encrypt:
in:
...
filters:
- type: encrypt
algorithm: AES-256-CBC
column_names: [password, ip]
key_hex: xxxxxxxxxxxxxxxxxxx
iv_hex: xxxxxxxxxxx
output_encoding: hex
out:
...Add a filter section, including the type of filter, to your load.yml. In the following example, the type is encrypt and key_type is s3:
in:
...
filters:
- type: encrypt
algorithm: AES-256-CBC
column_names: [password, ip]
output_encoding: hex
key_type: s3
aws_params:
region: us-east-2
access_key: XXXXXXXXXXXXXXXXXXXX
secret_key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
bucket: com.sample.keys
path: key.aes
out:
...The S3 key file should be in valid YAML format: (iv_hex is required if mode of the algorithm is CBC)
key_hex: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
key_iv: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx