# Bulk Data Import

Bulk data import loads large amounts of data into a database in a relatively short period of time. When you need to load a lot of data all at once, inserting one row at a time is inefficient; bulk import operations use more efficient methods.

Bulk operations might bypass triggers and integrity checks (such as constraints). Bypassing these can significantly improve data loading performance.

## Bulk import integration options 

Use Embulk to bulk import data into Treasure AI. Treasure AI offers integrations that allow you to import data in bulk from the following:

* [Embulk Bulk Import from AWS S3](https://docs.treasuredata.com/smart/project-integrations/embulk-bulk-import-from-aws-s3)
* [Embulk Bulk Import from Google Sheets](https://docs.treasuredata.com/smart/project-integrations/embulk-bulk-import-from-google-sheets)
* [Bulk Import from MySQL](https://docs.treasuredata.com/smart/project-integrations/embulk-bulk-import-from-mysql)


## Installing Embulk 

You can import data using Treasure AI's open-source bulk data loader [Embulk](http://www.embulk.org/docs/). Embulk helps transfer data between various databases, storage locations, file formats, and cloud services.

### Prerequisites

* Basic knowledge of Treasure AI
* Basic knowledge of [Embulk](http://www.embulk.org/docs/)
* Java installed (Embulk is a Java application)
* [JRuby](https://www.jruby.org/download) installed and configured (Embulk v0.10.50 and v0.11.0 do not include JRuby; see the "JRuby" section of [Embulk v0.11 is coming soon](https://www.embulk.org/articles/2023/04/13/embulk-v0.11-is-coming-soon.html))


### Installing Embulk from the command line

Linux / macOS / BSD

```bash
curl --create-dirs -o ~/.embulk/bin/embulk -L "http://dl.embulk.org/embulk-latest.jar"
chmod +x ~/.embulk/bin/embulk
echo 'export PATH="$HOME/.embulk/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc
```

Windows (PowerShell)

```powershell
Invoke-WebRequest http://dl.embulk.org/embulk-latest.jar -OutFile embulk.bat
```

### Installing the Embulk Treasure Data plugin

Embulk plugins load data to or from various systems and file formats. See the [list of Embulk plugins](https://plugins.embulk.org/).

Install the `embulk-output-td` plugin (imports records to Treasure AI):


```bash
embulk gem install embulk-output-td
```

### Using a proxy server

If you cannot upload, verify whether your network uses a proxy. Set the proxy with command-line options:

Linux

```bash
embulk -J-Dhttp.proxyHost=HOST -J-Dhttp.proxyPort=PORT -J-Dhttp.proxyUser=USER -J-Dhttp.proxyPassword=PASS run config.yml
```

Windows

```powershell
embulk.bat "-J-Dhttps.proxyHost=HOST" "-J-Dhttps.proxyPort=PORT" "-J-Dhttp.proxyUser=USER" "-J-Dhttp.proxyPassword=PASS" run config.yml
```

Java (direct)

```bash
java -Dhttps.proxyHost=HOST -Dhttps.proxyPort=PORT -jar embulk.bat run config.yml
```

## Using environment variables with bulk import

See [Using Environment Variables with Bulk Import](/products/customer-data-platform/integration-hub/batch/import/using-environment-variables-with-bulk-import).