{"templateId":"markdown","sharedDataIds":{"sidebar":"sidebar-@l10n/ja/sidebars.yaml"},"props":{"metadata":{"markdoc":{"tagList":["admonition"]},"type":"markdown"},"seo":{"title":"Embulk Bulk Import from CSV files","description":"Treasure Data Product Documentation · Collect and Unify · Segment and Activate · Experiment and Analyze · Decisioning Automate with AI Scale and Trust.","siteUrl":"https://docs.treasuredata.com","lang":"en-US","llmstxt":{"hide":false,"sections":[{"title":"Table of contents","includeFiles":["**/*"],"excludeFiles":[]}],"excludeFiles":[]}},"dynamicMarkdocComponents":[],"compilationErrors":[],"ast":{"$$mdtype":"Tag","name":"article","attributes":{},"children":[{"$$mdtype":"Tag","name":"Heading","attributes":{"level":1,"id":"embulk-bulk-import-from-csv-files","__idx":0},"children":["Embulk Bulk Import from CSV files"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["オープンソースのバルクデータローダーであるEmbulkを使用して、CSVファイルからTreasure Dataにデータをインポートできます。Embulkを使用すると、さまざまなデータベース、ストレージの場所、ファイル形式、クラウドサービス間でデータを転送できます。"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":[{"$$mdtype":"Tag","name":"img","attributes":{"src":"/assets/image-20191021-194315.1d20dc43b99774785560764470f9f3522e77b6b3188edfe480ebf89b39071dba.75c1e439.png","alt":""},"children":[]}]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Bulk Importプログラム(td-import)を使用してCSVファイルからデータをインポートすることもできます。ただし、td-importは積極的にメンテナンスされておらず、廃止の候補となっています。そのため、Embulkの使用を強くお勧めします。"]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"前提条件","__idx":1},"children":["前提条件"]},{"$$mdtype":"Tag","name":"ul","attributes":{},"children":[{"$$mdtype":"Tag","name":"li","attributes":{},"children":["Treasure Dataの基本的な知識"]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"http://www.embulk.org/docs/"},"children":["Embulk"]},"の基本的な知識"]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":["Embulkは、Javaアプリケーションです。Javaがインストールされていることを確認してください。"]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"/products/customer-data-platform/integration-hub/batch/import/bulk-data-import#installing-bulk-data-import"},"children":["Installing Bulk Data Import"]},"の手順に従ってください。"]}]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"シード設定ファイルの作成","__idx":2},"children":["シード設定ファイルの作成"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["お好みのテキストエディタを使用して、入力ファイルと出力Treasure Dataパラメータを定義するEmbulk設定ファイル(例:seed.yml)を作成します。"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["例"]}]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"yaml","header":{"controls":{"copy":{}}},"source":"in:\n  type: file\n  path_prefix: /path/to/files/sample_\nout:\n  type: td\n  apikey: xxxxxxxxxxxx\n  endpoint: api.treasuredata.com\n  database: dbname\n  table: tblname\n  time_column: time\n  mode: replace\n  default_timestamp_format: '%Y-%m-%d %H:%M:%S'\n","lang":"yaml"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["これはサンプルデータです。"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"header":{"controls":{"copy":{}}},"source":"id,account,time,purchase,comment\n1,32864,2015-01-27 19:23:49,20150127,embulk\n2,14824,2015-01-27 19:01:23,20150127,embulk jruby\n3,27559,2015-01-28 02:20:02,20150128,\"Embulk \"\"csv\"\" parser plugin\"\n4,11270,2015-01-29 11:54:36,20150129,NULL\n"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["embulk-local-file-inputで使用可能な追加パラメータの詳細については、",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"http://www.embulk.org/docs/built-in.html#local-file-input-plugin"},"children":["Embulk Local file input"]},"を参照してください。また、embulk-output-tdの詳細については、",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://github.com/treasure-data/embulk-output-td#td-output-plugin-for-embulk"},"children":["TD output plugin for Embulk"]},"を参照してください。"]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"フィールドの推測loadymlの生成","__idx":3},"children":["フィールドの推測(load.ymlの生成)"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Embulkのguessオプションは、",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["seed.yml"]},"を使用してターゲットファイルを読み取り、カラムタイプと設定を自動的に推測し、この情報を含む新しいファイル",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["load.yml"]},"を作成します。"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"bash","header":{"controls":{"copy":{}}},"source":"embulk guess seed.yml -o load.yml\n","lang":"bash"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["生成されたload.ymlファイル。"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"yaml","header":{"controls":{"copy":{}}},"source":"in:\n  type: file\n  path_prefix: /path/to/files/sample_\n  'last_path:': /path/to/files/sample_02.csv\n  parser:\n    charset: UTF-8\n    newline: CRLF\n    type: csv\n    delimiter: ','\n    quote: '\"'\n    escape: '\"'\n    null_string: 'NULL'\n    trim_if_not_quoted: false\n    skip_header_lines: 1\n    allow_extra_columns: false\n    allow_optional_columns: false\n    columns:\n    - {name: id, type: long}\n    - {name: account, type: long}\n    - {name: time, type: timestamp, format: '%Y-%m-%d %H:%M:%S'}\n    - {name: purchase, type: timestamp, format: '%Y%m%d'}\n    - {name: comment, type: string}\nout: {type: td, apikey: xxxxx, endpoint: api.treasuredata.com, database: dbname, table: tblname, time_column: time, mode: replace, default_timestamp_format: '%Y-%m-%d %H:%M:%S'}\n","lang":"yaml"},"children":[]},{"$$mdtype":"Tag","name":"Admonition","attributes":{"type":"info"},"children":[{"$$mdtype":"Tag","name":"p","attributes":{},"children":["ベストプラクティス:load.ymlに「auto_create_table: true」パラメータを追加して、存在しないテーブルが自動的に作成されるようにします。"]}]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["これは、.ymlファイル内のauto_create_tableパラメータのサンプルです。"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"yaml","header":{"controls":{"copy":{}}},"source":"out:\n  type: td\n  apikey: your apikey\n  endpoint: api.treasuredata.com\n  database: dbname\n  table: tblname\n  time_column: created_at\n  auto_create_table: true\n  mode: append\n","lang":"yaml"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["ロードジョブを実行する前に、TDでデータベースとテーブルを作成する必要があります。"]},{"$$mdtype":"Tag","name":"ul","attributes":{},"children":[{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"p","attributes":{},"children":["代替方法:次の場合:"]},{"$$mdtype":"Tag","name":"ul","attributes":{},"children":[{"$$mdtype":"Tag","name":"li","attributes":{},"children":["データベースを追加する必要がある場合"]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":[".ymlファイルにauto_create_tableパラメータを追加せず、テーブルを追加する必要がある場合"]}]}]}]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["次のTDコマンドを実行します:"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"bash","header":{"controls":{"copy":{}}},"source":"td database:create dbname\ntd table:create dbname tblname\n","lang":"bash"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":[{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://console.treasuredata.com/app/databases"},"children":["Treasure コンソール"]},"を使用してデータベースとテーブルを作成することもできます。"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["embulk preview load.yml"]},"コマンドを使用してデータをプレビューできます。カラムタイプやデータが正しくない場合は、",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["load.yml"]},"ファイルを直接編集して、再度プレビューして確認できます。",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["guess"]},"オプションが満足のいく結果を生成しない場合は、",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"http://www.embulk.org/docs/built-in.html#csv-parser-plugin"},"children":["CSV/TSV parser plugin options"]},"を使用して、要件に応じて",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["load.yml"]},"のパラメータを手動で変更できます。"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"bash","header":{"controls":{"copy":{}}},"source":"embulk preview load.yml\n","lang":"bash"},"children":[]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":1,"id":"ロードジョブの実行","__idx":4},"children":["ロードジョブの実行"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["次のコマンドを実行してインポートジョブを発行します:"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"bash","header":{"controls":{"copy":{}}},"source":"embulk run load.yml\n","lang":"bash"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["データのサイズによっては、ジョブが完了するまでに数分から数時間かかる場合があります。"]}]},"headings":[{"value":"Embulk Bulk Import from CSV files","id":"embulk-bulk-import-from-csv-files","depth":1},{"value":"前提条件","id":"前提条件","depth":2},{"value":"シード設定ファイルの作成","id":"シード設定ファイルの作成","depth":2},{"value":"フィールドの推測(load.ymlの生成)","id":"フィールドの推測loadymlの生成","depth":2},{"value":"ロードジョブの実行","id":"ロードジョブの実行","depth":1}],"frontmatter":{"seo":{"title":"Embulk Bulk Import from CSV files"}},"lastModified":"2026-06-02T03:56:21.000Z","pagePropGetterError":{"message":"","name":""}},"slug":"/ja/int/embulk-bulk-import-from-csv-files","userData":{"isAuthenticated":false,"teams":["anonymous"]},"isPublic":true}