{"templateId":"markdown","sharedDataIds":{"sidebar":"sidebar-@l10n/ja/sidebars.yaml"},"props":{"metadata":{"markdoc":{"tagList":[]},"type":"markdown"},"seo":{"title":"CSV、TSV、LTSVフォーマットを使用したログのインポート","description":"Treasure Data Product Documentation · Collect and Unify · Segment and Activate · Experiment and Analyze · Decisioning Automate with AI Scale and Trust.","siteUrl":"https://docs.treasuredata.com","lang":"en-US","llmstxt":{"hide":false,"sections":[{"title":"Table of contents","includeFiles":["**/*"],"excludeFiles":[]}],"excludeFiles":[]}},"dynamicMarkdocComponents":[],"compilationErrors":[],"ast":{"$$mdtype":"Tag","name":"article","attributes":{},"children":[{"$$mdtype":"Tag","name":"Heading","attributes":{"level":1,"id":"csvtsvltsvフォーマットを使用したログのインポート","__idx":0},"children":["CSV、TSV、LTSVフォーマットを使用したログのインポート"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Treasure Agent (td-agent) から CSV、TSV、LTSV 形式のログをインポートして、アクセスログをクラウドに継続的にインポートできます。"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["td-agentはログローテーションを処理します"]},"。td-agentはログの最終位置を記録し、td-agentプロセスがダウンした場合でも各行が正確に1回だけ読み取られることを保証します。ただし、情報はファイルに保持されるため、ファイルが破損すると「exactly once」の保証は無効になります。"]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"前提条件","__idx":1},"children":["前提条件"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["td-agentは",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"http://fluentd.org/"},"children":["Fluentdプロジェクト"]},"の下にあります。td-agentはTreasure Data用のカスタムプラグインでFluentdを拡張しています。"]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"td-agentのインストール","__idx":2},"children":["td-agentのインストール"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["アプリケーションサーバーにtd-agentをインストールします。td-agentは、あらゆる種類の時系列データのストリーミングアップロード専用のデーモンプログラムです。td-agentはFluentdプロジェクトの下にあります。td-agentはTreasure Data用のカスタムプラグインでFluentdを拡張しています。"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["td-agentをセットアップするには、以下の記事を参照してください。Linuxシステム用のdeb/rpmパッケージを提供しています。"]},{"$$mdtype":"Tag","name":"div","attributes":{"className":"md-table-wrapper"},"children":[{"$$mdtype":"Tag","name":"table","attributes":{"className":"md"},"children":[{"$$mdtype":"Tag","name":"thead","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"th","attributes":{"data-label":""},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["環境"]}]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":""},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["参照先"]}]}]}]},{"$$mdtype":"Tag","name":"tbody","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["MacOS X"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://docs.fluentd.org/installation/install-by-dmg"},"children":["Installing td-agent on MacOS X"]}]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Ubuntu System"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://docs.fluentd.org/installation/install-by-deb"},"children":["Installing td-agent for Debian and Ubuntu"]}]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["RHEL / CentOS System"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://docs.fluentd.org/installation/install-by-rpm"},"children":["Installing td-agent for Redhat and CentOS"]}]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["AWS Elastic Beanstalk"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://github.com/treasure-data/elastic-beanstalk-td-agent"},"children":["Installing td-agent on AWS Elastic Beanstalk"]}]}]}]}]}]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"td-agentconfの変更","__idx":3},"children":["td-agent.confの変更"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["apikey"]},"オプションを設定して認証キーを指定します。APIキーはTreasure コンソールから確認できます。"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["/etc/td-agent/td-agent.conf"]},"にアクセスして、",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["apikey"]},"オプションを設定します。"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":[{"$$mdtype":"Tag","name":"em","attributes":{},"children":["YOUR_API_KEY"]},"は、あなたのAPIキー文字列に置き換えてください。"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"conf","header":{"controls":{"copy":{}}},"source":"# Tailing the CSV formatted Logs\n<source>\n  type tail\n  format csv\n  path /path/to/log/foo.csv\n  pos_file /var/log/td-agent/foo.pos\n  tag td.production.foo\n\n  keys key1, key2, key3\n  time_key key3\n</source>\n\n# Tailing the TSV formatted Logs\n<source>\n  type tail\n  format tsv\n  path /path/to/log/bar.tsv\n  pos_file /var/log/td-agent/bar.pos\n  tag td.production.bar\n\n  keys key1, key2, key3\n  time_key key3\n</source>\n\n# Tailing the LTSV formatted Logs\n<source>\n  type tail\n  format ltsv\n  path /path/to/log/buz.ltsv\n  pos_file /var/log/td-agent/buz.pos\n  tag td.production.buz\n\n  time_key time_field_name\n</source>\n\n# Treasure Data Input and Output\n<match td.*.*>\n  type tdlog\n  endpoint api.treasuredata.com\n  apikey YOUR_API_KEY\n  auto_create_table\n  buffer_type file\n  buffer_path /var/log/td-agent/buffer/td\n  use_ssl true\n</match>\n","lang":"conf"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["以下の行が配置されたら、agentを再起動します。"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"header":{"controls":{"copy":{}}},"source":"$ sudo /etc/init.d/td-agent restart\n"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["td-agentはファイルをtailし、ログをバッファリングし(",{"$$mdtype":"Tag","name":"em","attributes":{},"children":["var/log/td-agent/buffer/td"]},")、自動的にログをクラウドにアップロードします。"]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":1,"id":"データインポートの確認","__idx":4},"children":["データインポートの確認"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["SIGUSR1シグナルを送信すると、td-agentのバッファがフラッシュされ、アップロードがすぐに開始されます。"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"header":{"controls":{"copy":{}}},"source":"# append new records to the logs\n$ ...\n\n# flush the buffer\n$ kill -USR1 `cat /var/run/td-agent/td-agent.pid`\n"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["データが正常にアップロードされたことを確認するには、次のように",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["td tables"]},"コマンドを実行します。"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"header":{"controls":{"copy":{}}},"source":"$ td tables\n+------------+------------+------+-----------+\n| Database   | Table      | Type | Count     |\n+------------+------------+------+-----------+\n| production | foo        | log  | 1         |\n| production | bar        | log  | 3         |\n| production | buz        | log  | 5         |\n+------------+------------+------+-----------+\n"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["正しく動作しない場合は、",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["/var/log/td-agent.log"]},"を確認してください。",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["td-agent:td-agent"]},"がログを読み取る権限を持っている必要があります。"]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"次のステップ","__idx":5},"children":["次のステップ"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["従来のRDBMSよりも柔軟なスキーマメカニズムを提供しています。クエリには、Hive Query Languageを活用しています。"]},{"$$mdtype":"Tag","name":"ul","attributes":{},"children":[{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"/products/customer-data-platform/data-workbench/databases/schema-management"},"children":["Schema Management"]}]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"/products/customer-data-platform/data-workbench/queries"},"children":["Hive Query Language"]}]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"/tools/cli-and-sdks/api-references-and-tool-references"},"children":["Programmatic Access with REST API and its Bindings"]}]}]}]},"headings":[{"value":"CSV、TSV、LTSVフォーマットを使用したログのインポート","id":"csvtsvltsvフォーマットを使用したログのインポート","depth":1},{"value":"前提条件","id":"前提条件","depth":2},{"value":"td-agentのインストール","id":"td-agentのインストール","depth":2},{"value":"td-agent.confの変更","id":"td-agentconfの変更","depth":2},{"value":"データインポートの確認","id":"データインポートの確認","depth":1},{"value":"次のステップ","id":"次のステップ","depth":2}],"frontmatter":{"seo":{"title":"CSV、TSV、LTSVフォーマットを使用したログのインポート"}},"lastModified":"2026-06-01T09:34:27.000Z","pagePropGetterError":{"message":"","name":""}},"slug":"/ja/int/logs-import-using-csv-tsv-and-ltsv-formats","userData":{"isAuthenticated":false,"teams":["anonymous"]},"isPublic":true}