{"templateId":"markdown","sharedDataIds":{"sidebar":"sidebar-sidebars.yaml"},"props":{"metadata":{"markdoc":{"tagList":[]},"redocly_category":"Integrations","type":"markdown"},"seo":{"title":"Legacy Bulk Import From Json File","description":"Treasure Data Product Documentation · Collect and Unify · Segment and Activate · Experiment and Analyze · Decisioning Automate with AI Scale and Trust.","siteUrl":"https://docs.treasuredata.com","lang":"en-US","llmstxt":{"hide":false,"sections":[{"title":"Table of contents","includeFiles":["**/*"],"excludeFiles":[]}],"excludeFiles":[]}},"dynamicMarkdocComponents":[],"compilationErrors":[],"ast":{"$$mdtype":"Tag","name":"article","attributes":{},"children":[{"$$mdtype":"Tag","name":"Heading","attributes":{"level":1,"id":"legacy-bulk-import-from-json-file","__idx":0},"children":["Legacy Bulk Import From Json File"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["This article explains how to import data from JSON files to Treasure Data."]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":1,"id":"install-bulk-loader","__idx":1},"children":["Install Bulk Loader"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Install the ",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://toolbelt.treasuredata.com/"},"children":["Toolbelt"]},", which includes our bulk loader program, on your computer."]},{"$$mdtype":"Tag","name":"ul","attributes":{},"children":[{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://toolbelt.treasuredata.com/win"},"children":["Toolbelt Installer for Windows"]}]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://toolbelt.treasuredata.com/mac"},"children":["Toolbelt Installer for Mac OS X"]}]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"/tools/cli-and-sdks/quickstart"},"children":["Toolbelt Installer for Linux"]}]}]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["After the installation, the ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["td"]}," command will be installed on your computer. Open up the terminal, and type ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["td"]}," to execute the command."]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Make sure you have ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["JAVA"]},"."]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Execute ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["td import:jar_update"]}," to download the up-to-date version of our bulk loader:"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"header":{"controls":{"copy":{}}},"source":"$ td\nusage: td [options] COMMAND [args]\n$ java\nUsage: java [-options] class [args...]\n$ td import:jar_update\nInstalled td-import.jar 0.x.xx into /path/to/.td/java\n"},"children":[]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":1,"id":"authenticate","__idx":2},"children":["Authenticate"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Log into your Treasure account."]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"header":{"controls":{"copy":{}}},"source":"$ td account -f\nEnter your Treasure Data credentials.\nEmail: xxxxx\nPassword (typing will be hidden): \nAuthenticated successfully.\nUse 'td db:create db_name' to create a database.\n"},"children":[]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"importing-data-from-a-json","__idx":3},"children":["Importing Data from a JSON"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["If you have a file called ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["data.json"]}," and its content looks like the following example:"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"header":{"controls":{"copy":{}}},"source":"$ head -n 1 data.json\n{\"host\":\"224.225.147.72\",\"user\":\"-\",\"method\":\"GET\",\"path\":\"/category/electronics\",\"code\":200,\"referer\":\"-\",\"size\":43,\"agent\":\"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)\",\"date_time\":\"2004-03-07 16:05:49\"}\n"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Execute the following command to upload the JSON file:"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"header":{"controls":{"copy":{}}},"source":"$ td db:create my_db\n$ td table:create my_db my_tbl\n$ td import:auto \\\n  --format json \\\n  --time-column date_time \\\n  --time-format \"%Y-%m-%d %H:%M:%S\" \\\n  --auto-create my_db.my_tbl \\\n  ./data.json\n"},"children":[]},{"$$mdtype":"Tag","name":"div","attributes":{"className":"md-table-wrapper"},"children":[{"$$mdtype":"Tag","name":"table","attributes":{"className":"md"},"children":[{"$$mdtype":"Tag","name":"thead","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"th","attributes":{"data-label":""},"children":[]}]}]},{"$$mdtype":"Tag","name":"tbody","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Because ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["td import:auto"]}," executes MapReduce jobs to check the invalid rows, it'll take at least ",{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["1-2 minutes"]},"."]}]}]}]}]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["| --- |"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["In the preceding command, we assumed that:"]},{"$$mdtype":"Tag","name":"ul","attributes":{},"children":[{"$$mdtype":"Tag","name":"li","attributes":{},"children":["The data file is called ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["data.json"]}," and is located in the current directory (hence ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["./data.json"]},")"]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":["The JSON format typically specifies the column names. If it does not, you must specify the column names with the ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["--columns"]}," options (and optionally the column types with ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["--column-types"]}," option), or use the ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["--column-types"]}," for each column in the file."]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":["The time field is called “date_time” and it’s specified with the ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["--time-column"]}," option"]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":["The time format is ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["%Y-%m-%d %H:%M:%S"]}," and it’s specified with the ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["--time-format"]}," option"]}]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"handling-nested-json-records","__idx":4},"children":["Handling Nested JSON Records"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Nested JSON records can be parsed by using Hive’s native ",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF"},"children":["get_json_object UDF"]}," function or Presto’s native ",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://trino.io/docs/423/functions/json.html"},"children":["JSON functions"]},". However, we recommend that you maintain a flat JSON structure to avoid additional CPU performance overhead."]}]},"headings":[{"value":"Legacy Bulk Import From Json File","id":"legacy-bulk-import-from-json-file","depth":1},{"value":"Install Bulk Loader","id":"install-bulk-loader","depth":1},{"value":"Authenticate","id":"authenticate","depth":1},{"value":"Importing Data from a JSON","id":"importing-data-from-a-json","depth":2},{"value":"Handling Nested JSON Records","id":"handling-nested-json-records","depth":2}],"frontmatter":{"seo":{"title":"Legacy Bulk Import From Json File"}},"lastModified":"2026-06-01T09:09:59.000Z","pagePropGetterError":{"message":"","name":""}},"slug":"/int/legacy-bulk-import-from-json-file","userData":{"isAuthenticated":false,"teams":["anonymous"]},"isPublic":true}