Apache Hive is a data warehouse system built on top of Apache Hadoop that facilitates easy data summarization, ad-hoc queries, and the analysis of large datasets stored in various databases and file systems that integrate with Hadoop. Hive offers a simple way to apply structure to large amounts of unstructured data and then perform batch SQL-like queries on that data.
The Treasure Data Hive service (TD Hive) provides batch data processing on data stored in Treasure Data’s data lake, based on Apache Hive. Treasure Data operates compute clusters for running Hive jobs.
The Customer Data Platform (CDP) application uses Hive jobs for some of its internal operations, and you can also run your own Hive jobs. You can submit SELECT or DML queries using the Hive query language (HiveQL), using Treasure Console, API calls, the TD Toolbelt, or from Treasure workflows. The service queues, executes the queries, and returns the results. You can also design your system so that results are delivered to destinations specified in your Result Output.
- TD Hive supports the flexible schema capabilities of the TD platform; schema can be inferred from data loaded into tables or can be set explicitly.
- Treasure Data supports HiveQL semantics, but unlike Apache Hive, Treasure Data allows the user to set and modify the schema at any time. We do not require that a table schema be defined upfront.
You can access Hive using the following:
- Treasure Console
- TD Toolbelt
- REST API
- Open the Treasure Console
- Open the Data Workbench
- Select Queries
- Click New Query button
- Select the query type of Hive, write a query on the editor and click "Run" button
- After you run your query, you can check the query processing log
You can use the Toolbelt command-line (CLI) client to submit Hive queries.
- Ensure that TD Toolbelt is installed.
- Execute
td querycommand like the following:
td query -w -T hive -d testdb "SELECT code, COUNT(1) FROM www_access GROUP BY code"You can use POST /v3/job/issue/hive/:database endpoint to submit Hive queries.