Use and installation of sqoop
SQOOP installation and use
SQOOP installation
1. Upload and unzip
Installation package download address http://archive.apache.org/dist/sqoop/1.4.7/
tar -zxvf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz -C /usr/local/soft/
2. Modify folder name
mv sqoop-1.4.7.bin__hadoop-2.6.0/ sqoop-1.4.7
3. Modify profile
# Switch to the sqoop profile ...
Posted on Mon, 06 Dec 2021 17:35:45 -0500 by CoreyR
3, Hive's DML data operation
3.1 data import
3.1.1 loading data into the table (Load)
(1) Grammar load data [local] inpath 'path of data' [overwrite] into table student [partition (partcol1=val1,...)]; load data: indicates loading data Local: indicates loading data from local to hive table; Otherwise, load data from HDFS to hive table inpath: indicates the path to load d ...
Posted on Sun, 05 Dec 2021 02:14:14 -0500 by shinyo
See how FusionInsight Spark supports the multi instance feature of JDBC server
Absrtact: the HA scheme with multi master instance mode can not only avoid the problem of service interruption during active / standby switching and realize no or less service interruption, but also improve the concurrency ability by horizontally expanding the cluster.
This article is shared from Huawei cloud community< FusionInsight Spar ...
Posted on Wed, 01 Dec 2021 12:47:39 -0500 by lurius
Hive processes Json data
Data processing in Json format
Json data format is a commonly used data format, such as embedded point data, business end data and front and rear end calls. Therefore, it is necessary for us to learn the processing method of this data format
Prepare data
cat json.data
{"movie":"1193","rate":"5","timeStamp":"978300760","uid":"1"}
{"movie":"661", ...
Posted on Wed, 01 Dec 2021 09:41:51 -0500 by coderb
Hive common functions, window functions, UDF, UDTF
1. System built-in function
View the functions provided by the systemhive> show functions;
Displays the usage of the built-in function//View the usage of the upper function
hive> desc function upper;
Displays the usage of the built-in function in detail//View the detailed usage of the upper function
hive> desc function extended ...
Posted on Sun, 28 Nov 2021 12:21:55 -0500 by filteredhigh
The differences between hive query statement and mysql
These are the two tables emp and Dept often used in online database examples. The first is dept and the second is emp
For ease of viewing, I added the field name of each column to it
1.hive does not support some sub queries
For example, this topic: query the names of employees with higher wages than SMITH
mysql can end ...
Posted on Sun, 28 Nov 2021 08:59:54 -0500 by mikes127
Python summary (primary data structure dictionary and collection)
Dictionary and collection Basics
What is a dictionary and what is a collection? A dictionary is a set of elements composed of key and value pairing. In Python 3.7 + dictionary, it is determined to be ordered (Note: in 3.6, dictionary order is an implementation detail, which is officially called a language feature in 3.7, so it cannot be 100% g ...
Posted on Mon, 22 Nov 2021 04:30:41 -0500 by horstuff
hiveSQL execution plan (explain the most detailed in the whole network!!)
0 - Preface
Hive SQL's execution plan describes the overall outline of the actual execution of SQL. Through the execution plan, you can understand the execution logic of the SQL program when it is converted into the corresponding computing engine. If you master the execution logic, you can better grasp the bottleneck of the program, so ...
Posted on Fri, 19 Nov 2021 18:11:54 -0500 by XxDeadmanxX
Hive Foundation (installation)
1, What is Hive
Hive is an essential tool in Hadop ecosystem. It can map the structured data stored in HDES into a table in the database, and provides an SQL dialect to query it. These SQL statements will eventually be translated into MapReduce program for execution. The essence of Hive is a framework generated to simplify users' writing M ...
Posted on Thu, 18 Nov 2021 09:03:54 -0500 by TRUSTINWEB
Hive calculates the maximum number of consecutive login days
catalogue1, Background2, Algorithm1. Step 1: sort2. Step 2: make the date difference between the second column and the third column3. Step 3: group and sum according to the second column4. Step 4: find the maximum number of times3, Extension (maximum trading days of stocks)
Brother Qiang said that he found the wealth password and recently devel ...
Posted on Wed, 03 Nov 2021 20:55:17 -0400 by seanrock