Day 3: HBase API

API call More commonly used in the work is to call and implement operations similar to HBase shell through HBase API. Environmental preparation IDEA + Maven + HBase <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.or ...

Posted on Sat, 20 Jun 2020 03:37:46 -0400 by Ting

Real-Time Label Development - Build a real-time user portrait from scratch

​ Data Access Access to data can be accessed by writing data to Kafka in real time, either directly or through real-time access methods such as oracle's ogg or mysql's binlog ogg Golden Gate (OGG) provides real-time capture, transformation and delivery of transaction data in heterogeneous environments. Through OGG, data from oracle can be writ ...

Posted on Wed, 10 Jun 2020 22:48:11 -0400 by swizenfeld

Flink Sql on Zeppelin -- environment preparation

Environmental preparation summary Why Sql At present, there are many ways to develop Flink. Generally speaking, students write JAVA/SCALA/PYTHON projects and submit them to the cluster for running This method is more flexible, because you can write tasks in the code. Any dimension table JOIN and ...

Posted on Mon, 08 Jun 2020 22:13:41 -0400 by Dave Liebman

PHP regular expression details how to extract characters between two strings (supports single & multiple)

preface: Sometimes it is necessary to extract some characters from a long string, which is often used for variable query and replacement. Scenario: If I have a hive SQL statement, I need to replace the variable {@ date} with a specific date, such as 20200526, select "F" as sex, c.moviename as name, avg(a.rate) as avgrate, co ...

Posted on Wed, 27 May 2020 10:49:36 -0400 by ts2000abc

The running in period of Flink and Hive

There is a lot of feedback from readers. Please refer to the previous article< Hive is finally waiting, Flink >When Flink is deployed and Hive is integrated, there are some bug s and compatibility problems. Although waiting, it is not available. So I added this article as a sister article. review In the previous article, the author use ...

Posted on Mon, 25 May 2020 03:57:37 -0400 by soccerstar_23

Test the query efficiency of mapping Hbase table to Hive table

1, Preparation: 1. Write a program to write 10 million pieces of data into the Hbase table; 2. Map the corresponding Hbase table to Hive table.   Execute the following command in Hive's shell hive> CREATE EXTERNAL TABLE IF NOT EXISTS t_hbase_person_his10(id string, NAME String, salary string,START_DATE string,END_DATE string) ST ...

Posted on Tue, 05 May 2020 00:24:07 -0400 by Hebbs

MySQL installation manual (5.7.20)

MySQL installation manual (5.7.20) 0. Installation summary Clear MYSQL or MariaDB of Linux environment; Download MYSQL RPM package; Installation, configuration and startup; Change the root password; Create database, user and authorization, etc; 1. Clear MYSQL or MariaDB of Linux Environment Check whether mysql is installed in the system: ...

Posted on Thu, 30 Apr 2020 23:37:13 -0400 by Clarkey_Boy

order by, sort by, distribute by, cluster by

1. Data preparation Create person table CREATE TABLE `person`( `id` int, `name` string, `address` string) Add the following data: hive> insert into person values(1, 'lisi', 'beijing'); hive> insert into person values(2, 'zhangsan', 'chengdu'); hive> insert into person values(3, 'wangwu', 'shanghai'); hive> insert into ...

Posted on Mon, 06 Apr 2020 10:08:31 -0400 by dvd420

Differences between Hive internal and external tables

Difference between internal table and external table Tables that are not modified by external are managed table s, and tables that are modified by external are external table s; The data of the internal table is managed by Hive itself, and the external table is managed by HDFS; The data storage location of the internal table is hive.metas ...

Posted on Thu, 19 Mar 2020 11:47:10 -0400 by Jyotsna

Learn sqoop II: the job task of sqoop exporting hadoop data to mysql and sqoop

1.sqoop exports hadoop data to mysql Premise: before exporting data from Hadoop ecosystem to RDBMS database, the target table must exist in the target database. There are three modes of export: 1.1 the default operation is to INSERT data from a file into a table using the INSERT statement. Observe ...

Posted on Fri, 13 Mar 2020 04:01:51 -0400 by eskimowned