Day 3: HBase API

API call More commonly used in the work is to call and implement operations similar to HBase shell through HBase API. Environmental preparation IDEA + Maven + HBase <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.or ...

Posted on Sat, 20 Jun 2020 03:37:46 -0400 by Ting

Spark streaming reads the database data extracted from Flume by Kafka and saves it in HBase. Hive maps HBase for query

Recently, the company is working on real-time flow processing. The specific requirements are: real-time import of relevant data tables in relational databases (MySQL, Oracle) into HBase, and use Hive mapping HBase for data query. The company uses the big data cluster built by CDH6.3.1~ 1, Configure ...

Posted on Wed, 10 Jun 2020 00:55:16 -0400 by jcleary

Several ways of reading and writing spark articles by HBase

1. Overview of how HBase is read and written It is mainly divided into: The way pure Java API s read and write HBase s; How Spark reads and writes HBase s; How Flink reads and writes HBase s; HBase is read and written through Phoenix; The first is the more original and efficient operation mode provided by HBase itself. The second and ...

Posted on Sat, 09 May 2020 23:05:48 -0400 by israfel

Test the query efficiency of mapping Hbase table to Hive table

1, Preparation: 1. Write a program to write 10 million pieces of data into the Hbase table; 2. Map the corresponding Hbase table to Hive table.   Execute the following command in Hive's shell hive> CREATE EXTERNAL TABLE IF NOT EXISTS t_hbase_person_his10(id string, NAME String, salary string,START_DATE string,END_DATE string) ST ...

Posted on Tue, 05 May 2020 00:24:07 -0400 by Hebbs

Installation of JDK, Hadoop, Hbase and GUI on Linux Server (Ubuntu 18.04)

Article Directory 0. Preparations 1. Get Ali Cloud ECS Server 2. Create Hadoop users(viii) 3. Configure SSH passwordless login 4. Install JDK 4.1 Method 1:_Manual installation (recommended)_) 4.1.1 Process 4.1.2 Download Linux JDK8 on PC 4.1.3 Use WinSCP to upload compressed packag ...

Posted on Fri, 17 Apr 2020 02:57:09 -0400 by zgkhoo

HBase Shell Top 10 Fancy Plays

Foreword: Everything must be done properly before it can be done properly. Today we will introduce the top ten HBase Shell fancy tools, which can be tried in daily operation and maintenance. 1. Interactive mode This is the Shell command line that we use most often. $ hbase shell hbase(main):001:0> list 2. Non-interactive mode $ ...

Posted on Thu, 16 Apr 2020 21:54:53 -0400 by trube

Reader's Letter | Just after the cluster of HBases was built, Phoenix started and HBases completely collapsed. What was the cause? (Resolved)

Foreword: Some friends and friends have discussed some problems with me before, and I think these problems are very valuable. So I want to open a question and answer column in this public number to facilitate technical exchange and sharing. The column name is "Letter from Reader".If you encounter a problem that is difficult to solve b ...

Posted on Mon, 06 Apr 2020 23:53:14 -0400 by Formula

Probe into ReadType of scan in HBase

background knowledge Linux level linux provides different functions for reading files, with the following references: When a random read-write operation is performed on the same file handle (under Windows) or file descriptor (under Linux), there are two steps to locate and read/write the file pointer, but since this is not an atomic operation, ...

Posted on Sat, 04 Apr 2020 17:10:10 -0400 by MrAlaska

Hadoop learning - HBASE installation, command line operation and java operation

Before using HABSE, install a zookeeper I used to write https://www.cnblogs.com/wpbing/p/11309761.html   Let's briefly introduce HBASE HBASE is a database, which can provide real-time random reading and writing of data He is a nosql database, which is not structured. He can only do some rough queries, such as the connection query between m ...

Posted on Mon, 23 Mar 2020 20:06:05 -0400 by jdbfitz

Hand-on instructions for building a distributed Java-based crawling system

Personal Blog Visit http://www.x0100.top  1 Overview Without the crawler framework, we tried to implement a distributed crawler system, and we could save the data in different places, such as MySQL, HBase, and so on. This system is developed based on the Interface-oriented coding idea, so it has s ...

Posted on Mon, 09 Mar 2020 22:27:41 -0400 by zero_ZX