Hadoop learning - HBASE installation, command line operation and java operation

Before using HABSE, install a zookeeper I used to write https://www.cnblogs.com/wpbing/p/11309761.html   Let's briefly introduce HBASE HBASE is a database, which can provide real-time random reading and writing of data He is a nosql database, which is not structured. He can only do some rough queries, such as the connection query between m ...

Posted on Mon, 23 Mar 2020 20:06:05 -0400 by jdbfitz

Hand-on instructions for building a distributed Java-based crawling system

Personal Blog Visit http://www.x0100.top  1 Overview Without the crawler framework, we tried to implement a distributed crawler system, and we could save the data in different places, such as MySQL, HBase, and so on. This system is developed based on the Interface-oriented coding idea, so it has s ...

Posted on Mon, 09 Mar 2020 22:27:41 -0400 by zero_ZX

Flume development - Custom sink

I. Introduction Sink continuously polls events in the Channel and removes them in batches, and writes them to the storage or index system in batches, or sends them to another Flume Agent. Sink is completely transactional. Before batch deleting data from the Channel, each sink starts a transaction with ...

Posted on Wed, 04 Mar 2020 23:25:40 -0500 by dodgeqwe

Install Sqoop on Linux (and connect mysql test)

Article directory Environment description Download and unzip Change configuration sqoop-env.sh After decompression Modify sqoop-env.sh Configure environment variables Copy mysql driver mysql start View sqoop version Test with mysql Environment description Software Edition operating syste ...

Posted on Tue, 25 Feb 2020 10:36:44 -0500 by FluxNYC

27. Data governance of yiee data operation system of Duoyi Education - atlas deployment and use

Catalog 1, atlas Compilation and packaging 2, atlas installation configuration 1. Compilation environment 2. Compilation steps 3. Installation steps 4. Hive hook configuration 5. Operation test 3, atlas configuration hive hook 4, Introduction to atlas         1,Base Search         ...

Posted on Tue, 11 Feb 2020 03:02:02 -0500 by bensonang

Ambari 2.7.0 + hdp3.1.4.0 installation, hdfs data backup and recovery, hive data backup and recovery, hbase data backup and recovery

Catalog 1 Ambari + HDP offline installation 1.1 INTRODUCTION 1.1.1 introduction to ambari 1.1.2 HDP 1.1.3 HDP-UTILS 1.2 address of ambari official website 1.3 Ambari and HDP Downloads 1.4 system requirements 1.4.1 software requirements 1.5 modify the maximum number of open files 1.6 cluster node plann ...

Posted on Fri, 07 Feb 2020 06:30:42 -0500 by ryanyoungsma

Basic commands of Hbase Shell

Article directory 1, Enter HBase command line 2, Operation of HBase table 3, create 4, View table list 5, View table details desc 6, Modify the definition of table alter 1. Add a column cluster 2. Delete a column cluster 3. Add column cluster hehe and delete column cluster myInfo 4. Clear table trunc ...

Posted on Wed, 05 Feb 2020 05:22:58 -0500 by djddb

How to integrate Hive and HBase

Version Description: HDP: 3.0.1.0 Hive: 3.1.0 HBase: 2.0.0 I. Preface Before learning HBase, we had doubts. Although HBase can store hundreds of millions or billions of rows of data, it is not very friendly for data analysis. It only provides a simple quick query ability based on Key values, and cannot perform a large number of conditional qu ...

Posted on Fri, 31 Jan 2020 14:23:36 -0500 by buck2bcr

Chapter 6 HBase API Operations - Data Operations and Data Migration

Previous: Chapter 6 HBase API Operations (2) 1. Encapsulation of data Encapsulate data using multithreaded thread security First, create a tool class: HbaseUtil Code implementation: package studey.bigdate.util; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.hbase.HBaseConfig ...

Posted on Fri, 24 Jan 2020 02:00:18 -0500 by kuma

Flink reads Kafka data Sink to MySQL and HBase databases

Flink reads Kafka data Sink to MySQL and HBase databases Flink transfers the stream data Sink to the database. Generally, it needs to implement its own custom Sink. The following example demonstrates the Sink to MySQL and HBase examples. Insert a code slice here import java.util.Properties import org.ap ...

Posted on Mon, 20 Jan 2020 09:21:47 -0500 by like_php