Hbase source code analysis MemStore flush processing (in) 2021SC@SDUSC

preface This article continues to introduce the main process and main details of memory flush on hregon, and how cacheflush handles flush requests. How does cacheflush handle flush requests Through the introduction of how to initialize cacheflush, we know that there are two queues and collections that store flush requests and their h ...

Posted on Fri, 03 Dec 2021 06:12:15 -0500 by JD^

HBase common Shell and Java API operations

1. Common shell namecommandCreate tablecreate 'table name', 'column cluster name 1', 'column cluster name 2'get dataget 'table name', 'row key', 'column cluster name: column name'Add dataput 'table name', 'row key', 'column cluster name: column name', 'value'View all tableslistView table structuredesc 'table name'Delete columndelete 'table ...

Posted on Wed, 01 Dec 2021 04:31:53 -0500 by modcar

2021SC@SDUSC Hbase project code analysis - flush

2021SC@SDUSC          This paper studies how to select an hregon to flush to relieve the pressure of MemStore, and how hregon flush is initiated.         As mentioned above, when flush processes threads, it may call the flush oneforglobalpressure() method. According to certain pol ...

Posted on Tue, 02 Nov 2021 08:28:53 -0400 by robin105

2021SC@SDUSC HBase project analysis: Master startup

2021SC@SDUSC catalogue Master introduction Master overall architecture Introduction to Master components Master startup process source code analysis 2021SC@SDUSC  Master introduction Master overall architecture Introduction to Master components ZooKeeperWatcher: all components that need to know and handle the state changes of ZNode ne ...

Posted on Sat, 30 Oct 2021 01:31:10 -0400 by davidkierz

Record troubleshooting after an hbase outage

hbase downtime troubleshooting The reason is that when using spark to write database data to hbase, it is found that during normal operation, three spark tasks are suddenly suspended. By checking the log, it is found that the program card is in the following position: 2021-10-18 18:23:58,158 INFO jdbc.Utils: Supplied authorities: 192.168.xx.x ...

Posted on Mon, 18 Oct 2021 22:22:13 -0400 by Vibralux

Basic principle and application of Hbase (Part 2)

HBase (medium) 12. Integration of HBase and MapReduce The data in HBase is ultimately stored on HDFS. HBase naturally supports Mr operation. We can directly process the data in HBase through Mr, and MR can directly store the processed results in HBase Requirement: read the data of one table in HBase, and then write the data to another table ...

Posted on Mon, 11 Oct 2021 18:02:49 -0400 by greedyisg00d

Hbase Foundation: Three Synchronization Schemes and Practice of MySQL Data Insertion for Billion Level Data from MySQL to Hbase

This article is about Learning Guide for Big Data Specialists from Zero (Full Upgrade) Supplement to the Base section. This section is titled: Three synchronization schemes and practices for billion-level data from MySQL to Hbase. First, you need to know how to quickly insert MySQL. Then MySQL data insertion will start with the following meth ...

Posted on Sun, 19 Sep 2021 19:52:55 -0400 by ben14

Start University Data Again-Hbase Article-day 55 Brief Talk about Bloom Filter, Hbase Read and Write, Hbase HA and Mapreduce Read and Write Hbase Data

Start University Data Again-Hbase Article-day 55 Brief Talk about Bloom Filter, Hbase Read and Write, Hbase HA and Mapreduce Read and Write Hbase Data Brief Talk about Bloom Filter Summary: _Bloom Filter (Bloom Filter) was proposed by Bloom in 1970. It is actually a long binary vector and a series of random mapping functions. Bloom Fil ...

Posted on Sat, 18 Sep 2021 22:55:02 -0400 by Stuie_b

Day 3: HBase API

API call More commonly used in the work is to call and implement operations similar to HBase shell through HBase API. Environmental preparation IDEA + Maven + HBase <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.or ...

Posted on Sat, 20 Jun 2020 03:37:46 -0400 by Ting

Spark streaming reads the database data extracted from Flume by Kafka and saves it in HBase. Hive maps HBase for query

Recently, the company is working on real-time flow processing. The specific requirements are: real-time import of relevant data tables in relational databases (MySQL, Oracle) into HBase, and use Hive mapping HBase for data query. The company uses the big data cluster built by CDH6.3.1~ 1, Configure ...

Posted on Wed, 10 Jun 2020 00:55:16 -0400 by jcleary