Hadoop advanced level
YARN: Hadoop resource scheduling system
What is YARN
Apache Hadoop YARN(Yet Another Resource Negotiator) is a sub project of Hadoop, which is introduced to separate Hadoop 2.0 resource management and computing components.YARN has enough universality, and customers support other distributed computing modes.
Posted on Sun, 05 Dec 2021 18:08:55 -0500 by snpo123
Link to Topic 1 Anhui Province Big Data and Artificial Intelligence Application Competition 2021-Topic Answer of MapReduce
Title: Use MapReduce to count each mobile phone number in calls.txt, call duration and number of calls, call duration, call number, and output format as mobile phone number, call duration, call number, call duration and ca ...
Posted on Fri, 26 Nov 2021 16:46:57 -0500 by haddydaddy
When the dead wood is in spring, it is not luxuriant, Young and cherish the people around the mirror.
Write in front
Today's exam was fairly smooth, but today's me, my heart is inexplicably uncomfortable, uncomfortable? If you feel uncomfortable, please write a blog. 3A requires the implementation of distributed kv server based on Raft
Posted on Thu, 25 Nov 2021 15:03:38 -0500 by bacarudaguy
0 - Preface
Hive SQL's execution plan describes the overall outline of the actual execution of SQL. Through the execution plan, you can understand the execution logic of the SQL program when it is converted into the corresponding computing engine. If you master the execution logic, you can better grasp the bottleneck of the program, so ...
Posted on Fri, 19 Nov 2021 18:11:54 -0500 by XxDeadmanxX
1. MapReduce definition
MapReduce is a programming framework for distributed computing programs and the core framework for users to develop "Hadoop based data analysis applications". The core function of MapReduce is to integrate the business logic code written by the user and its own default components into a complete distributed co ...
Posted on Thu, 18 Nov 2021 10:48:22 -0500 by Meltdown
The friend recommendation function is simply a demand to predict whether two people know each other and recommend them as friends.
2, Train of thought
For two users who are not friends, the more common friends they have, the more likely they are to know each other.
For example, the raw data are as follows
Tom Cat Hello Hado ...
Posted on Thu, 11 Nov 2021 03:44:53 -0500 by flashicon
Brief introduction of research content
Last week, we completed the analysis of the core code in org.apache.hadoop.mapreduce.Counters. This week, we will continue the analysis from org.apache.hadoop.mapreduce.ID.
org.apache.hadoop.mapreduce.ID source code analysis
import java.io.DataInput ...
Posted on Wed, 10 Nov 2021 19:51:37 -0500 by Roddy87
MongoDB Map Reduce
MAP REDUCE is a computing model, which simply means that a large number of work (data) are decomposed (MAP) and executed, and then the results are combined into the final result (REDUCE).
The map reduce provided by MongoDB is very flexible and practical for large-scale data analysis.
The following is t ...
Posted on Thu, 28 Oct 2021 02:25:17 -0400 by rekha
Code case based on IDEA environment
#Simply write a wordcount text
vim word count.txt
hdfs dis -midair -p /user/root/input
#Send to virtual machine, run
hadoop jar ......
Serialization case of Hadoop
demand ·Count the total uplink traffic / downlink traffic / status / return total traffic consumed by the mobile phone number &middo ...
Posted on Tue, 26 Oct 2021 09:42:32 -0400 by shaitand