Talk about storm's Custom Stream Grouping

order This article mainly studies storm's Custom Stream Grouping. CustomStreamGrouping storm-2.0.0/storm-client/src/jvm/org/apache/storm/grouping/CustomStreamGrouping.java public interface CustomStreamGrouping extends Serializable { /** * Tells the stream grouping at runtime the tasks in the target bolt. This information should be use ...

Posted on Wed, 30 Jan 2019 23:12:15 -0500 by MaxBodine

A MapReduce program example details determine success or failure (1)

Looking at MapReduce recently, I realized that Copy had always been the way to write. Suddenly I want to try writing a simple mr program by myself. Details determine success or failure, do not try not know, try to find the details that are not usually noticed. Below is a program that I wrote in a short time. Note that this one ...

Posted on Wed, 30 Jan 2019 22:48:15 -0500 by Sarok

Details of a MapReduce program example determine success or failure (2): Observing logs and Counter

Below is a map reduce program that calculates the number of a~z single characters in the input file. package wordcount; import java.io.IOException; import org.apache.commons.lang.StringUtils; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.conf.Configured; import org.apache.hadoop.fs.Path; import org.ap ...

Posted on Wed, 30 Jan 2019 21:06:14 -0500 by sazzie

A MapReduce program example details determine success or failure (3): Combiner

In the last article, we wrote a mr program for counting the number of characters in a~z input file. By looking at the Count count of the running log, you can see how much network traffic is between map and reduce. In this article, we will introduce Compiner as a very important component, which can be used to reduce network tran ...

Posted on Wed, 30 Jan 2019 20:27:15 -0500 by walkonet

Simple application of flume

  The main implementation is data source to flume and sink to the corresponding topic, and then the project will write to hbase.   First, the address of the data source is created and then the file is put in.   Now the data source is complete. Then we configure things on flume events.sources = eventsSource events.cha ...

Posted on Wed, 30 Jan 2019 16:30:14 -0500 by blacksmoke26

Upload, delete and query of files on simple Hadoop service hdfs

Upload, delete and query of files on simple Hadoop service hdfs Last time Hadoop Cluster Edition Build completed, then how to upload and download files? Upload, download and query on Linux Start the Hadoop service first Start up master, slave 1, slave 2 virtual machines Start the Hadoop service on master start-all.sh Then jps ...

Posted on Wed, 30 Jan 2019 05:42:15 -0500 by ChompGator

MapReduce-Eclipse Learning Notes

Catalog 1. Configuration file 2. Code examples Code interpretation Errors and Solutions Practice MapReducer in eclipse under windows. eclipse environment collocation 1. Configuration file 1.xml configuration file 2.jar package Jar package under common, mapreduce, tools, yarn and jar package in bin in directory 2. Cod ...

Posted on Wed, 30 Jan 2019 03:24:14 -0500 by Showcase

Talk about direct grouping of storm

order This paper mainly studies direct grouping of storm. direct grouping Direct grouping is a special kind of grouping, which is directly specified by the upstream producer which task downstream receives the tuple it emits. The use of direct grouping has the following steps: 1. The upstream saves the taskId list of the downstream bolt in the p ...

Posted on Wed, 30 Jan 2019 02:51:15 -0500 by redrabbit

Spring MVC uploads files using MultipartFile

1. Basic introduction to MultipartFile MultipartFile is a relatively complete file upload component officially provided by Spring mvc. MultipartFile is an organizational interface whose implementation classes include org.springframework.web.multipart.commons.CommonsMultipartFile org.springframework.mock.web.MockMultipartFile     It is used in ...

Posted on Wed, 30 Jan 2019 02:42:15 -0500 by dicamille

springboot Configuration kafka Producer and Consumer Details

Add a new kafka dependency ar package under the original pom.xml dependency <!--kafka--> <dependency> <groupId>org.springframework.kafka</groupId> <artifactId>spring-kafka</artifactId> <version>1.1.1.RELEASE</version> </dependency> <dependency> <g ...

Posted on Tue, 29 Jan 2019 21:42:16 -0500 by first_lady_the_queen