Action operator of rdd

An action operator is an operator that triggers an action. Triggering an action means real calculation data. collect Collect is to collect data from the executor side to the driver side. For example, a simple wordcount program: object CollectAction { def main(args: Array[String]): Unit = { val conf: SparkConf = new SparkConf().set ...

Posted on Thu, 14 Oct 2021 20:09:17 -0400 by pkSML

byKey series of rdd operators

There are some operators of xxxByKey in spark. Let's see. groupByKey explain Suppose we want to group some string lists: object GroupByKeyOperator { def main(args: Array[String]): Unit = { val conf = new SparkConf().setMaster("local[*]").setAppName("RDD") val context = new SparkContext(conf) val rdd: RDD[String] = contex ...

Posted on Thu, 14 Oct 2021 19:38:43 -0400 by ErnesTo

Careful analysis of Netty source code (emphasis! Reprint the article!!!)

Emphasis: This article is a reprint of the article, and the link is: javadoop , this big guy's articles are of great value. Click the original text to see the experience more register operation of Channel After the foreshadowing in front, we already have a certain foundation. Let's start to rub together the contents we learned in front. In th ...

Posted on Sat, 02 Oct 2021 19:21:40 -0400 by benwestgarth

QtPromise source code analysis CPP template meta programming

catalogue Promise concept QtPromise open source template library Design patterns used in QtPromise template library   one   Build patterns in QtPromise   two   Decoration mode in QtPromise Metaprogramming skills used in QtPromise template library 1. Template meta function two   Full specialization & partia ...

Posted on Tue, 28 Sep 2021 15:11:21 -0400 by trp

Analysis of MyBatis delayed loading principle

🏆 Analysis of MyBatis delayed loading principle 1. Introduction and use of delayed loading This paper will analyze the principle of delayed loading (lazy loading) provided by MyBatis. 1.1 what is delayed loading? In short, delayed loading is to load when data is needed, and not when data is not needed. Suppose there are two tables in the d ...

Posted on Mon, 20 Sep 2021 13:04:14 -0400 by qaokpl