Find Common Friends - Data Mining - Scala Edition

Hello, there are many language implementations on the Internet about the algorithm of "Find common friends". When I have time today, I have studied the writing of the Scala algorithm myself. The complete code can refer to the Git address:https://github.com/benben7466/SparkDemo/blob/master/spark-test/src/main/scala/testCommendFriend.s ...

Posted on Sat, 04 Jul 2020 10:58:46 -0400 by EGNJohn

Building analysis data lake with Apache Spark and Apache Hudi

Welcome to WeChat official account: ApacheHudi 1. Introduction Most modern data lakes are built on some kind of distributed file system (DFS), such as HDFS or cloud based storage, such as AWS S3. One of the basic principles to follow is the "write once read many" access model for files. This ...

Posted on Sun, 14 Jun 2020 22:39:40 -0400 by tozanni

akka-typed - PersistentActor has EventSourcedBehavior

There is no Persistent Actor in akka-typed.It was replaced by an actor with EventSourcedBehavior, an actor that specifically supports the EventSource pattern.The principles and functions of EventSource have been described in detail in previous blogs, and will not be repeated here.This article begins with the specific application of Eventsourced ...

Posted on Tue, 02 Jun 2020 20:45:53 -0400 by richclever

Akka typed - typed actor communication mode and protocol

Akka system is a distributed message driven system. Akka applications are composed of a group of actors who are responsible for different operations. Each actor is passively waiting for some external message to drive its own jobs. So, popular point description: akka application is a system that a group of actors send messages to each other, and ...

Posted on Fri, 29 May 2020 02:04:37 -0400 by Masca

Akka typed - actor lifecycle management

Akka typed actors are different from akka classic in terms of life cycle management, such as creation, enabling, state transition, deactivation and monitoring. In this article, we will introduce akka typed actor lifecycle management. Each kind of actor forms a template by defining its behavior attribute behavior, and then generates an actor ins ...

Posted on Wed, 27 May 2020 08:44:11 -0400 by Hafkas

Learn scala together -- control structure and function

Conditional expression Unlike java/c + +, if / else syntax in scala has a return value, which is the value of the last expression after if / else. val s:String = if(1>0) "yes" else "no" // The return value is the common super type of the return value of the if/else expression. Here is String. val value: Any = if (1 > 0) "yes" else 1 // I ...

Posted on Tue, 19 May 2020 07:00:39 -0400 by ale1981

RDD common operators of spark notes

hello everyone! Here are the saprk operator notes I learned during the epidemic holiday. I just spent the whole afternoon sorting them out and sharing them with you! It's not easy to code. If it helps you, remember to like it! Article catalog 1, spark action operator 2, spark single value type 3, spark double value type 4, spa ...

Posted on Mon, 18 May 2020 04:02:17 -0400 by Mattyspatty

Scala_ Common functions

1, If Else… val x = 5 val s = if (x > 0) 1 else -1 // s type is Int val m = if (x > 0) "positive" else -1 // Public super type Any with s type String and Int // In Scala, each expression should have a value. If the else part is missing, the following statement is equivalent to: if (x > 0) 1 else(), which returns a null ...

Posted on Fri, 15 May 2020 11:08:22 -0400 by mrjap1

search (13) - elastic4s histograms: aggregate histogram

In aggregate group statistics, we will face two types of group elements: continuous type such as time, natural number, discrete type such as location, product, etc. Discrete data itself represents different groups, but continuous data needs to be manually segmented at equal intervals. Here is an example of aggregation by price segment: POST /c ...

Posted on Fri, 15 May 2020 10:15:27 -0400 by carrotcake1029

search (12) - elastic4s aggregate = bucket + Measure

This article introduces the aggregation function of ES. Aggregation is the main tool for visualizing index data into readable and useful data. Aggregation consists of bucket and metrics. The so-called bucket is the group by of SQL, as follows: GET /cartxns/_search { "size" : 2, "aggs": { "color": { "terms": {"field": "color.keywo ...

Posted on Tue, 12 May 2020 12:03:27 -0400 by spicey