Find Common Friends - Data Mining - Scala Edition
Hello, there are many language implementations on the Internet about the algorithm of "Find common friends". When I have time today, I have studied the writing of the Scala algorithm myself.
The complete code can refer to the Git address:https://github.com/benben7466/SparkDemo/blob/master/spark-test/src/main/scala/testCommendFriend.s ...
Posted on Sat, 04 Jul 2020 10:58:46 -0400 by EGNJohn
Building analysis data lake with Apache Spark and Apache Hudi
Welcome to WeChat official account: ApacheHudi
1. Introduction
Most modern data lakes are built on some kind of distributed file system (DFS), such as HDFS or cloud based storage, such as AWS S3. One of the basic principles to follow is the "write once read many" access model for files. This ...
Posted on Sun, 14 Jun 2020 22:39:40 -0400 by tozanni
akka-typed - PersistentActor has EventSourcedBehavior
There is no Persistent Actor in akka-typed.It was replaced by an actor with EventSourcedBehavior, an actor that specifically supports the EventSource pattern.The principles and functions of EventSource have been described in detail in previous blogs, and will not be repeated here.This article begins with the specific application of Eventsourced ...
Posted on Tue, 02 Jun 2020 20:45:53 -0400 by richclever
Akka typed - typed actor communication mode and protocol
Akka system is a distributed message driven system. Akka applications are composed of a group of actors who are responsible for different operations. Each actor is passively waiting for some external message to drive its own jobs. So, popular point description: akka application is a system that a group of actors send messages to each other, and ...
Posted on Fri, 29 May 2020 02:04:37 -0400 by Masca
Akka typed - actor lifecycle management
Akka typed actors are different from akka classic in terms of life cycle management, such as creation, enabling, state transition, deactivation and monitoring. In this article, we will introduce akka typed actor lifecycle management.
Each kind of actor forms a template by defining its behavior attribute behavior, and then generates an actor ins ...
Posted on Wed, 27 May 2020 08:44:11 -0400 by Hafkas
Learn scala together -- control structure and function
Conditional expression
Unlike java/c + +, if / else syntax in scala has a return value, which is the value of the last expression after if / else.
val s:String = if(1>0) "yes" else "no"
// The return value is the common super type of the return value of the if/else expression. Here is String.
val value: Any = if (1 > 0) "yes" else 1
// I ...
Posted on Tue, 19 May 2020 07:00:39 -0400 by ale1981
RDD common operators of spark notes
hello everyone! Here are the saprk operator notes I learned during the epidemic holiday. I just spent the whole afternoon sorting them out and sharing them with you! It's not easy to code. If it helps you, remember to like it!
Article catalog
1, spark action operator
2, spark single value type
3, spark double value type
4, spa ...
Posted on Mon, 18 May 2020 04:02:17 -0400 by Mattyspatty
Scala_ Common functions
1, If Else…
val x = 5
val s = if (x > 0) 1 else -1 // s type is Int
val m = if (x > 0) "positive" else -1 // Public super type Any with s type String and Int
// In Scala, each expression should have a value. If the else part is missing, the following statement is equivalent to: if (x > 0) 1 else(), which returns a null ...
Posted on Fri, 15 May 2020 11:08:22 -0400 by mrjap1
search (13) - elastic4s histograms: aggregate histogram
In aggregate group statistics, we will face two types of group elements: continuous type such as time, natural number, discrete type such as location, product, etc. Discrete data itself represents different groups, but continuous data needs to be manually segmented at equal intervals. Here is an example of aggregation by price segment:
POST /c ...
Posted on Fri, 15 May 2020 10:15:27 -0400 by carrotcake1029
search (12) - elastic4s aggregate = bucket + Measure
This article introduces the aggregation function of ES. Aggregation is the main tool for visualizing index data into readable and useful data. Aggregation consists of bucket and metrics.
The so-called bucket is the group by of SQL, as follows:
GET /cartxns/_search
{
"size" : 2,
"aggs": {
"color": {
"terms": {"field": "color.keywo ...
Posted on Tue, 12 May 2020 12:03:27 -0400 by spicey