11-flink-1.10.1- Flink window API

catalogue 1. Window API concept 2. Window API type   2.1 Time Window 2.2 counting window 3 window function   four   window api combination code 1. Window API concept   Half of the real streams are unbounded. How to deal with unbounded streams?Unbounded streams can be segmented to obtain limited data sets for proce ...

Posted on Fri, 24 Sep 2021 11:22:42 -0400 by willl

Big data Flink asynchronous IO

1 Introduction 1.1 requirements for asynchronous IO operation https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/stream/operators/asyncio.html Async I/O is a very popular feature contributed by Alibaba to the community. It was introduced in version 1.2. The main purpose is to solve the problem that network delay becomes th ...

Posted on Mon, 13 Sep 2021 21:10:56 -0400 by jimbo_head

Big data Flink performance optimization

1 History Server performance optimization Flynk's HistoryServer is mainly used to store and view the history of tasks. See the official website for specific information https://ci.apache.org/projects/flink/flink-docs-release- 1.12/deployment/advanced/historyserver.html # Directory to upload completed jobs to. Add this directory to the l ...

Posted on Mon, 13 Sep 2021 16:20:09 -0400 by ayok

Big data Flink Transformation

1. Official website API list https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/stream/operators/ Overall, operations on streaming data can be divided into four categories. The first type is the operation for a single record, such as filtering out unqualified records (Filter operation), or making a conversion for each reco ...

Posted on Wed, 08 Sep 2021 19:09:45 -0400 by bateman

Big data Flink Source

1 predefined Source 1.1 set based Source ⚫ API It is generally used when learning tests and making up data 1. Env.fromelements (variable parameters); 2. Env.fromcolletion (various sets); 3. Env.generatesequence (start, end); 4. Env.fromsequence (start, end); ⚫ Code demonstration: package cn.oldlu.source; import org.apache.flink.api ...

Posted on Wed, 08 Sep 2021 16:53:14 -0400 by erick_w

Flink streaming API

5. Flink streaming API Implicit transformation in Scala programming import org.apache.flink.streaming.api.scala._ import org.apache.flink.table.api._ import org.apache.flink.table.api..scala._ 5.1,Environment getExecutionEnvironment Create an execution environment that represents the context of the current executing program.If the ...

Posted on Mon, 06 Sep 2021 20:29:53 -0400 by phpdragon

Detailed explanation of the source code of the new back pressure monitoring index in Flink1.13

Back pressure (also known as back pressure) is one of the most important monitoring indicators of flink. It can intuitively reflect whether the downstream task can process the received data in time. For details of back pressure, please refer to the official website: Monitoring back pressure. Note: 1.13 the introduction of back pressure on the ...

Posted on Fri, 03 Sep 2021 20:47:51 -0400 by phphelpme

Flink window Basics

Flink window Foundation (I) 1. Window concept: Streaming computing is a data processing engine designed to process infinite data sets, which refers to a growing essentially infinite data set, and Window is a means to cut infinite data into finite blocks. 2. Classification of windows: Time driven: time-based windows **Time driven → tum ...

Posted on Thu, 02 Sep 2021 04:36:42 -0400 by tlawless