Real-Time Label Development - Build a real-time user portrait from scratch

​ Data Access Access to data can be accessed by writing data to Kafka in real time, either directly or through real-time access methods such as oracle's ogg or mysql's binlog ogg Golden Gate (OGG) provides real-time capture, transformation and delivery of transaction data in heterogeneous environments. Through OGG, data from oracle can be writ ...

Posted on Wed, 10 Jun 2020 22:48:11 -0400 by swizenfeld

Add a SpringBootAdmin monitoring

0. Preface In the previous chapters, we first built a project framework, and then built a gateway project using nacos. There are not too many things configured in the gateway project. Now we will build another important project in Spring Cloud microservices spring boot admin 1. Introduction to spring boot admin Spring Boot Admin is used to mon ...

Posted on Wed, 10 Jun 2020 21:43:28 -0400 by ptolomea

Spark streaming reads the database data extracted from Flume by Kafka and saves it in HBase. Hive maps HBase for query

Recently, the company is working on real-time flow processing. The specific requirements are: real-time import of relevant data tables in relational databases (MySQL, Oracle) into HBase, and use Hive mapping HBase for data query. The company uses the big data cluster built by CDH6.3.1~ 1, Configure ...

Posted on Wed, 10 Jun 2020 00:55:16 -0400 by jcleary

Flexible use of Spark window functions lead and lag for online time length statistics

brief introduction In data statistics, it is often necessary to count some time-consuming data, such as online time. Some of these data are better to count, and some are a little bit more troublesome. For example, count the online time of users according to the log in and log out. We can use the window functions lead and lag to complete, which ...

Posted on Tue, 09 Jun 2020 23:56:54 -0400 by dpiland

Spring cloud: service provision and Feign call

In this article, there are three roles: service provider, service consumer, and the protagonist of the previous article, the registration center Eureka (just use the stand-alone version, and the example in this article will also use the stand-alone version of Eureka). The overall process is as follows: ...

Posted on Tue, 09 Jun 2020 23:28:52 -0400 by jabapyth

How can a Tomcat deploy multiple projects? With a variety of solutions and detailed steps!

This article comes from an interview question from an interview many years ago. Brother migrant workers summed it up and shared it with you. I hope it will help you, or maybe it will be used in the interview in the future. First, let's take a look at the common Java Web servers. Tomcat: a Web server provided by Apache organization, which ...

Posted on Tue, 09 Jun 2020 00:17:28 -0400 by Negligence

Flink Sql on Zeppelin -- environment preparation

Environmental preparation summary Why Sql At present, there are many ways to develop Flink. Generally speaking, students write JAVA/SCALA/PYTHON projects and submit them to the cluster for running This method is more flexible, because you can write tasks in the code. Any dimension table JOIN and ...

Posted on Mon, 08 Jun 2020 22:13:41 -0400 by Dave Liebman

A variety of ways of distributed system from the perspective of message middleware

               As an important member of the distributed system, message middleware has many solutions for major companies and open source. At present, the mainstream open source solutions include RabbitMQ, RocketMQ, Kafka, ActiveMQ, etc. News is simple and difficult to say. The simplicity lies in the ease of use, the simplicity o ...

Posted on Sun, 07 Jun 2020 05:00:58 -0400 by PAZII

Zookeeper Watcher process analysis (combined with source code)

summary ZK provides the publish / subscribe function of distributed data. A typical publish / subscribe model system defines a one to many subscription relationship, which enables multiple subscribers to listen to a topic object at the same time. When the status of the topic object changes, all subscribers will be notified. Watcher mechanism is ...

Posted on Sat, 06 Jun 2020 01:32:13 -0400 by kmarsh

Summary of Java Concurrent Programming Practice

premise First, the scene is a hotel-based business.I simplified my business for friends to read easily. Business: After opening a room, a bill will be added, and a room schedule record will be added. The room schedule is mainly for the time used in the room to be non-conflicting.For example: Bill A, use Room 1, use period of time is 220-06-01 1 ...

Posted on Fri, 05 Jun 2020 22:12:46 -0400 by isaacsf