SpringBoot Dubbo and Zookeeper integration

What is a distributed system?

In the Book Principles and models of distributed systems, it is defined as follows: "a distributed system is a collection of several independent computers, which are like a single related system to users";

Distributed system is a system composed of a group of computer nodes that communicate through the network and coordinate to complete common tasks. The emergence of distributed system is to use cheap and ordinary machines to complete the computing and storage tasks that a single computer cannot complete. Its purpose is to use more machines to process more data.

distributed system is a software system based on network.

First of all, it should be clear that we need to consider distributed systems only when the processing capacity of a single node can not meet the increasing computing and storage tasks, and the improvement of hardware (adding memory, adding disk and using better CPU) is too high to pay off, and the application can not be further optimized. Because the problems to be solved by the distributed system itself are the same as those of the single machine system. Due to the multi node and network communication topology of the distributed system, many problems that the single machine system does not have will be introduced. In order to solve these problems, more mechanisms and agreements will be introduced to bring more problems...

Dubbo document

With the development of the Internet and the continuous expansion of the scale of website applications, the conventional vertical application architecture can not cope with it. The distributed service architecture and mobile computing architecture are imperative, and a governance system is urgently needed to ensure the orderly evolution of the architecture.

There is such a picture on Dubbo's official website

Single application architecture

When the website traffic is very small, only one application is needed to deploy all functions together to reduce the deployment nodes and costs. At this time, the data access framework (ORM) used to simplify the workload of addition, deletion, modification and query is the key.

It is applicable to small websites and small management systems. All functions are deployed into one function, which is simple and easy to use.

Disadvantages:

1. Performance scaling is difficult

2. Collaborative development problem

3. Not conducive to upgrade and maintenance

Vertical application architecture

When the number of visits increases gradually, the acceleration brought by a single application increases and the machine decreases. The application is divided into several unrelated applications to improve efficiency. At this point, the Web framework (MVC) for accelerating front-end page development is the key.

The independent deployment of each module is realized through business segmentation, which reduces the difficulty of maintenance and deployment. The team performs its own duties, which is easier to manage, and the performance expansion is more convenient and targeted.

Disadvantages: common modules cannot be reused, which is a waste of development

Distributed service architecture

When there are more and more vertical applications, the interaction between applications is inevitable. Extract the core business as an independent service, and gradually form a stable service center, so that the front-end applications can respond to the changing market demand more quickly. At this time, the distributed service framework (RPC) for improving business reuse and integration is the key.

Flow computing architecture

When there are more and more services, problems such as capacity evaluation and waste of small service resources gradually appear. At this time, it is necessary to add a dispatching center to manage the cluster capacity in real time based on the access pressure and improve the cluster utilization. At this time, the resource scheduling and Governance Center (SOA) [Service Oriented Architecture] used to improve machine utilization is the key.

What is RPC

RPC [Remote Procedure Call] refers to Remote Procedure Call. It is a way of inter process communication. It is a technical idea, not a specification. It allows a program to call a procedure or function in another address space (usually on another machine sharing a network) without the programmer explicitly encoding the details of the remote call. That is, whether programmers call local or remote functions, the calling code written by programmers is basically the same.

In other words, two servers A and B, one application deployed on server A, want to call the functions / methods provided by the application on server B. because they are not in the same memory space, they cannot be called directly. They need to express the calling semantics and convey the calling data through the network. Why use RPC? It is A requirement that cannot be completed in A process or even in A computer through local call, such as communication between different systems, or even communication between different organizations. Because the computing power needs to be expanded horizontally, applications need to be deployed on A cluster composed of multiple machines. RPC is to call remote functions like calling local functions;

Recommended reading: https://www.jianshu.com/p/2accc2840a1b

RPC Fundamentals

Step resolution:

RPC has two core modules: communication and serialization.
Serialization data transfer requires conversion

Test environment construction

Dubbo

Apache Dubbo | ˈ d ʌ b əʊ| It is a high-performance and lightweight open source Java RPC framework. It provides three core capabilities: interface oriented remote method invocation, intelligent fault tolerance and load balancing, and automatic service registration and discovery.

dubbo official website http://dubbo.apache.org/zh-cn/index.html

1. Understand the characteristics of Dubbo

2. View official documents

dubbo basic concept

Service Provider: the service Provider that exposes the service. When the service Provider starts, it registers the services it provides with the registry.

Service Consumer: the service Consumer that invokes the remote service. When starting, the service Consumer subscribes to the service required by the registry. The service Consumer selects one provider from the provider address list based on the soft load balancing algorithm to call. If the call fails, select another one to call.

Registry: the registry returns the service provider address list to the consumer. If there is any change, the registry will push the change data to the consumer based on the long connection

Monitor ing center: service consumers and providers accumulate call times and call times in memory, and regularly send statistical data to the monitoring center every minute

Call relation description

l the service container is responsible for starting, loading and running the service provider.

l when a service provider starts, it registers its services with the registry.

l service consumers subscribe to their required services from the registry when starting.

l the registry returns the service provider address list to the consumer. If there is any change, the registry will push the change data to the consumer based on the long connection.

l the service consumer selects one provider from the provider address list based on the soft load balancing algorithm to call. If the call fails, select another provider to call.

l service consumers and providers accumulate call times and call times in memory, and regularly send statistical data to the monitoring center every minute.

Dubbo environment construction

Click the official dubbo document and recommend that we use the Zookeeper registry

What is zookeeper? You can view official documents

Installing zookeeper under Window

1. Download zookeeper: address , we download 3.4.14, the latest version! Unzip zookeeper

2. Run / bin/zkServer.cmd, and an error will be reported during the initial operation, and there is no zoo.cfg configuration file;

Possible problems: flash back!

Solution: edit the zkServer.cmd file and add pause at the end. In this way, if there is an error in operation, it will not exit, and an error message will be prompted to find the reason.

3. Modify the zoo.cfg configuration file

Put the zoo under the conf folder_ Copy a copy of sample.cfg and rename it zoo.cfg.

Note several important points:

dataDir =. / directory of temporary data store (writable relative path)

Clientport = 2181 port number of zookeeper

When the modification is complete, start zookeeper again

4. Test with zkCli.cmd

ls /: lists all nodes saved under the root of zookeeper

[zk: 127.0.0.1:2181(CONNECTED) 4] ls /
[zookeeper]

Create a kuangshen node with a value of 123

create –e /kuangshen 123

Gets the value of the / kuangshen node

get /kuangshen

Let's look at the nodes again

Install Dubbo admin under Windows

dubbo itself is not a service software. It is actually a jar package that can help your java program connect to zookeeper and use zookeeper to consume and provide services.

However, in order to enable users to better manage and monitor many dubbo services, the official provides a visual monitoring program dubbo admin, but this monitoring will not affect the use even if it is not installed.

Let's install it here:

1. Download Dubbo admin

Address: https://github.com/apache/dubbo-admin/tree/master

2. Unzip into the directory

Modify Dubbo admin \ SRC \ main \ resources \ application.properties to specify the zookeeper address

server.port=7001
spring.velocity.cache=false
spring.velocity.charset=UTF-8
spring.velocity.layout-url=/templates/default.vm
spring.messages.fallback-to-system-locale=false
spring.messages.basename=i18n/message
spring.root.password=root
spring.guest.password=guest

dubbo.registry.address=zookeeper://127.0.0.1:2181

3. Package Dubbo admin in the project directory

mvn clean package -Dmaven.test.skip=true

The first packaging process is a little slow and needs to wait patiently! Until success!

4. Execute dubbo-admin-0.0.1-SNAPSHOT.jar under dubbo-admin\target

java -jar dubbo-admin-0.0.1-SNAPSHOT.jar

[Note: the zookeeper service must be turned on!]

After execution, let's go and visit http://localhost:7001/ At this time, we need to enter the login account and password. We are the default root root;

After successful login, view the interface

Installation complete!

SpringBoot + Dubbo + zookeeper

Frame construction

1. Start zookeeper

2. IDEA creates an empty project;

3. Create a module to implement the service provider: provider server, and select web dependency

4. After the project is created, we write a service, such as ticket selling service;

Write a provider interface

package com.kuang.provider.service;

public interface TicketService {
   public String getTicket();
}

Write the implementation class TicketServiceImpl

package com.kuang.provider.service;

public class TicketServiceImpl implements TicketService {
   @Override
   public String getTicket() {
       return "<Madness theory Java>";
  }
}

5. Create a module to realize the service consumer: consumer server, and select web dependency

6. After the project is created, we write a service, such as the user's service;

Write service

package com.kuang.consumer.service;

public class UserService {
   //We need to get the service from the registration center
}

Demand: now our users want to use the ticket buying service. How do we do this?

Service provider

1. To register the service provider in the registry, we need to integrate Dubbo and zookeeper, so we need to import the package

We go to github from dubbo's official website, look at the help document below, find dubbo springboot, and find the dependent package

<!-- Dubbo Spring Boot Starter -->
<dependency>
   <groupId>org.apache.dubbo</groupId>
   <artifactId>dubbo-spring-boot-starter</artifactId>
   <version>2.7.3</version>
</dependency>    

Zookeeper's package is downloaded from maven warehouse, ZooKeeper Client;

<!-- https://mvnrepository.com/artifact/com.github.sgroschupf/zkclient -->
<dependency>
   <groupId>com.github.sgroschupf</groupId>
   <artifactId>zkclient</artifactId>
   <version>0.1</version>
</dependency>

[new version of the pit] zookeeper and its dependency package. To solve log conflicts, you also need to eliminate log dependencies;

<!-- introduce zookeeper -->
<dependency>
   <groupId>org.apache.curator</groupId>
   <artifactId>curator-framework</artifactId>
   <version>2.12.0</version>
</dependency>
<dependency>
   <groupId>org.apache.curator</groupId>
   <artifactId>curator-recipes</artifactId>
   <version>2.12.0</version>
</dependency>
<dependency>
   <groupId>org.apache.zookeeper</groupId>
   <artifactId>zookeeper</artifactId>
   <version>3.4.14</version>
   <!--Exclude this slf4j-log4j12-->
   <exclusions>
       <exclusion>
           <groupId>org.slf4j</groupId>
           <artifactId>slf4j-log4j12</artifactId>
       </exclusion>
   </exclusions>
</dependency>

2. Configure dubbo related properties in the springboot configuration file!

#Service application name
dubbo.application.name=provider-server
#Address of Registration Center
dubbo.registry.address=zookeeper://127.0.0.1:2181
#Scan the services under the specified package
dubbo.scan.base-packages=com.kuang.provider.service

3. Configure the service annotation in the service implementation class and publish the service! Pay attention to the problem of guide package

import org.apache.dubbo.config.annotation.Service;
import org.springframework.stereotype.Component;

@Service //It can be scanned and automatically registered with the registry as soon as the project is started
@Component //Put it in the container and try not to use the service annotation after using Dubbo. Service has two annotations, springframewo and Dubbo, which are difficult to distinguish
public class TicketServiceImpl implements TicketService {
   @Override
   public String getTicket() {
       return "<Madness theory Java>";
  }
}

Logical understanding: when the application starts, dubbo will scan the service with @ component annotation under the specified package and publish it in the specified registry!

Service consumers

1. Import dependency, the same as the previous dependency;

<!--dubbo-->
<!-- Dubbo Spring Boot Starter -->
<dependency>
   <groupId>org.apache.dubbo</groupId>
   <artifactId>dubbo-spring-boot-starter</artifactId>
   <version>2.7.3</version>
</dependency>
<!--zookeeper-->
<!-- https://mvnrepository.com/artifact/com.github.sgroschupf/zkclient -->
<dependency>
   <groupId>com.github.sgroschupf</groupId>
   <artifactId>zkclient</artifactId>
   <version>0.1</version>
</dependency>
<!-- introduce zookeeper -->
<dependency>
   <groupId>org.apache.curator</groupId>
   <artifactId>curator-framework</artifactId>
   <version>2.12.0</version>
</dependency>
<dependency>
   <groupId>org.apache.curator</groupId>
   <artifactId>curator-recipes</artifactId>
   <version>2.12.0</version>
</dependency>
<dependency>
   <groupId>org.apache.zookeeper</groupId>
   <artifactId>zookeeper</artifactId>
   <version>3.4.14</version>
   <!--Exclude this slf4j-log4j12-->
   <exclusions>
       <exclusion>
           <groupId>org.slf4j</groupId>
           <artifactId>slf4j-log4j12</artifactId>
       </exclusion>
   </exclusions>
</dependency>

2. Configure the application.properties parameter of the consumer server

#To consume this service, you need to expose your name
dubbo.application.name=consumer-server
#Address of Registration Center
dubbo.registry.address=zookeeper://127.0.0.1:2181

3. The original normal step is to package the service provider's interface and import it with pom file. Here, we use a simple method to directly take the service interface. The path must be correct, that is, the same as that of the service provider;

  1. To improve the service class of consumer server, you need to create a TicketService with the same path as the TicketService of provider server in consumer server
package com.kuang.consumer.service;

import com.kuang.provider.service.TicketService;
import org.apache.dubbo.config.annotation.Reference;
import org.springframework.stereotype.Service;

@Service //Put it in a container
public class UserService {

   @Reference //If the specified service is referenced remotely, it will match according to the full class name to see who has registered the full class name in the registry
   TicketService ticketService;

   public void bugTicket(){
       String ticket = ticketService.getTicket();
       System.out.println("Buy it at the registry"+ticket);
  }

}
  1. Test class preparation;
@RunWith(SpringRunner.class)
@SpringBootTest
public class ConsumerServerApplicationTests {

   @Autowired
   UserService userService;

   @Test
   public void contextLoads() {

       userService.bugTicket();

  }

}

Start test

  1. Start zookeeper

  2. Open Dubbo admin to realize monitoring [can be omitted]

  3. Open server

  4. Consumer consumption test, results:

Monitoring center:

Steps:

ok, this is the application of SpingBoot + dubbo + zookeeper to realize distributed development. In fact, it is an idea of service splitting

Source: Madness theory( https://t.hk.uy/a5En)

Tags: Spring Boot

Posted on Sat, 04 Dec 2021 16:03:07 -0500 by regexpert