Service avalanche effect problems and Solutions

Article directory

1. Service avalanche effect

When a request depends on multiple services:
Normal access

However, when the requested service has problems such as inaccessibility, exception, timeout (I in the figure), the user's request will be blocked

If there are unreachable services in the requests of multiple users, they will all fall into a blocked state.

How can we solve it?

  • The introduction of Hystrix can solve this problem through service fusing and service degradation.

2. Service fuse service degradation

Introduction to Hystrix circuit breaker

The Chinese name of hystrix is "porcupine". Porcupine is covered with thorns, which can protect itself from natural enemies. It represents a defense mechanism, which coincides with the function of hystrix itself. Therefore, Netflix team named the framework hystrix and used the corresponding cartoon image as the logo.

In a distributed system, many dependencies inevitably fail to call, such as timeout, exception, etc. How to ensure that in the case of a dependency failure, the whole service will not fail? This is what hystrix needs to do. Hystrix provides functions such as fusing, isolation, Fallback, cache, monitoring, etc., which can ensure the system is still available when one or more dependencies have problems at the same time.

Hystrix service fuse service degraded @ HystrixCommand fallbackMethod

Fusing mechanism is a micro service link protection mechanism to deal with avalanche effect.
When a service is unavailable or the response time is timeout, the service will be degraded, and then the service call of the node will be fused to quickly return the customized error impact page information.

Let's write a project to test;

We write a new service provider project with service fuse microservice-student-provider-hystrix-1004

Import pom dependency related to Hystri

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>com.hyf</groupId>
        <artifactId>microservice</artifactId>
        <version>1.0-SNAPSHOT</version>
    </parent>

    <artifactId>microservice-student-provider-hystrix-1004</artifactId>

    <properties>
        <java.version>1.8</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-tomcat</artifactId>
        </dependency>
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>druid-spring-boot-starter</artifactId>
        </dependency>
        <dependency>
            <groupId>com.hyf</groupId>
            <artifactId>microservice-common</artifactId>
            <version>1.0-SNAPSHOT</version>
            <scope>compile</scope>
        </dependency>

        <!--Add registry Eureka Related configuration-->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-eureka</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <!-- actuator Monitoring introduction -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

        <!--Hystrix Dependent dependence-->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-hystrix</artifactId>
        </dependency>

    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

application.yml modify the lower port and instance name

server:
  port: 1004
  context-path: /
spring:
  datasource:
    type: com.alibaba.druid.pool.DruidDataSource
    driver-class-name: com.mysql.jdbc.Driver
    url: jdbc:mysql://localhost:3306/tb_student?useUnicode=true&characterEncoding=utf8
    username: root
    password: 123
  jpa:
    hibernate:
      ddl-auto: update
    show-sql: true
  application:
    name: microservice-student
  profiles: provider-hystrix-1004

eureka:
  instance:
    hostname: localhost
    appname: microservice-student
    instance-id: microservice-student:1004
    prefer-ip-address: true
  client:
    service-url:
      defaultZone: http://eureka2001:2001/eureka/,http://eureka2002:2002/eureka/,http://eureka2003:2003/eureka/

info:
  groupId: com.hyf.testSpringcloud
  artifactId: microservice-student-provider-hystrix-1004
  version: 1.0-SNAPSHOT
  userName: http://hyf.com
  phone: 18188888888

3. The startup class studentproviderhystrixapplication_isannotated to support @ EnableCircuitBreaker

The business logic code will not be written. It has been written before.

Add the following code to the StudentProviderController for testing:

Here, my normal access returns 200 business data xxxxx
But here we have Thread.sleep(2000) simulation timeout;
Here we add @ HystrixCommand annotation and fallback method
It indicates that this method can return normal business data only when there is no exception or timeout (hystrix defaults to 1 second);
Otherwise, enter the local method specified by our fallback. We make a mistake in the 500 system and try again later to effectively solve the avalanche effect and return it to the user interface
Good error message;

/**
     * Test Hystrix service degradation
     * @return
     * @throws InterruptedException
     */
    @ResponseBody
    @GetMapping(value="/hystrix")
    @HystrixCommand(fallbackMethod="hystrixFallback")
    public Map<String,Object> hystrix() throws InterruptedException{
//        Thread.sleep(2000);
        Map<String,Object> map=new HashMap<String,Object>();
        map.put("code", 200);
        map.put("info","Job number ["+port+"]Serving you");
        return map;
    }

    public Map<String,Object> hystrixFallback() throws InterruptedException{
        Map<String,Object> map=new HashMap<String,Object>();
        map.put("code", 500);
        map.put("info", "System ["+port+"]Busy, try again later");
        return map;
    }

Visitors also need to add corresponding methods

/**
     * Test Hystrix service degradation
     * @return
     */
    @GetMapping(value="/hystrix")
    @ResponseBody
    public Map<String,Object> hystrix(){
        return restTemplate.getForObject(SERVER_IP_PORT+"/student/hystrix/", Map.class);

Then let's test it
Normal access:

When the rest code is released to simulate the avalanche state

3. Default timeout setting of Hystrix

The default timeout of hystrix is 1 second. We can see it through the hystrix source code,
Find the HystrixCommandProperties class under the package hystrix-core.jar com.netflix.hystrix
Default? Executiontimeoutinmilliseconds property default timeout for situations

If we want to customize the default time of hystrix in our system
application.yml configuration file plus

hystrix:
  command:
    default:
      execution:
        isolation:
          thread:
            timeoutInMilliseconds: 3000

When we visit again:

4. Hystrix service monitoring Dashboard

Dashboard for Hystrix service monitoring

Hystrix provides a quasi real-time service call monitoring project Dashboard, which can record the execution of requests initiated through hystrix in real time,
It can be displayed to users in the form of charts.

Our new project: microservice-student-consumer-hystrix-dashboard-90
Import pom dependency

<!--Hystrix Service monitoring Dashboard rely on-->
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-hystrix</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-hystrix-dashboard</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

application.yml configuration

server:
  port: 90
  context-path: /

Add the required annotation to the startup class:

We started the project;
Then the browser enters: http://localhost:90/hystrix,
If it appears, it means OK;

For our monitoring, the path http://localhost:1004/hystrix.stream is enough;
Always ping, and then data returns data;

In graphical terms:
,http://localhost:1004/hystrix.stream

After entering:

Index meaning:

Various situations:

71 original articles published, praised, 6 visits 2097
Private letter follow

Tags: Spring Maven MySQL Apache

Posted on Sat, 11 Jan 2020 06:27:52 -0500 by ccbayer