Overview of Hystrix

Due to the network or its own reasons, the service cannot be guaranteed to be 100% available. If a single service has a problem, thread blocking will occur when calling the service. At this time, if a large number of requests flood in, the thread resources of the Servlet container will be consumed, resulting in service paralysis. The dependence between services, the failure will spread, and will cause disastrous and serious consequences to the whole microservice system, which is the "avalanche" effect of service failure.

Spring Cloud Hystrix is encapsulated on the basis of Hystrix (open source software developed by Netflix) and provides functions such as service fusing, service degradation and thread isolation. Through these functions, the fault tolerance of services can be provided.

The Hystrix component implements the circuit breaker mode. If the underlying service fails, it will lead to chain failure. When the unavailability of a call to a specific service reaches a threshold (the Hystric is 20 times in 5 seconds), the circuit breaker will be opened. After the circuit breaker is opened, the interlock fault can be avoided, and the fallback method can directly return a fixed value.

Use of Hystrix

Correlation dependency

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                 <version>Hoxton.SR9</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
        </dependency>
    </dependencies>

Eureka registry

@EnableEurekaServer
@SpringBootApplication
public class DemoApplication {
    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }
}

server:
  port: 8080

spring:
  application:
    name: Eureka-Server

  security:
    user:
      name: admin
      password: admin123

eureka:
  instance:
    hostname: localhost
  client:
    register-with-eureka: false
    fetch-registry: false
    serviceUrl:
      defaultZone: http://admin:admin123@localhost:8080/eureka/

Security configuration, release

@EnableWebSecurity
@Configuration
public class WebSecurityConfig extends WebSecurityConfigurerAdapter {

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        // Scheme I
        http.csrf().disable(); // Turn off csrf
        http.authorizeRequests().anyRequest().authenticated().and().httpBasic(); // Turn on authentication

        // Scheme II
        //	http.csrf().ignoringAntMatchers("/eureka/**");
        //	super.configure(http);
    }
}

Service provider

server:
  port: ${PORT:8081}
  
spring:
  application:
    name: Server-Produce
    
eureka:
  instance:
    hostname: localhost
  client:
    register-with-eureka: true
    fetch-registry: true
    serviceUrl:
      defaultZone: http://admin:admin123@localhost:8080/eureka/

@RestController
public class TestController {
    
    @GetMapping("/test")
    public String hello() {
        return "hello world";
    }
}

@EnableDiscoveryClient
@SpringBootApplication
public class DemoApplication {

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }
}

Service consumers

server:
  port: 8083
  
spring:
  application:
    name: Server-Consumer
    
eureka:
  client:
    serviceUrl:
      defaultZone: http://admin:admin123@localhost:8080/eureka/

@SpringCloudApplication
public class DemoApplication {

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }

    @Bean
    @LoadBalanced
    RestTemplate restTemplate() {
        return new RestTemplate();
    }
}

@Service
public class TestService {

    private Logger log = LoggerFactory.getLogger(this.getClass());

    @Autowired
    private RestTemplate restTemplate;

    @HystrixCommand
    public String test() {
        return restTemplate.getForObject("http://Server-Produce/test/", String.class);
    }

    public String testError() {
        return "The service is temporarily unavailable,Please try again later";
    }
}

@RestController
public class TestController {
    
    @Autowired
    private TestService testService;

    @RequestMapping("test/{id}")
    public String test(){
        return testService.test();
    }
}

Add @ EnableHystrix or @ enablercircuitbreaker annotation. These two annotations are equivalent

@EnableDiscoveryClient
@SpringBootApplication
@EnableCircuitBreaker
public class DemoApplication {

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }

    @Bean
    @LoadBalanced
    RestTemplate restTemplate() {
        return new RestTemplate();
    }
}

@EnableHystrix source code

@Target({ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Inherited
@EnableCircuitBreaker
public @interface EnableHystrix {
}

@The combination of enablercircuitbreaker, @ EnableDiscoveryClient and @ SpringBootApplication annotations can be replaced by @ SpringCloudApplication

@SpringCloudApplication
public class DemoApplication {

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }

    @Bean
    @LoadBalanced
    RestTemplate restTemplate() {
        return new RestTemplate();
    }
}

@EnableHystrix annotation source code

@Target({ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Inherited
@SpringBootApplication
@EnableDiscoveryClient
@EnableCircuitBreaker
public @interface SpringCloudApplication {
}

Perform test

@Detailed explanation of HystrixCommand

@HystrixCommand Annotations also contain many other attribute functions

service degradation

If the method throws an exception, use the @ HystrixCommand annotation to specify the service degradation method

@Service
public class TestService {
    @Autowired
    private RestTemplate restTemplate;

    @HystrixCommand(fallbackMethod = "testError")
    public String test() {
        return restTemplate.getForObject("http://Server-Produce/test/", String.class);
    }

    public String testError() {
        return "The service is temporarily unavailable,Please try again later";
    }
}

exception handling

For the method annotated with @ HystrixCommand annotation, except for the HystrixBadRequestException exception, other exceptions will trigger service degradation. If you want to specify that an exception does not trigger service degradation, you can ignore it using the ignoreExceptions attribute of the @ HystrixCommand annotation.

@Service
public class TestService {
    @Autowired
    private RestTemplate restTemplate;

    @HystrixCommand(fallbackMethod = "testError",ignoreExceptions = {ArrayIndexOutOfBoundsException.class})
    public String test() {
        String[] strArray=new  String[]{"1","2"};
        System.out.println(strArray[2]);
        return restTemplate.getForObject("http://Server-Produce/test/", String.class);
    }

    public String testError() {
        return "The service is temporarily unavailable,Please try again later";
    }
}

java.lang.ArrayIndexOutOfBoundsException: 2
	at com.example.demo.service.TestService.test(TestService.java:21) ~[classes/:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_201]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_201]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_201]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_201]

The exception information thrown by the method can be obtained by using the Throwable object in the method of service degradation

@Service
public class TestService {
    @Autowired
    private RestTemplate restTemplate;

    @HystrixCommand(fallbackMethod = "testError")
    public String test(){
        String[] strArray=new  String[]{"1","2"};
        try {
            System.out.println(strArray[2]);
        } catch (Exception e) {
            throw new ArrayIndexOutOfBoundsException("Array out of bounds exception");
        }
        return restTemplate.getForObject("http://Server-Produce/test/", String.class);
    }

    public String testError(Throwable e) {
        System.out.println(e.toString());
        return "The service is temporarily unavailable,Please try again later";
    }
}

java.lang.ArrayIndexOutOfBoundsException: Array out of bounds exception

Naming and grouping

Specify the commandKey, groupKey and threadPoolKey properties of the @ HystrixCommand annotation to set the command name, grouping and thread pool division

@Service
public class TestService {
    @Autowired
    private RestTemplate restTemplate;

    @HystrixCommand(fallbackMethod = "testError", commandKey = "test", groupKey = "testGroup", threadPoolKey = "testThread")
    public String test() {
        return restTemplate.getForObject("http://Server-Produce/test/", String.class);
    }

    public String testError(Throwable e) {
        return "The service is temporarily unavailable,Please try again later";
    }
}

By setting the command group, Hystrix will organize and count the alarm, dashboard and other information of the command according to the group. By default, the Hystrix command divides the thread pool through the group name, that is, commands with the same group name are placed in the same thread pool. If the thread pool name is set through the threadPoolKey, it is divided according to the thread pool name.

INFO 23288 --- [ix-testThread-2] cn.ybzy.demo.service.TestService     : implement test()....

INFO 23288 --- [rix-testGroup-1] cn.ybzy.demo.service.TestService     : implement test()....

Hystrix cache

Call the same method multiple times

@RestController
public class TestController {

    @Autowired
    private TestService testService;

    @RequestMapping("test")
    public String test() {
        for (int i = 0; i < 5; i++) {
            testService.test();
        }
        return testService.test();
    }
}

    INFO 23288 --- [ix-testThread-9] cn.ybzy.demo.service.TestService     : implement test()....
    INFO 23288 --- [x-testThread-10] cn.ybzy.demo.service.TestService     : implement test()....
    INFO 23288 --- [x-testThread-10] cn.ybzy.demo.service.TestService     : implement test()....
    INFO 23288 --- [x-testThread-10] cn.ybzy.demo.service.TestService     : implement test()....
    INFO 23288 --- [x-testThread-10] cn.ybzy.demo.service.TestService     : implement test()....
    INFO 23288 --- [x-testThread-10] cn.ybzy.demo.service.TestService     : implement test()....

By enabling the cache, the method can be called only once, and the other calls can be obtained directly from the cache.

use@CacheResult Comments can be found in Hystrix Enable cache in

Hystrix The returned object will be cached key It defaults to all parameters of the method. There is only one parameter here id Parameters, so cached key For users id. 

@Service
public class TestService {

    private Logger log = LoggerFactory.getLogger(this.getClass());

    @Autowired
    private RestTemplate restTemplate;

    @CacheResult
    @HystrixCommand(fallbackMethod = "testError", commandKey = "test", groupKey = "testGroup", threadPoolKey = "testThread")
    public String test(Long id) {
        log.info("implement test({})....",id);
        return restTemplate.getForObject("http://Server-Produce/test/", String.class);
    }

    public String testError(Long id,Throwable e) {
        log.info("Request parameters ID value:{}",id);
        return "The service is temporarily unavailable,Please try again later";
    }
}

The request cache is not available. The HystrixRequestContext needs to be initialized

java.lang.IllegalStateException: Request caching is not available. Maybe you need to initialize the HystrixRequestContext?
	at com.netflix.hystrix.HystrixRequestCache.get(HystrixRequestCache.java:104) ~[hystrix-core-1.5.18.jar:1.5.18]
	at com.netflix.hystrix.AbstractCommand$7.call(AbstractCommand.java:478) ~[hystrix-core-1.5.18.jar:1.5.18]
	at com.netflix.hystrix.AbstractCommand$7.call(AbstractCommand.java:454) ~[hystrix-core-1.5.18.jar:1.5.18]
	at rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:46) ~[rxjava-1.3.8.jar:1.3.8]
	at rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:35) ~[rxjava-1.3.8.jar:1.3.8]

@Component
@WebFilter(filterName = "hystrixRequestContextServletFilter", urlPatterns = "/*", asyncSupported = true)
public class HystrixRequestContextServletFilter implements Filter {
    @Override
    public void init(FilterConfig filterConfig) {
    }

    @Override
    public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
        HystrixRequestContext context = HystrixRequestContext.initializeContext();
        filterChain.doFilter(servletRequest, servletResponse);
        context.close();
    }

    @Override
    public void destroy() {
    }
}

INFO 22740 --- [ix-testThread-1] cn.ybzy.demo.service.TestService     : implement test(1)....

The breakpoint finds that each request needs to initialize the HystrixRequestContext. After the first call in the same request, the result will be cached, and the cache life cycle is within the current request!

Set key value

You can explicitly specify the size of the cache key value

appoint key There are two ways to change the value of

adopt@CacheKey Annotation assignment

    @CacheResult
    @HystrixCommand(fallbackMethod = "testError", commandKey = "test", groupKey = "testGroup", threadPoolKey = "testThread")
    public String test(@CacheKey("id") Long id) {
        log.info("implement test({})....",id);
        return restTemplate.getForObject("http://Server-Produce/test/", String.class);
    }

Specifies that the internal attribute of the parameter object is the key value

    @CacheResult
    @HystrixCommand(fallbackMethod = "testError", commandKey = "test", groupKey = "testGroup", threadPoolKey = "testThread")
    public String test(@CacheKey("id") User user) {
        return restTemplate.getForObject("http://Server-Produce/test/", String.class);
    }

Specified by method, the return value of the method must be of type String

    public String getCacheKey(Long id) {
        return String.valueOf(id);
    }

    @CacheResult(cacheKeyMethod = "getCacheKey")
    @HystrixCommand(fallbackMethod = "testError", commandKey = "test", groupKey = "testGroup", threadPoolKey = "testThread")
    public String test(Long id) {
        return restTemplate.getForObject("http://Server-Produce/test/", String.class);
    }

Cache clear

@The commandKey attribute of CacheRemove is consistent with that defined in getUser

    @CacheResult
    @HystrixCommand(fallbackMethod = "testError", commandKey = "updateUser", groupKey = "testGroup", threadPoolKey = "testThread")
    public String test(@CacheKey("id") Long id) {
        return restTemplate.getForObject("http://Server-Produce/test/", String.class);
    }

    @CacheRemove(commandKey = "updateUser")
    @HystrixCommand
    public void updateUser(@CacheKey("id") User user) {
        this.restTemplate.put("http://Server-Produce/updateUser/", user);
    }

Request merge

Request merging is to merge multiple single requests into one request to call the service provider, so as to reduce the load of the service provider. It is a solution to deal with high concurrency.

An @ HystrixCollapser annotation is provided in Hystrix, which can integrate multiple requests that depend on the same service in a very short period of time (10 milliseconds by default) and initiate requests in batch.

Service provider

    @GetMapping("testRequestMerge/{id}")
    public User get(@PathVariable long id) {
        return new User(Long.valueOf(id), "user" + id, "123456");
    }

    @GetMapping("testRequestMerge")
    public List<User> testRequestMerge(String ids) {
        log.info("Obtain user information in batch");
        List<User> list = new ArrayList<>();
        for (String id : ids.split(",")) {
            list.add(new User(Long.valueOf(id), "user" + id, "123456"));
        }
        return list;
    }

Service consumers

    @HystrixCollapser(batchMethod = "findUserBatch", scope = com.netflix.hystrix.HystrixCollapser.Scope.GLOBAL,collapserProperties = {
            @HystrixProperty(name = "maxRequestsInBatch",value = "5"),
            @HystrixProperty(name = "timerDelayInMilliseconds", value = "100")
    })
    public Future<User> findUser(Long id) {
        log.info("Get individual user information");
        return new AsyncResult<User>() {
            @Override
            public User invoke() {
                return restTemplate.getForObject("http://Server-Produce/testRequestMerge/{id}", User.class, id);
            }
        };
    }

    @HystrixCommand
    public List<User> findUserBatch(List<Long> ids) {
        log.info("Obtain user information in batch,ids: " + ids);
        User[] users = restTemplate.getForObject("http://Server-Produce/testRequestMerge?ids={1}", User[].class, StringUtils.join(ids, ","));
        return Arrays.asList(users);
    }

batchMethod Property specifies the method of batch processing

timerDelayInMilliseconds: Set how long batch processing is performed. The default value is 10 ms

maxRequestsInBatch Set the maximum number of requests for batch processing. The default value is Integer.MAX_VALUE

test method

    @GetMapping("testRequestMerge")
    public void testRequerstMerge() throws ExecutionException, InterruptedException {
        Future<User> f1 = testService.findUser(1L);
        Future<User> f2 = testService.findUser(2L);
        Future<User> f3 = testService.findUser(3L);
        f1.get();
        f2.get();
        f3.get();
        Thread.sleep(200);
        Future<User> f4 = testService.findUser(4L);
        f4.get();
    }

In the test method, the findUser method is called four times. Before the last call (f4), let the thread wait for 200 milliseconds (greater than 100 milliseconds defined in timerDelayInMilliseconds). Therefore, it is expected that the first three calls will be merged, and the last call will not be merged.

  INFO 23312 --- [-TestService-10] com.example.demo.service.TestService     : Obtain user information in batch,ids: [1, 3, 2]
  INFO 23312 --- [-TestService-10] com.example.demo.service.TestService     : Obtain user information in batch,ids: [4]

The console does not print the log of obtaining individual user information in the findUser method. In fact, the findUser method will not be called, so the above code can be simplified as follows:

    @HystrixCollapser(batchMethod = "findUserBatch", scope = com.netflix.hystrix.HystrixCollapser.Scope.GLOBAL,collapserProperties = {
            @HystrixProperty(name = "maxRequestsInBatch",value = "5"),
            @HystrixProperty(name = "timerDelayInMilliseconds", value = "100")
    })
    public Future<User> findUser(Long id) {
        return null;
    }

    @HystrixCommand
    public List<User> findUserBatch(List<Long> ids) {
        log.info("Obtain user information in batch,ids: " + ids);
        User[] users = restTemplate.getForObject("http://Server-Produce/testRequestMerge?ids={1}", User[].class, StringUtils.join(ids, ","));
        return Arrays.asList(users);
    }

Using Hystrix in Ribbon and Feign

Using circuit breaker in ribbon

Annotate the method with @ HystrixCommand. This annotation creates the fuse function for this method, and specifies the fallbackMethod fuse method, which directly returns a string.

@Service
public class HelloService {

	@Autowired
	private RestTemplate restTemplate;

	//Service id
	String serviceId = "eureka-client1";

    @HystrixCommand(fallbackMethod = "error")
    public String test(String name) {

		String forObject = restTemplate.getForObject("http://" + serviceId + "/test", String.class);
		System.out.println(forObject);
		return  forObject ;
	}

    public String error(String name) {
        return "call:"+serviceId +"Service failure!";
    }
}

Use of circuit breakers in Feign

Feign has its own circuit breaker, which needs to be configured in the configuration file

feign:
  hystrix:
    enabled: true

Add the specified class of fallback in the annotation of the interface

@FeignClient(value = "service-hi",fallback=TestControllerError .class)

@Component
@FeignClient(value = "eureka-client1") //Specifies the service name of the remote call
public interface TestController {
    //Remote call test interface
    @GetMapping("/test")//Use GetMapping to identify the method type of http called remotely
    public String test();
}

TestControllerError needs to implement the TestController interface and inject it into the Ioc container

public class TestControllerError implements TestController{

	public String test() {
		return "sorry";
	}
}

Hystrix Dashboard

Hystrix provides the Hystrix Dashboard to monitor the operation of hystrix in real time. The real-time information fed back by the Hystrix Dashboard can help quickly find the problems in the system and take countermeasures in time.

Monitoring a single Hystrix instance

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                 <version>Hoxton.SR8</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-hystrix-dashboard</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
    </dependencies>

spring:
  application:
    name: Hystrix-Dashboard
server:
  port: 9000

Add the annotation @ EnableHystrixDashboard on the entry class to enable the function of the Hystrix Dashboard.

@SpringBootApplication
@EnableHystrixDashboard
public class DemoApplication {

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }
}

Start project access http://localhost:9002/hystrix