The way to upgrade Spring Cloud - Hoxton - 3. Load balancing replaced by Spring Cloud loadbalancer from ribbon

Examples of this series and glue code address: https://github.com/HashZhang/spring-cloud-scaffold

Replace the load balancing Ribbon with Spring Cloud Load Balancer

Spring Cloud Load Balancer is not an independent project, but a module of spring cloud Commons. Eureka and related starter s are used in the project. It is almost impossible to completely eliminate the related dependency of the Ribbon. People in the spring community also see this. Turn off the Ribbon and enable spring cloud loadbalancer through configuration.

spring.cloud.loadbalancer.ribbon.enabled=false

After the ribbon is closed, the Spring Cloud LoadBalancer will be loaded as the default load balancer.

The structure of Spring Cloud LoadBalancer is as follows:

Among them:

  1. There is only one BlockingLoadBalancerClient in the global, which is responsible for executing all load balancing requests.
  2. BlockingLoadBalancerClient loads the load balancing configuration of the corresponding microservice from LoadBalancerClientFactory.
  3. Each microservice has its own LoadBalancer, which contains the algorithm of load balancing. For example, RoundRobin. According to the algorithm, select an instance from the list of instances returned by ServiceInstanceListSupplier to return.

1. Realize zone isolation

To achieve zone isolation, you should do something from the ServiceInstanceListSupplier. The default implementation contains the ServiceInstanceListSupplier for zone isolation - > org.springframework.cloud . loadbalancer.core.ZonePreferenceServiceInstanceListSupplier :

private List<ServiceInstance> filteredByZone(List<ServiceInstance> serviceInstances) {
	if (zone == null) {
		zone = zoneConfig.getZone();
	}
	//If the zone is not null and there is a live instance under the zone, the instance list will be returned
	//Otherwise, all instances are returned
	if (zone != null) {
		List<ServiceInstance> filteredInstances = new ArrayList<>();
		for (ServiceInstance serviceInstance : serviceInstances) {
			String instanceZone = getZone(serviceInstance);
			if (zone.equalsIgnoreCase(instanceZone)) {
				filteredInstances.add(serviceInstance);
			}
		}
		if (filteredInstances.size() > 0) {
			return filteredInstances;
		}
	}
	// If the zone is not set or there are no zone-specific instances available,
	// we return all instances retrieved for given service id.
	return serviceInstances;
}

In this case, if no zone is specified or there is no surviving instance in the zone, all the found instances will be returned, regardless of zone. This does not meet our requirements, so we modify and implement our own com.github.hashjang.hoxton.service.consumer.config.SameZoneOnlyServiceInstanceListSupplier:

private List<ServiceInstance> filteredByZone(List<ServiceInstance> serviceInstances) {
    if (zone == null) {
        zone = zoneConfig.getZone();
    }
    if (zone != null) {
        List<ServiceInstance> filteredInstances = new ArrayList<>();
        for (ServiceInstance serviceInstance : serviceInstances) {
            String instanceZone = getZone(serviceInstance);
            if (zone.equalsIgnoreCase(instanceZone)) {
                filteredInstances.add(serviceInstance);
            }
        }
        if (filteredInstances.size() > 0) {
            return filteredInstances;
        }
    }
    //If it is not found, an empty list will be returned, and the instances of other clusters will never be returned
    return List.of();
}

Then let's take a look at the default Spring Cloud LoadBalancer, which is cached:

org.springframework.cloud.loadbalancer.annotation.LoadBalancerClientConfiguration

@Bean
@ConditionalOnBean(ReactiveDiscoveryClient.class)
@ConditionalOnMissingBean
public ServiceInstanceListSupplier discoveryClientServiceInstanceListSupplier(
		ReactiveDiscoveryClient discoveryClient, Environment env,
		ApplicationContext context) {
	DiscoveryClientServiceInstanceListSupplier delegate = new DiscoveryClientServiceInstanceListSupplier(
			discoveryClient, env);
	ObjectProvider<LoadBalancerCacheManager> cacheManagerProvider = context
			.getBeanProvider(LoadBalancerCacheManager.class);
	if (cacheManagerProvider.getIfAvailable() != null) {
		return new CachingServiceInstanceListSupplier(delegate,
				cacheManagerProvider.getIfAvailable());
	}
	return delegate;
}

DiscoveryClientServiceInstanceListSupplier pulls the instance list from Eureka every time, and cacheingserviceinstancelistsupplier provides cache, so it is not necessary to pull from Eureka every time. It can be seen that cacheserviceinstancelistsupplier is an implementation of a proxy mode, which is the same as SameZoneOnlyServiceInstanceListSupplier.

Let's assemble our ServiceInstanceListSupplier. Because we are a synchronized environment, we only need to implement the synchronized ServiceInstanceListSupplier.

public class CommonLoadBalancerConfig {

    /**
     * ServiceInstanceListSupplier in synchronous environment
     * SameZoneOnlyServiceInstanceListSupplier Limit the return of instances under the same zone only (note)
     * CachingServiceInstanceListSupplier Enable caching without accessing the list of eureka request instances every time
     *
     * @param discoveryClient
     * @param env
     * @param zoneConfig
     * @param context
     * @return
     */
    @Bean
    @Order(Integer.MIN_VALUE)
    public ServiceInstanceListSupplier discoveryClientServiceInstanceListSupplier(
            DiscoveryClient discoveryClient, Environment env,
            LoadBalancerZoneConfig zoneConfig,
            ApplicationContext context) {
        ServiceInstanceListSupplier delegate = new SameZoneOnlyServiceInstanceListSupplier(
                new DiscoveryClientServiceInstanceListSupplier(discoveryClient, env),
                zoneConfig
        );
        ObjectProvider<LoadBalancerCacheManager> cacheManagerProvider = context
                .getBeanProvider(LoadBalancerCacheManager.class);
        if (cacheManagerProvider.getIfAvailable() != null) {
            return new CachingServiceInstanceListSupplier(
                    delegate,
                    cacheManagerProvider.getIfAvailable()
            );
        }
        return delegate;
    }
}

2. When the next retry is implemented, if there are other instances, they will definitely retry other instances different from this one

The default roundrobin loadbalancer, in which the polling position is of an Atomic type, is shared by all threads and all requests under the call request of a microservice (a roundrobin loadbalancer will be created when calling other microservices). When using, there will be such a problem:

  • Suppose A microservice has two instances, instance A and instance B
  • A request X is sent to instance a, position = position + 1
  • When the request does not return, request Y arrives and is sent to instance B. position = position + 1
  • Failed to request A, Retry, retry instance or instance A

In this way, in the case of retry, the retry of a request may be sent to the last instance for retry, which is not what we want. For this, I proposed an Issue: Enhance RoundRoubinLoadBalancer position . The idea I modified is that we need a single request isolation position, which can get the instance to which the request is to be sent by taking the surplus of instance number. So how to isolate requests?

The first thing I think about is thread isolation, but this is not good. The bottom layer of Spring Cloud LoadBalancer uses the reactor framework, which results in the actual thread carrying the selected instance, not the business thread, but the thread pool in the reactor, as shown in the figure: Therefore, position cannot be implemented in the way of ThreadLocal.

Because we use sleuth, the context of the general request will pass the traceId. We can distinguish different requests according to the traceId and implement our LoadBalancer:

RoundRobinBaseOnTraceIdLoadBalancer

//This timeout needs to be set longer than your request's connectTimeout + readTimeout
private final LoadingCache<Long, AtomicInteger> positionCache = Caffeine.newBuilder().expireAfterWrite(3, TimeUnit.SECONDS).build(k -> new AtomicInteger(ThreadLocalRandom.current().nextInt(0, 1000)));

private Response<ServiceInstance> getInstanceResponse(List<ServiceInstance> serviceInstances) {
    if (serviceInstances.isEmpty()) {
        log.warn("No servers available for service: " + this.serviceId);
        return new EmptyResponse();
    }
    //If there is no traceId, a new one will be generated, but it's better to check why it doesn't
    //If MQ consumption does not actively generate traceId, it is better to actively generate traceId.
    Span currentSpan = tracer.currentSpan();
    if (currentSpan == null) {
        currentSpan = tracer.newTrace();
    }
    long l = currentSpan.context().traceId();
    int seed = positionCache.get(l).getAndIncrement();
    return new DefaultResponse(serviceInstances.get(seed % serviceInstances.size()));
}

3. Replace the default load balancing related Bean implementation

To replace the default implementation with the above two classes, first write a configuration class:

public class CommonLoadBalancerConfig {

    private volatile boolean isValid = false;

    /**
     * ServiceInstanceListSupplier in synchronous environment
     * SameZoneOnlyServiceInstanceListSupplier Limit the return of instances under the same zone only (note)
     * CachingServiceInstanceListSupplier Enable caching without accessing the list of eureka request instances every time
     *
     * @param discoveryClient
     * @param env
     * @param zoneConfig
     * @param context
     * @return
     */
    @Bean
    @Order(Integer.MIN_VALUE)
    public ServiceInstanceListSupplier discoveryClientServiceInstanceListSupplier(
            DiscoveryClient discoveryClient, Environment env,
            LoadBalancerZoneConfig zoneConfig,
            ApplicationContext context) {
        isValid = true;
        ServiceInstanceListSupplier delegate = new SameZoneOnlyServiceInstanceListSupplier(
                new DiscoveryClientServiceInstanceListSupplier(discoveryClient, env),
                zoneConfig
        );
        ObjectProvider<LoadBalancerCacheManager> cacheManagerProvider = context
                .getBeanProvider(LoadBalancerCacheManager.class);
        if (cacheManagerProvider.getIfAvailable() != null) {
            return new CachingServiceInstanceListSupplier(
                    delegate,
                    cacheManagerProvider.getIfAvailable()
            );
        }
        return delegate;
    }

    @Bean
    public ReactorLoadBalancer<ServiceInstance> reactorServiceInstanceLoadBalancer(
            Environment environment,
            ServiceInstanceListSupplier serviceInstanceListSupplier,
            Tracer tracer) {
        if (!isValid) {
            throw new IllegalStateException("should use the ServiceInstanceListSupplier in this configuration, please check config");
        }
        String name = environment.getProperty(LoadBalancerClientFactory.PROPERTY_NAME);
        return new RoundRobinBaseOnTraceIdLoadBalancer(
                name,
                serviceInstanceListSupplier,
                tracer
        );
    }
}

Then, specify the default load balancing configuration to take this configuration. Through the note:

@LoadBalancerClients(defaultConfiguration = {CommonLoadBalancerConfig.class})

Tags: Programming Spring github

Posted on Thu, 04 Jun 2020 00:48:48 -0400 by stakes