Analysis of Hystrix data collection and sliding window mechanism

Dependencies are as follows

<dependency>
    <groupId>com.netflix.hystrix</groupId>
    <artifactId>hystrix-core</artifactId>
    <version>1.5.18</version>
</dependency>

What is Hystrix

The last version was released on November 2018. At present, it is in the maintenance stage and will not be upgraded

  • Purpose:

    • Stop cascading failure. fallback and elegant degradation, Fail fast and fast recovery
    • Real time monitoring and configuration changes
    • Resource isolation, partial unavailability will not lead to overall system unavailability
  • Scenario: in the commodity list interface, you need to obtain red envelope, price, label and other data. At this time, you can give this a thread pool.
    If the thread pool is full, the non commodity list interface of the current service will not be affected

  • Framework used: hystrix mainly uses Rxjava. For reference: https://www.jianshu.com/p/5e93c9101dc5

Sliding window execution case

Execution steps:

  1. Assuming that the total sliding window is 1s and there are 4 buckets, the space of each bucket is 0.25s, and the total current limit is 10 (recorded as max=10)
  2. Then, the access records of each bucket are as follows:
    2.1... Bucket t1 (between 0~0.25s), visited 4 times. At this time, the total number of accesses is recorded as total=4 (at this time, total < = max)
    2.2... Barrel t2 (between 0.25~0.50s), visited 4 times. At this time, the total number of accesses is recorded as total=8 (at this time, total < = max)
    2.3... Barrel t3 (between 0.50~0.75s), visited once. At this time, the total number of accesses is recorded as total=9 (at this time, total < = max)
    2.4... Bucket t4 (between 0.75 and 1.00s), attempted to visit twice. Because max=10, access will be denied when total=11, that is, the second access, because the current total number of windows online is full (total=10 at this time)
    2.5... Bucket t5 (between 1.00 and 1.25s), attempted to visit 3 times. Since t1 has been out of the window at this time, the 4 times of t1 access will be released,
    At this time, the total number of access words is 10 - 4 + 3 = 9 < max = 10, so it can be accessed normally and will not be rejected

The basic flow of sliding window is as follows:

Implementation example of sliding window execution

The implementation example of sliding window is as follows

/**
 * Custom sliding time window demo - Hystrix is similar to this.
 * - The runnable method is used to control the sliding action and reset the bucket value and total value
 *
 * @author lidishan
 */
public class MyDefinedSlideWinDemoLimiter implements RateLimiter, Runnable {
    /** A maximum of 5 requests per second is allowed. This is the default value, which you can specify through the construction method**/
    private static final int DEFAULT_ALLOWED_VISIT_PER_SECOND = 5;
    /** Maximum access per second**/
    private long maxVisitPerSecond;
    /** By default, 1s is divided into ten buckets, which is the default value**/
    private static final int DEFAULT_BUCKET = 10;
    private int bucket;
    /** Current requests per bucket**/
    private static AtomicInteger[] countPerBucket = null;

    /** Total requests**/
    private AtomicInteger count;
    private volatile int index;

    /** Constructor**/
    public MyDefinedSlideWinDemoLimiter() {
        this(DEFAULT_BUCKET, DEFAULT_ALLOWED_VISIT_PER_SECOND);
    }
    public MyDefinedSlideWinDemoLimiter(int bucket, long maxVisitPerSecond) {
        this.bucket = bucket;
        this.maxVisitPerSecond = maxVisitPerSecond;
        countPerBucket = new AtomicInteger[bucket];
        for (int i = 0; i < bucket; i++) {
            countPerBucket[i] = new AtomicInteger();
        }
        count = new AtomicInteger(0);
    }
    /**
     * Exceed limit: whether the current total number of QPS exceeds the maximum value (5 per second by default)
     * Note: this should be > =. Because in fact, if the number of accesses in the bucket is equal to 5, you should restrict the access from the outside
     */
    @Override
    public boolean isOverLimit() {
        return currentQps() >= maxVisitPerSecond;
    }
    @Override
    public int currentQps() {
        return count.get();
    }
    /**
     * Access once, times + 1 (as long as the request comes in + 1), and tell whether to load
     * Please note: put it in the specified bucket
     */
    @Override
    public boolean visit() {
        countPerBucket[index].incrementAndGet();
        count.incrementAndGet();
        return isOverLimit();
    }
    @Override
    public void run() {
        System.out.println("~~~~~~~~~~~~~~~~~~~~~~~~~~Slide the window back~~~~~~~~~~~~~~~~~~~~~~~~~~");
        // The pointer in the bucket slides forward: it indicates that the subsequent visit request should be sent to the next bucket
        index = (index + 1) % bucket;
        // Initialize a new bucket. And take out the old value (in fact, release the value of the current bucket, and then check whether the bucket has been accessed before. If so, subtract the total number of count s, and then tell that it can be accessed)
        int val = countPerBucket[index].getAndSet(0);
        // This step must not change: because a bucket is discarded, the total value must be subtracted~
        if (val == 0) {
            // This bucket is equal to 0, indicating that there is no flow at this time
            System.out.println("~~~~~~~~~~~~~~~~~~~~~~~~~~The window fails to release the flow and continues to maintain the current limit~~~~~~~~~~~~~~~~~~~~~~~~~~");
        } else {
            count.addAndGet(-val);
            System.out.println("~~~~~~~~~~~~~~~~~~~~~~~~~~The window is released[" + val + "]There are two places to visit. You can visit~~~~~~~~~~~~~~~~~~~~~~~~~~");
        }
    }


    public static void main(String[] args) throws Exception {
        MyDefinedSlideWinDemoLimiter rateLimiter = new MyDefinedSlideWinDemoLimiter();
        // Use a thread to slide the window regularly: once every 100ms (generally keep the span of barrels consistent)
        ScheduledExecutorService scheduledExecutorService = Executors.newSingleThreadScheduledExecutor();
        scheduledExecutorService.scheduleAtFixedRate(rateLimiter, 100, 100, TimeUnit.MILLISECONDS);

        // Using single threaded access here, you can transform it into a multi-threaded version
        while (true) {
            String currThreadName = Thread.currentThread().getName();
            boolean overLimit = rateLimiter.isOverLimit();
            if (overLimit) {
                System.out.printf("thread [%s]===The current is limited===´╝îBecause the number of accesses has exceeded the threshold[%s]\n%n", currThreadName, rateLimiter.currentQps());
            } else {
                rateLimiter.visit();
                System.out.printf("thread [%s]Successful access, total number of current accesses[%s]\n%n", currThreadName, rateLimiter.currentQps());
            }
            Thread.sleep(10);
        }
    }
}
public interface RateLimiter {
    // Do you want to limit current
    boolean isOverLimit();
    // Current total QPS value (that is, the total number of accesses in the window period)
    int currentQps();
    // touch; Increase one visit
    boolean visit();
}

Implementation of Hystrix sliding window

Hystrix counts the data through a sliding window. A sliding window contains 10 barrels. The width of each bucket is 1 second, which is responsible for the statistics of success, failure, timeout and rejection times of 1 second in the current time period

That is, four indicators are recorded for each bucket: success amount, failure amount, timeout amount and rejection amount. The total number of current sliding time windows = success amount + failure amount + timeout amount + rejection amount (all bucket buckets)
The core implementation of its Hystrix sliding time window is (just look, rxjava doesn't need to know)

  • The HealthCountsStream provides real-time health check data, in which an object HealthCounts records the number of requests (total number, number of failures, percentage of failures) during the sliding window
  • There is a sliding time window, and there must be a continuous accumulation window, BucketedCumulativeCounterStream
public abstract class BucketedRollingCounterStream<Event extends HystrixEvent, Bucket, Output> extends BucketedCounterStream<Event, Bucket, Output> {
    private Observable<Output> sourceStream;
    private final AtomicBoolean isSourceCurrentlySubscribed = new AtomicBoolean(false);
    protected BucketedRollingCounterStream(HystrixEventStream<Event> stream, final int numBuckets, int bucketSizeInMs,
                                           final Func2<Bucket, Event, Bucket> appendRawEventToBucket,
                                           final Func2<Output, Bucket, Output> reduceBucket) {
        super(stream, numBuckets, bucketSizeInMs, appendRawEventToBucket);
        Func1<Observable<Bucket>, Observable<Output>> reduceWindowToSummary = window -> window.scan(getEmptyOutputValue(), reduceBucket).skip(numBuckets);
        this.sourceStream = bucketedStream      // Data flow, each object represents the bucket stream broken up into buckets generated by the unit window
                .window(numBuckets, 1)          // Aggregate buckets according to the number of buckets in the sliding window emit overlapping windows of buckets
                .flatMap(reduceWindowToSummary) // Aggregate a series of buckets into the last data object convert a window of bucket summaries into a single summary
                .doOnSubscribe(() -> isSourceCurrentlySubscribed.set(true))
                .doOnUnsubscribe(() -> isSourceCurrentlySubscribed.set(false))
                .share()                        // share. The data seen by different subscribers is consistent. multiple subscribers should get same data
                .onBackpressureDrop();          // if there are slow consumers, data should not buffer
    }
    @Override
    public Observable<Output> observe() {
        return sourceStream;
    }
    /* package-private */ boolean isSourceCurrentlySubscribed() {
        return isSourceCurrentlySubscribed.get();
    }
}
public static class HealthCounts {
    private final long totalCount;// total
    private final long errorCount;// Total errors
    private final int errorPercentage;// Error percentage
}

Here is another flowchart of hytrix degradation:

Tags: Java Hystrix

Posted on Mon, 27 Sep 2021 08:59:34 -0400 by anthylon