Using Resilience4j framework to implement asynchronous timeout processing in Java projects

So far in this series, we have learned about Resilience4j and its application Retry and RateLimiter modular. In this article, we will continue to explore Resilience4j through TimeLimiter. We'll see what problems it solves, when and how to use it, and look at some examples.

Code example

Attached to this article On GitHub Working code example for.

What is Resilience4j?

Please refer to the description in the previous article for a quick understanding General working principle of Resilience4j.

What is time limit?

Setting a limit on the time we are willing to wait for the operation to complete is called a time limit. If the operation does not complete within the time we specify, we want to be notified by a timeout error.

Sometimes, this is also called "setting a deadline".

One of the main reasons we do this is to ensure that we don't let users or customers wait indefinitely. A slow service that does not provide any feedback may frustrate users.

Another reason we set time limits on operations is to ensure that we do not consume server resources indefinitely. The timeout value we specified when using Spring's @ Transactional annotation is an example -- in this case, we don't want to consume database resources for a long time.

When will Resilience4j TimeLimiter be used?

Resilience4j TimeLimiter Allows you to set the time limit (timeout) for asynchronous operations implemented using CompleteableFutures.

The completabilefuture class introduced in Java 8 makes asynchronous, non blocking programming easier. You can execute slow methods on different threads to release the current thread to handle other tasks. We can provide a callback to execute when slowMethod() returns:

int slowMethod() {
  // time-consuming computation or remote operation
return 42;
}

CompletableFuture.supplyAsync(this::slowMethod)
.thenAccept(System.out::println);

slowMethod() here can be some computing or remote operation. Typically, we want to set a time limit when making such asynchronous calls. We don't want to wait indefinitely for slowMethod() to return. For example, if slowMethod() takes more than one second, we may want to return the previously calculated, cached value, or even make an error.

In Java 8's completable future, there is no simple way to set the time limit for asynchronous operations. Completabilefuture implements the future interface. Future has an overloaded get() method to specify how long we can wait:

CompletableFuture<Integer> completableFuture = CompletableFuture
  .supplyAsync(this::slowMethod);
Integer result = completableFuture.get(3000, TimeUnit.MILLISECONDS);
System.out.println(result);

But there is a problem -- the get() method is a blocking call. Therefore, it first violates the purpose of using completable future, that is, to release the current thread.

This is what Resilience4j's TimeLimiter solves -- it lets us set time limits on asynchronous operations while retaining the benefits of non blocking when using completable future in Java 8.

This limitation of completable future has been addressed in Java 9. In Java 9 and later versions, we can directly set the time limit using methods such as orTimeout() or completeOnTimeout() on completable future. However, with Resilience4J index and event , it still provides added value compared to a normal Java 9 solution.

Resilience4j TimeLimiter concept

TimeLimiter supports Future and completable Future. But using it with Future is equivalent to Future. Get (long timeout, timeunit). Therefore, we will focus on completable Future in the rest of this article.

Like other Resilience4j modules, TimeLimiter works by decorating our code with the required functions - if the operation is not completed within the specified timeoutDuration in this case, TimeoutException is returned.

We provide timeoutDuration, ScheduledExecutorService and asynchronous operation itself for TimeLimiter, which is represented as the Supplier of CompletionStage. It returns a decoration Supplier of CompletionStage.

Internally, it uses the scheduler to schedule a timeout task -- complete the task of completable future by throwing a TimeoutException. If the operation completes first, TimeLimiter cancels the internal timeout task.

In addition to timeoutDuration, there is another configuration associated with TimeLimiter, cancelRunningFuture. This configuration only Apply to Future is not applicable to completable future. When a timeout occurs, it cancels the running future before throwing a TimeoutException.

Using Resilience4j TimeLimiter module

TimeLimiterRegistry, TimeLimiterConfig, and TimeLimiter are resilience4j-timelimiter The main abstraction of.

Timelimitterregistry is a factory for creating and managing TimeLimiter objects.

TimeLimiterConfig encapsulates timeoutDuration and cancelRunningFuture configurations. Each TimeLimiter object is associated with a TimeLimiterConfig.

TimeLimiter provides auxiliary methods to create or execute decorators for Future and completable Future suppliers.

Let's look at how to use the various functions available in the TimeLimiter module. We will use the same example as the previous articles in this series. Suppose we are setting up a website for an airline to allow its customers to search and book flights. Our service talks with the remote service encapsulated by the FlightSearchService class.

The first step is to create a TimeLimiterConfig:

TimeLimiterConfig config = TimeLimiterConfig.ofDefaults();

This will create a TimeLimiterConfig with default values of timeoutDuration (1000ms) and cancelRunningFuture (true).

Suppose we want to set the timeout value to 2s instead of the default value:

TimeLimiterConfig config = TimeLimiterConfig.custom()
  .timeoutDuration(Duration.ofSeconds(2))
  .build();

Then we create a TimeLimiter:

TimeLimiterRegistry registry = TimeLimiterRegistry.of(config);

TimeLimiter limiter = registry.timeLimiter("flightSearch");

We want asynchronous calls
FlightSearchService.searchFlights(), which returns a list < flight >. Let's express it as supplier < completionstage < list < flight > >:

Supplier<List<Flight>> flightSupplier = () -> service.searchFlights(request);
Supplier<CompletionStage<List<Flight>>> origCompletionStageSupplier =
() -> CompletableFuture.supplyAsync(flightSupplier);

Then we can decorate the Supplier with TimeLimiter:

ScheduledExecutorService scheduler =
  Executors.newSingleThreadScheduledExecutor();
Supplier<CompletionStage<List<Flight>>> decoratedCompletionStageSupplier =  
  limiter.decorateCompletionStage(scheduler, origCompletionStageSupplier);

Finally, let's call the decorated asynchronous operation:

decoratedCompletionStageSupplier.get().whenComplete((result, ex) -> {
  if (ex != null) {
    System.out.println(ex.getMessage());
  }
  if (result != null) {
    System.out.println(result);
  }
});

The following is an example output of a successful flight search, which takes less than 2 seconds timeoutDuration we specified:

Searching for flights; current time = 19:25:09 783; current thread = ForkJoinPool.commonPool-worker-3

Flight search successful

[Flight{flightNumber='XY 765', flightDate='08/30/2020', from='NYC', to='LAX'}, Flight{flightNumber='XY 746', flightDate='08/30/2020', from='NYC', to='LAX'}] on thread ForkJoinPool.commonPool-worker-3

This is an example output of a timed out flight search:

Exception java.util.concurrent.TimeoutException: TimeLimiter 'flightSearch' recorded a timeout exception on thread pool-1-thread-1 at 19:38:16 963

Searching for flights; current time = 19:38:18 448; current thread = ForkJoinPool.commonPool-worker-3

Flight search successful at 19:38:18 461

The timestamp and thread name above indicate that the calling thread will receive a TimeoutException even if the asynchronous operation is completed later on another thread.

If we want to create a decorator and reuse it in different locations in the code base, we will use decorateCompletionStage(). If we want to create it and execute supplier < completionstage > immediately, we can use the executeCompletionStage() instance method instead:

CompletionStage<List<Flight>> decoratedCompletionStage =  
  limiter.executeCompletionStage(scheduler, origCompletionStageSupplier);

TimeLimiter event

TimeLimiter has an EventPublisher that generates events of type TimeLimiterOnSuccessEvent, TimeLimiterOnErrorEvent, and TimeLimiterOnTimeoutEvent. We can listen to these events and record them, for example:

TimeLimiter limiter = registry.timeLimiter("flightSearch");

limiter.getEventPublisher().onSuccess(e -> System.out.println(e.toString()));

limiter.getEventPublisher().onError(e -> System.out.println(e.toString()));

limiter.getEventPublisher().onTimeout(e -> System.out.println(e.toString()));

The sample output shows the contents of the record:

2020-08-07T11:31:48.181944: TimeLimiter 'flightSearch' recorded a successful call.

... other lines omitted ...

2020-08-07T11:31:48.582263: TimeLimiter 'flightSearch' recorded a timeout exception.

TimeLimiter metrics

TimeLimiter tracks the number of successful, failed, and timed out calls.

First, we create TimeLimiterConfig, TimeLimiterRegistry, and TimeLimiter as usual. Then, we create a MeterRegistry and bind the TimeLimiterRegistry to it:

MeterRegistry meterRegistry = new SimpleMeterRegistry();
TaggedTimeLimiterMetrics.ofTimeLimiterRegistry(registry)
  .bindTo(meterRegistry);

After several time limited operations, we display the captured indicators:

Consumer<Meter> meterConsumer = meter -> {
  String desc = meter.getId().getDescription();
  String metricName = meter.getId().getName();
  String metricKind = meter.getId().getTag("kind");
  Double metricValue =
    StreamSupport.stream(meter.measure().spliterator(), false)
    .filter(m -> m.getStatistic().name().equals("COUNT"))
    .findFirst()
    .map(Measurement::getValue)
    .orElse(0.0);
  System.out.println(desc + " - " +
                     metricName +
                     "(" + metricKind + ")" +
                     ": " + metricValue);
};
meterRegistry.forEachMeter(meterConsumer);

Here are some sample outputs:

The number of timed out calls - resilience4j.timelimiter.calls(timeout): 6.0

The number of successful calls - resilience4j.timelimiter.calls(successful): 4.0

The number of failed calls - resilience4j.timelimiter.calls(failed): 0.0

In practical application, we will regularly export the data to the monitoring system and analyze it on the dashboard.

Pitfalls and good practices in implementing time limits

Typically, we handle two operations - query (or read) and command (or write). It is safe to time limit queries because we know they will not change the state of the system. The searchFlights() operation we saw is an example of a query operation.

Commands usually change the state of the system. The bookFlights() operation will be an example of the command. When time limiting a command, we must remember that when we time out, the command is likely to still be running. For example, TimeoutException on the bookFlights() call does not necessarily mean that the command failed.

In this case, we need to manage the user experience - perhaps at the timeout, we can inform the user that the operation takes longer than we expected. We can then query the upstream to check the status of the operation and notify the user later.

conclusion

In this article, we learned how to use the TimeLimiter module of Resilience4j to set time limits for asynchronous, non blocking operations. We learned when to use it and how to configure it through some practical examples.

You can use On GitHub The code demonstrates a complete application to illustrate these ideas.

This article is translated from:
https://reflectoring.io/time-limiting-with-resilience4j/

Tags: Java

Posted on Thu, 25 Nov 2021 23:13:05 -0500 by raytri