CompletableFuture principle and practice

1 Why Parallel Loading is Needed

For example: The API service of the food delivery merchant is a typical I/O-intensive (I/O Bound) service. In addition, there are two major features of the takeaway business transaction business:

• The server must return all the contents of the order card at one time : According to the "Incremental Synchronization Protocol Note 1" between the merchant and the server, the server must return all the information of the order at one time, including the main information of the order, commodities, settlement, delivery, user information, Rider information, meal damage, refunds, customer service compensation (refer to the screenshot of the order card of Meituan or Ele.me), etc., need to obtain data from more than 30 downstream services. Under certain conditions, such as logging in for the first time or not logging in for a long time, the client will pull multiple orders in pages, so that more remote calls will be initiated.
• Frequent interactions between merchants and servers : Merchants are sensitive to changes in order status, and multiple push-pull mechanisms ensure that each change can reach the merchant, resulting in frequent interactions between the App and the server, and each change needs to pull the latest full content of the order.

With such a large traffic on the takeaway transaction link, in order to ensure the user experience of the merchant and the high performance of the interface, it is inevitable to obtain data from the downstream in parallel.

2 Implementation of parallel loading

Obtaining data from downstream in parallel can be divided into synchronous model and asynchronous model in terms of IO model .

2.1 Synchronization model

The most common way to get data from various services is to call synchronously, as shown in the following figure:

In the scenario of synchronous calls, the interface takes a long time and has poor performance, and the interface response time is T > T1+T2+T3+...+Tn. At this time, in order to shorten the response time of the interface, thread pools are generally used to obtain data in parallel. Merchants This is the way the end order card is assembled.

This method leads to low resource utilization due to the following two reasons:

• A large amount of CPU resources are wasted on blocking and waiting , resulting in low utilization of CPU resources. Before Java 8, callbacks were generally used to reduce blocking, but extensive use of callbacks caused the notorious callback hell problem, which greatly reduced code readability and maintainability.
• In order to increase concurrency, more additional thread pools will be introduced . As the number of CPU scheduling threads increases, it will lead to more serious resource contention. Precious CPU resources will be wasted on context switching, and the threads themselves will also occupy the system. resources, and cannot be increased indefinitely.

Under the synchronous model, hardware resources cannot be fully utilized , and system throughput is likely to reach the bottleneck.

2.2 NIO asynchronous model

We mainly use the following two methods to reduce the scheduling overhead and blocking time of the thread pool:

• The number of threads can be reduced by means of RPC NIO asynchronous calls, thereby reducing the overhead of scheduling (context switching). For example, for asynchronous calls of Dubbo, please refer to the article "Dubbo Caller Asynchronous" .
• By introducing CompletableFuture (hereinafter referred to as CF) to orchestrate the business process and reduce the blocking between dependencies. This article mainly describes the use and principle of CompletableFuture.

2.3 Why choose CompletableFuture?

We first conducted a horizontal survey on widely popular solutions in the industry, mainly including Future, CompletableFuture Note 2, RxJava, and Reactor. Their characteristics are compared as follows:

	Future	CompletableFuture	RxJava	Reactor
Composable (combinable)	❌	✔️	✔️	✔️
Asynchronous	✔️	✔️	✔️	✔️
Operator fusion	❌	❌	✔️	✔️
Lazy (delayed execution)	❌	❌	✔️	✔️
Backpressure	❌	❌	✔️	✔️

• Composable : Multiple dependent operations can be arranged in different ways. For example, CompletableFuture provides thenCompose, thenCombine and other methods starting with then. These methods support the "combinable" feature.
• Operation Fusion : Combining multiple operators used in a data flow in some way reduces overhead (time, memory).
• Delayed execution : Actions are not executed immediately, but are triggered when explicitly instructed to do so. For example, Reactor only triggers operations when there are subscribers.
• Back pressure : The processing speed of some asynchronous stages cannot keep up, and direct failure will lead to a large amount of data loss, which is unacceptable for the business. At this time, feedback to the upstream producer is required to reduce the amount of calls.

RxJava and Reactor are obviously more powerful. They provide more function calling methods and support more features, but they also bring greater learning costs. The features we most need for this integration are "asynchronous" and "combinable". After comprehensive consideration, we chose CompletableFuture, which has a relatively low learning cost.

3 CompletableFuture use and principle

3.1 Background and definition of CompletableFuture

3.1.1 Problems solved by CompletableFuture

CompletableFuture was introduced by Java 8. Before Java8, we generally implemented asynchrony through Future.

• Future is used to represent the results of asynchronous calculations. The results can only be obtained by blocking or polling, and it does not support setting callback methods. Before Java 8, if you want to set callbacks, you usually use Guava's ListenableFuture. The introduction of callbacks will lead to notorious The callback hell (the following example will be specifically demonstrated through the use of ListenableFuture).
• CompletableFuture extends Future, which can process calculation results by setting callbacks. It also supports combination operations and further orchestration, and at the same time solves the problem of callback hell to a certain extent.

The following will illustrate the asynchronous difference through ListenableFuture and CompletableFuture. Assume that there are three operations step1, step2, and step3 that have dependencies, and the execution of step3 depends on the results of step1 and step2.

The implementation of Future(ListenableFuture) (callback hell) is as follows:

ExecutorService executor = Executors.newFixedThreadPool(5);
ListeningExecutorService guavaExecutor = MoreExecutors.listeningDecorator(executor);
ListenableFuture<String> future1 = guavaExecutor.submit(() -> {
    
    
    //step 1
    System.out.println("执行step 1");
    return "step1 result";
});
ListenableFuture<String> future2 = guavaExecutor.submit(() -> {
    
    
    //step 2
    System.out.println("执行step 2");
    return "step2 result";
});
ListenableFuture<List<String>> future1And2 = Futures.allAsList(future1, future2);
Futures.addCallback(future1And2, new FutureCallback<List<String>>() {
    
    
    @Override
    public void onSuccess(List<String> result) {
    
    
        System.out.println(result);
        ListenableFuture<String> future3 = guavaExecutor.submit(() -> {
    
    
            System.out.println("执行step 3");
            return "step3 result";
        });
        Futures.addCallback(future3, new FutureCallback<String>() {
    
    
            @Override
            public void onSuccess(String result) {
    
    
                System.out.println(result);
            }        
            @Override
            public void onFailure(Throwable t) {
    
    
            }
        }, guavaExecutor);
    }

    @Override
    public void onFailure(Throwable t) {
    
    
    }}, guavaExecutor);

CompletableFuture is implemented as follows:

ExecutorService executor = Executors.newFixedThreadPool(5);
CompletableFuture<String> cf1 = CompletableFuture.supplyAsync(() -> {
    
    
    System.out.println("执行step 1");
    return "step1 result";
}, executor);
CompletableFuture<String> cf2 = CompletableFuture.supplyAsync(() -> {
    
    
    System.out.println("执行step 2");
    return "step2 result";
});
cf1.thenCombine(cf2, (result1, result2) -> {
    
    
    System.out.println(result1 + " , " + result2);
    System.out.println("执行step 3");
    return "step3 result";
}).thenAccept(result3 -> System.out.println(result3));

Obviously, the implementation of CompletableFuture is more concise and more readable.

3.1.2 Definition of CompletableFuture

CompletableFuture implements two interfaces (as shown in the figure above): Future and CompletionStage. Future represents the result of asynchronous calculation, and CompletionStage is used to represent a step (Stage) in the asynchronous execution process. This step may be triggered by another CompletionStage. With the completion of the current step, it may also trigger the execution of a series of other CompletionStages. . Therefore, we can arrange and combine these steps in a variety of ways according to the actual business. The CompletionStage interface defines such capabilities. We can combine and arrange these steps through functional programming methods such as thenAppy and thenCompose provided by it.

3.2 The use of CompletableFuture

Let's use an example to explain how to use CompletableFuture. Using CompletableFuture is also the process of building a dependency tree. The completion of a CompletableFuture triggers the execution of a series of other CompletableFutures that depend on it:

As shown in the figure above, here is a flow of a business interface, including 5 steps of CF1\CF2\CF3\CF4\CF5, and depicts the dependencies between these steps, each step can be an RPC call , a database operation or a local method call, etc., when using CompletableFuture for asynchronous programming, each step in the diagram will generate a CompletableFuture object, and the final result will also be represented by a CompletableFuture.

According to the number of CompletableFuture dependencies, it can be divided into the following categories: zero dependencies, unary dependencies, binary dependencies, and multiple dependencies.

3.2.1 Zero dependency: creation of CompletableFuture

Let's first look at how to create a new CompletableFuture without relying on other CompletableFutures:

As shown in the red link in the above figure, after the interface receives the request, it first initiates two asynchronous calls CF1 and CF2. There are three main methods:

ExecutorService executor = Executors.newFixedThreadPool(5);
//1、使用runAsync或supplyAsync发起异步调用
CompletableFuture<String> cf1 = CompletableFuture.supplyAsync(() -> {
    
    
  return "result1";
}, executor);
//2、CompletableFuture.completedFuture()直接创建一个已完成状态的CompletableFuture
CompletableFuture<String> cf2 = CompletableFuture.completedFuture("result2");
//3、先初始化一个未完成的CompletableFuture，然后通过complete()、completeExceptionally()，完成该CompletableFuture
CompletableFuture<String> cf = new CompletableFuture<>();
cf.complete("success");

A typical usage scenario of the third method is to convert the callback method to CompletableFuture, and then rely on the capability of CompletableFure for call arrangement. The example is as follows:

@FunctionalInterface
public interface ThriftAsyncCall {
    
    
    void invoke() throws TException;
}
 /**
  * 该方法为rpc注册监听的封装，可以作为其他实现的参照
  * OctoThriftCallback 为thrift回调方法
  * ThriftAsyncCall 为自定义函数，用来表示一次thrift调用（定义如上）
  */
  public static <T> CompletableFuture<T> toCompletableFuture(final OctoThriftCallback<?,T> callback , ThriftAsyncCall thriftCall) {
    
    
   //新建一个未完成的CompletableFuture
   CompletableFuture<T> resultFuture = new CompletableFuture<>();
   //监听回调的完成，并且与CompletableFuture同步状态
   callback.addObserver(new OctoObserver<T>() {
    
    
       @Override
       public void onSuccess(T t) {
    
    
           resultFuture.complete(t);
       }
       @Override
       public void onFailure(Throwable throwable) {
    
    
           resultFuture.completeExceptionally(throwable);
       }
   });
   if (thriftCall != null) {
    
    
       try {
    
    
           thriftCall.invoke();
       } catch (TException e) {
    
    
           resultFuture.completeExceptionally(e);
       }
   }
   return resultFuture;
  }

3.2.2 Unary dependency: depend on a CF

As shown in the red link above, CF3 and CF5 depend on CF1 and CF2 respectively. This dependence on a single CompletableFuture can be realized through methods such as thenApply, thenAccept, thenCompose, and the code is as follows:

CompletableFuture<String> cf3 = cf1.thenApply(result1 -> {
    
    
  //result1为CF1的结果
  //......
  return "result3";
});
CompletableFuture<String> cf5 = cf2.thenApply(result2 -> {
    
    
  //result2为CF2的结果
  //......
  return "result5";
});

3.2.3 Binary dependency: depends on two CFs

As shown in the red link in the above figure, CF4 depends on two CF1 and CF2 at the same time. This binary dependency can be realized through callbacks such as thenCombine, as shown in the following code:

CompletableFuture<String> cf4 = cf1.thenCombine(cf2, (result1, result2) -> {
    
    
  //result1和result2分别为cf1和cf2的结果
  return "result4";
});

3.2.4 Multiple dependencies: rely on multiple CFs

As shown in the red link in the above figure, the end of the whole process depends on three steps CF3, CF4, and CF5. This multi-dependency can be achieved by the allOfor anyOfmethod. The difference is that it is used when multiple dependencies are required to be completed allOf. When multiple dependencies Any one of them can be used when it is ready anyOf, as shown in the following code:

CompletableFuture<Void> cf6 = CompletableFuture.allOf(cf3, cf4, cf5);
CompletableFuture<String> result = cf6.thenApply(v -> {
    
    
  //这里的join并不会阻塞，因为传给thenApply的函数是在CF3、CF4、CF5全部完成时，才会执行 。
  result3 = cf3.join();
  result4 = cf4.join();
  result5 = cf5.join();
  //根据result3、result4、result5组装最终result;
  return "result";
});

3.3 Principle of CompletableFuture

CompletableFuture contains two fields: result and stack . result is used to store the results of the current CF, and stack (Completion) indicates the Dependency Actions (Dependency Actions) that need to be triggered after the completion of the current CF to trigger the calculation of the CF that depends on it. There can be multiple dependent actions (indicating that there are multiple dependencies on it CF), stored in the form of a stack ( Treiber stack ), and stack represents the top element of the stack.

This method is similar to the "observer pattern", and the dependent actions (Dependency Action) are encapsulated in a separate Completion subclass. The following is the Completion class relationship structure diagram. Each method in CompletableFuture corresponds to a subclass of Completion in the diagram, and Completion itself is the base class of observers .

• UniCompletion inherits Completion and is the base class of unary dependencies. For example, UniApply, the implementation class of thenApply, inherits from UniCompletion.
• BiCompletion inherits UniCompletion, which is the base class of binary dependencies and also the base class of multiple dependencies. For example, thenCombine's implementation class BiRelay inherits from BiCompletion.

3.3.1 The design idea of ​​CompletableFuture

According to the design idea similar to "observer mode", the principle analysis can start from two aspects: "observer" and "observed". Since there are many types of callbacks, but the structural differences are not large, so here we only take thenApply in the unary dependency as an example, and do not enumerate all callback types. As shown below:

3.3.1.1 Observed

1. Each CompletableFuture can be regarded as an observer, and there is a linked list member variable stack of Completion type inside it, which is used to store all observers registered to it. When the observed object completes execution, the stack property will pop up, and the observers registered in it will be notified in turn. In the above example, step fn2 is encapsulated in UniApply as an observer.
2. The result attribute in the observed CF is used to store the returned result data. This may be the return value of an RPC call, or any object, which corresponds to the execution result of step fn1 in the above example.

3.3.1.2 Observers

CompletableFuture supports many callback methods, such as thenAccept, thenApply, exceptionally, etc. These methods receive a parameter f of function type, generate an object of Completion type (ie observer), and assign the input function f to the member variable fn of Completion, Then check whether the current CF is in the completed state (that is, result != null), if it is completed, directly trigger fn, otherwise, add the observer Completion to the observer chain stack of CF, and try to trigger again, if the observed object has not finished executing Then the notification is triggered after its execution is completed.

1. The dep attribute in the observer: points to its corresponding CompletableFuture. In the above example, dep points to CF2.
2. The src attribute in the observer: points to the CompletableFuture it depends on. In the above example, src points to CF1.
3. The fn attribute in the Completion of the observer: used to store the specific function waiting to be called back. It should be noted here that different callback methods (thenAccept, thenApply, exceptionally, etc.) receive different function types, that is, there are many types of fn, and in the above example, fn points to fn2.

3.3.2 Overall process

3.3.2.1 Unary dependencies

Here still take thenApply as an example to illustrate the process of unary dependency:

1. Register the observer Completion to CF1, and CF1 pushes Completion onto the stack.
2. When the operation of CF1 is completed, the result will be assigned to the result attribute in CF1.
3. Pop the stack one by one to notify the observer to try to run.

The preliminary process design is shown in the figure above. Here are a few concurrent issues about registration and notification. You can think about it:

Q1 : Before the observer is registered, if the CF has been executed and the notification has been issued, will the observer never be triggered because the notification is missed? A1 : No. When registering, check whether the dependent CF has been completed. If not completed (result == null), the observer will be pushed onto the stack, and if completed (result != null), the observer operation will be triggered directly.

Q2 : There will be a judgment of "result == null" before "push into the stack". These two operations are non-atomic operations. The implementation of CompletableFufure does not lock the two operations, and the completion time is between these two operations. , the observer is still not notified, is it still not triggered?

A2 : No. After being pushed into the stack, check whether the CF is completed again, and trigger if it is completed.

Q3 : When relying on multiple CFs, the observer will be pushed into the stack of all dependent CFs, and will be performed when each CF is completed. Will it cause an operation to be executed multiple times? As shown in the figure below, how to prevent CF3 from being triggered multiple times when CF1 and CF2 are completed at the same time.

A3 : The implementation of CompletableFuture solves this problem in this way: before execution, the observer will first set a status bit through the CAS operation, and change the status from 0 to 1. If the observer has already executed, the CAS operation will fail and the execution will be cancelled.

Through the analysis of the above three problems, it can be seen that when CompletableFuture handles parallel problems, there is no locking operation in the whole process, which greatly improves the execution efficiency of the program. After we take parallel issues into consideration, we can get a complete overall flow chart as follows:

The callback methods supported by CompletableFuture are very rich, but as described in the overall flow chart in the previous chapter, their overall process is consistent. All callbacks reuse the same process architecture, and different callback listeners are differentiated through strategy patterns .

3.3.2.2 Binary dependencies

Let's take thenCombine as an example to illustrate binary dependencies:

The thenCombine operation expresses a dependency on two CompletableFutures. Its observer implementation class is BiApply. As shown in the figure above, BiApply associates the two dependent CFs through the src and snd attributes, and the type of the fn attribute is BiFunction. Different from a single dependency, if the dependent CF is not completed, thenCombine will try to push BiApply into the stack of the two dependent CFs, and each dependent CF will try to trigger the observer BiApply when it is completed. BiApply will check whether both dependencies are complete, and if so, start execution. In order to solve the problem of repeated triggering, the CAS operation mentioned in the previous chapter is also used. When executing, the status bit will be set through CAS to avoid repeated triggering.

3.3.2.3 Multiple dependencies

The callback methods that depend on multiple CompletableFutures include allOf, anyOf, and the difference is that allOfthe implementation class of the observer is BiRelay, and the callback will only be executed after all the dependent CFs are completed; while the anyOfimplementation class of the observer is OrRelay, and any one of the dependent CFs will be completed. trigger. Both are implemented by building multiple dependent CFs into a balanced binary tree, and notifying layer by layer of execution results until the root node triggers callback monitoring.

3.3.3 Summary

This chapter is a science popularization of the implementation principle of CompletableFuture, which aims to explain the implementation principle of CompletableFuture clearly through structural diagrams, flowcharts, and text descriptions without pasting the source code. Translate the obscure source code into the flow chart of the "Overall Process" chapter, and integrate the logic of concurrent processing for everyone to understand.

4 Practice Summary

In the process of asynchronization of merchant-side API, we encountered some problems, some of which may be hidden, and the experience of dealing with these problems is sorted out below. I hope to help more students, and everyone can avoid some pitfalls.

4.1 Thread blocking problem

4.1.1 On which thread is the code executed?

To reasonably manage thread resources, the most basic prerequisite is to clearly know which thread each line of code will execute on when writing code. Let's take a look at the execution thread of CompletableFuture.

CompletableFuture implements the CompletionStage interface, supports various combination operations through rich callback methods, and each combination scene has two methods: synchronous and asynchronous.

Synchronous methods (that is, methods without the Async suffix) have two cases.

• If the dependent operation has been executed during registration, it will be executed directly by the current thread.
• If the dependent operation has not been executed at the time of registration, it will be executed by the callback thread.

Asynchronous method (that is, the method with the Async suffix): you can choose whether to pass the thread pool parameter Executor to run in the specified thread pool; when the Executor is not passed, the shared thread pool CommonPool in the ForkJoinPool will be used (the size of the CommonPool is the number of CPU cores - 1. If it is an IO-intensive application, the number of threads may become a bottleneck).

For example:

ExecutorService threadPool1 = new ThreadPoolExecutor(10, 10, 0L, TimeUnit.MILLISECONDS, new ArrayBlockingQueue<>(100));
CompletableFuture<String> future1 = CompletableFuture.supplyAsync(() -> {
    
    
    System.out.println("supplyAsync 执行线程：" + Thread.currentThread().getName());
    //业务操作
    return "";
}, threadPool1);
//此时，如果future1中的业务操作已经执行完毕并返回，则该thenApply直接由当前main线程执行；否则，将会由执行以上业务操作的threadPool1中的线程执行。
future1.thenApply(value -> {
    
    
    System.out.println("thenApply 执行线程：" + Thread.currentThread().getName());
    return value + "1";
});
//使用ForkJoinPool中的共用线程池CommonPool
future1.thenApplyAsync(value -> {
    
    
//do something
  return value + "1";
});
//使用指定线程池
future1.thenApplyAsync(value -> {
    
    
//do something
  return value + "1";
}, threadPool1);

4.2 Notes on thread pool

4.2.1 Asynchronous callbacks need to be passed to the thread pool

As mentioned earlier, the asynchronous callback method can choose whether to pass the thread pool parameter Executor. Here we recommend forcing the thread pool to be passed, and the thread pool is isolated according to the actual situation .

When the thread pool is not passed, the common thread pool CommonPool in the ForkJoinPool will be used. All calls here will share the thread pool. The number of core threads = the number of processors - 1 (the number of single-core core threads is 1), and all asynchronous callbacks will be shared. In the CommonPool, core and non-core businesses compete for threads in the same pool, which can easily become a system bottleneck. Manually passing thread pool parameters can make it easier to adjust parameters, and different thread pools can be allocated to different businesses to isolate resources and reduce mutual interference between different businesses.

4.2.2 Thread pool circular reference can lead to deadlock

public Object doGet() {
    
    
  ExecutorService threadPool1 = new ThreadPoolExecutor(10, 10, 0L, TimeUnit.MILLISECONDS, new ArrayBlockingQueue<>(100));
  CompletableFuture cf1 = CompletableFuture.supplyAsync(() -> {
    
    
  //do sth
    return CompletableFuture.supplyAsync(() -> {
    
    
        System.out.println("child");
        return "child";
      }, threadPool1).join();//子任务
    }, threadPool1);
  return cf1.join();
}

As shown in the above code block, the third line of the doGet method requests threads from threadPool1 through supplyAsync, and the internal subtasks request threads from threadPool1. The size of threadPool1 is 10. When 10 requests arrive at the same time, threadPool1 will be full. When subtasks request threads, they will enter the blocking queue, but the completion of the parent task depends on the subtasks. At this time, the subtasks cannot get threads. , the parent task could not be completed. The main thread executes cf1.join() into a blocked state and can never recover.

In order to fix this problem, it is necessary to isolate the thread pool of the parent task and the child task, and the two tasks request different thread pools to avoid blocking caused by circular dependencies.

4.2.3 Asynchronous RPC call Be careful not to block the IO thread pool

After the service is asynchronized, many steps will depend on the result of the asynchronous RPC call. At this time, special attention should be paid. If the asynchronous RPC based on NIO (such as Netty) is used, the return result is set by the IO thread, that is, the callback method is set by IO thread triggers, CompletableFuture synchronous callback (such as thenApply, thenAccept and other methods without Async suffix), if the return result of the dependent asynchronous RPC call, then these synchronous callbacks will run on the IO thread, and the entire service has only one IO thread pool, which means It is necessary to ensure that there is no time-consuming logic such as blocking in the synchronous callback, otherwise the IO thread will be occupied until the execution of these logics is completed, affecting the response of the entire service.

4.3 Others

4.3.1 Exception handling

Since asynchronously executed tasks are executed on other threads, and exception information is stored in the thread stack, the current thread cannot catch exceptions through try\catch unless it is blocked and waits for the result to be returned. CompletableFuture provides exception catch callback exceptionally, which is equivalent to try\catch in synchronous call. The method of use is as follows:

@Autowired
private WmOrderAdditionInfoThriftService wmOrderAdditionInfoThriftService;//内部接口
public CompletableFuture<Integer> getCancelTypeAsync(long orderId) {
    
    
    CompletableFuture<WmOrderOpRemarkResult> remarkResultFuture = wmOrderAdditionInfoThriftService.findOrderCancelledRemarkByOrderIdAsync(orderId);//业务方法，内部会发起异步rpc调用
    return remarkResultFuture
      .exceptionally(err -> {
    
    //通过exceptionally 捕获异常，打印日志并返回默认值
         log.error("WmOrderRemarkService.getCancelTypeAsync Exception orderId={}", orderId, err);
         return 0;
      });
}

One thing to note is that CompletableFuture wraps the exception in the callback method. Most exceptions are encapsulated into CompletionException and thrown, and the real exception is stored in the cause attribute. Therefore, if the call chain has been processed by a callback method, you need to use the Throwable.getCause() method to extract the real exception. However, in some cases, the real exception will be returned directly ( Stack Overflow discussion ), it is better to use the tool class to extract the exception, as shown in the following code:

@Autowired
private WmOrderAdditionInfoThriftService wmOrderAdditionInfoThriftService;//内部接口
public CompletableFuture<Integer> getCancelTypeAsync(long orderId) {
    
    
    CompletableFuture<WmOrderOpRemarkResult> remarkResultFuture = wmOrderAdditionInfoThriftService.findOrderCancelledRemarkByOrderIdAsync(orderId);//业务方法，内部会发起异步rpc调用
    return remarkResultFuture
          .thenApply(result -> {
    
    //这里增加了一个回调方法thenApply，如果发生异常thenApply内部会通过new CompletionException(throwable) 对异常进行包装
      //这里是一些业务操作
        })
      .exceptionally(err -> {
    
    //通过exceptionally 捕获异常，这里的err已经被thenApply包装过，因此需要通过Throwable.getCause()提取异常
         log.error("WmOrderRemarkService.getCancelTypeAsync Exception orderId={}", orderId, ExceptionUtils.extractRealException(err));
         return 0;
      });
}

A custom tool class ExceptionUtils is used in the above code to extract CompletableFuture exceptions. When using CompletableFuture for asynchronous programming, you can directly use this tool class to handle exceptions. The implementation code is as follows:

public class ExceptionUtils {
    
    
    public static Throwable extractRealException(Throwable throwable) {
    
    
          //这里判断异常类型是否为CompletionException、ExecutionException，如果是则进行提取，否则直接返回。
        if (throwable instanceof CompletionException || throwable instanceof ExecutionException) {
    
    
            if (throwable.getCause() != null) {
    
    
                return throwable.getCause();
            }
        }
        return throwable;
    }
}

4.3.2 Introduction of precipitation tools and methods

In the process of practice, we have accumulated some common tools and methods, which can be used directly when developing with CompletableFuture. For details, please refer to the "Appendix".

5 Asynchronous Benefits

Through the asynchronous transformation, the performance of the API system has been significantly improved, and the benefits compared with those before the transformation are as follows:

• The throughput of the core interface has been greatly improved. Before the transformation of the order polling interface, the TP99 was 754ms, but after the transformation, it was reduced to 408ms.
• The number of servers is reduced by 1/3.

6 References

1. CompletableFuture (Java Platform SE 8 )
2. java - Does CompletionStage always wrap exceptions in CompletionException? - Stack Overflow
3. exception - Surprising behavior of Java 8 CompletableFuture exceptionally method - Stack Overflow
4. Documentation | Apache Dubbo

7 Glossary and Remarks

Note 1: "Incremental synchronization" refers to the order incremental data synchronization protocol between the merchant client and the server. The client uses this protocol to obtain new orders and orders whose status has changed.

Note 2: The Java version that all technical points involved in this article depend on is JDK 8, and the feature analysis supported by CompletableFuture is also based on this version.

appendix

custom function

@FunctionalInterface
public interface ThriftAsyncCall {
    
    
    void invoke() throws TException ;
}

CompletableFuture processing tool class

/**
 * CompletableFuture封装工具类
 */
@Slf4j
public class FutureUtils {
    
    
/**
 * 该方法为rpc注册监听的封装，可以作为其他实现的参照
 * OctoThriftCallback 为thrift回调方法
 * ThriftAsyncCall 为自定义函数，用来表示一次thrift调用（定义如上）
 */
public static <T> CompletableFuture<T> toCompletableFuture(final OctoThriftCallback<?,T> callback , ThriftAsyncCall thriftCall) {
    
    
    CompletableFuture<T> thriftResultFuture = new CompletableFuture<>();
    callback.addObserver(new OctoObserver<T>() {
    
    
        @Override
        public void onSuccess(T t) {
    
    
            thriftResultFuture.complete(t);
        }
        @Override
        public void onFailure(Throwable throwable) {
    
    
            thriftResultFuture.completeExceptionally(throwable);
        }
    });
    if (thriftCall != null) {
    
    
        try {
    
    
            thriftCall.invoke();
        } catch (TException e) {
    
    
            thriftResultFuture.completeExceptionally(e);
        }
    }
    return thriftResultFuture;
}
  /**
   * 设置CF状态为失败
   */
  public static <T> CompletableFuture<T> failed(Throwable ex) {
    
    
   CompletableFuture<T> completableFuture = new CompletableFuture<>();
   completableFuture.completeExceptionally(ex);
   return completableFuture;
  }
  /**
   * 设置CF状态为成功
   */
  public static <T> CompletableFuture<T> success(T result) {
    
    
   CompletableFuture<T> completableFuture = new CompletableFuture<>();
   completableFuture.complete(result);
   return completableFuture;
  }
  /**
   * 将List<CompletableFuture<T>> 转为 CompletableFuture<List<T>>
   */
  public static <T> CompletableFuture<List<T>> sequence(Collection<CompletableFuture<T>> completableFutures) {
    
    
   return CompletableFuture.allOf(completableFutures.toArray(new CompletableFuture<?>[0]))
           .thenApply(v -> completableFutures.stream()
                   .map(CompletableFuture::join)
                   .collect(Collectors.toList())
           );
  }
  /**
   * 将List<CompletableFuture<List<T>>> 转为 CompletableFuture<List<T>>
   * 多用于分页查询的场景
   */
  public static <T> CompletableFuture<List<T>> sequenceList(Collection<CompletableFuture<List<T>>> completableFutures) {
    
    
   return CompletableFuture.allOf(completableFutures.toArray(new CompletableFuture<?>[0]))
           .thenApply(v -> completableFutures.stream()
                   .flatMap( listFuture -> listFuture.join().stream())
                   .collect(Collectors.toList())
           );
  }
  /*
   * 将List<CompletableFuture<Map<K, V>>> 转为 CompletableFuture<Map<K, V>>
   * @Param mergeFunction 自定义key冲突时的merge策略
   */
  public static <K, V> CompletableFuture<Map<K, V>> sequenceMap(
       Collection<CompletableFuture<Map<K, V>>> completableFutures, BinaryOperator<V> mergeFunction) {
    
    
   return CompletableFuture
           .allOf(completableFutures.toArray(new CompletableFuture<?>[0]))
           .thenApply(v -> completableFutures.stream().map(CompletableFuture::join)
                   .flatMap(map -> map.entrySet().stream())
                   .collect(Collectors.toMap(Entry::getKey, Entry::getValue, mergeFunction)));
  }
  /**
   * 将List<CompletableFuture<T>> 转为 CompletableFuture<List<T>>，并过滤调null值
   */
  public static <T> CompletableFuture<List<T>> sequenceNonNull(Collection<CompletableFuture<T>> completableFutures) {
    
    
   return CompletableFuture.allOf(completableFutures.toArray(new CompletableFuture<?>[0]))
           .thenApply(v -> completableFutures.stream()
                   .map(CompletableFuture::join)
                   .filter(e -> e != null)
                   .collect(Collectors.toList())
           );
  }
  /**
   * 将List<CompletableFuture<List<T>>> 转为 CompletableFuture<List<T>>，并过滤调null值
   * 多用于分页查询的场景
   */
  public static <T> CompletableFuture<List<T>> sequenceListNonNull(Collection<CompletableFuture<List<T>>> completableFutures) {
    
    
   return CompletableFuture.allOf(completableFutures.toArray(new CompletableFuture<?>[0]))
           .thenApply(v -> completableFutures.stream()
                   .flatMap( listFuture -> listFuture.join().stream().filter(e -> e != null))
                   .collect(Collectors.toList())
           );
  }
  /**
   * 将List<CompletableFuture<Map<K, V>>> 转为 CompletableFuture<Map<K, V>>
   * @Param filterFunction 自定义过滤策略
   */
  public static <T> CompletableFuture<List<T>> sequence(Collection<CompletableFuture<T>> completableFutures,
                                                     Predicate<? super T> filterFunction) {
    
    
   return CompletableFuture.allOf(completableFutures.toArray(new CompletableFuture<?>[0]))
           .thenApply(v -> completableFutures.stream()
                   .map(CompletableFuture::join)
                   .filter(filterFunction)
                   .collect(Collectors.toList())
           );
  }
  /**
   * 将List<CompletableFuture<List<T>>> 转为 CompletableFuture<List<T>>
   * @Param filterFunction 自定义过滤策略
   */
  public static <T> CompletableFuture<List<T>> sequenceList(Collection<CompletableFuture<List<T>>> completableFutures,
                                                         Predicate<? super T> filterFunction) {
    
    
   return CompletableFuture.allOf(completableFutures.toArray(new CompletableFuture<?>[0]))
           .thenApply(v -> completableFutures.stream()
                   .flatMap( listFuture -> listFuture.join().stream().filter(filterFunction))
                   .collect(Collectors.toList())
           );
  }
/**
 * 将CompletableFuture<Map<K,V>>的list转为 CompletableFuture<Map<K,V>>。 多个map合并为一个map。 如果key冲突，采用新的value覆盖。
 */
  public static <K, V> CompletableFuture<Map<K, V>> sequenceMap(
       Collection<CompletableFuture<Map<K, V>>> completableFutures) {
    
    
   return CompletableFuture
           .allOf(completableFutures.toArray(new CompletableFuture<?>[0]))
           .thenApply(v -> completableFutures.stream().map(CompletableFuture::join)
                   .flatMap(map -> map.entrySet().stream())
                   .collect(Collectors.toMap(Entry::getKey, Entry::getValue, (a, b) -> b)));
  }}

Exception extraction tool class

  public class ExceptionUtils {
    
    
   /**
    * 提取真正的异常
    */
   public static Throwable extractRealException(Throwable throwable) {
    
    
       if (throwable instanceof CompletionException || throwable instanceof ExecutionException) {
    
    
           if (throwable.getCause() != null) {
    
    
               return throwable.getCause();
           }
       }
       return throwable;
   }
  }

print log

  @Slf4j
  public abstract class AbstractLogAction<R> {
    
    
  protected final String methodName;
  protected final Object[] args;
public AbstractLogAction(String methodName, Object... args) {
    
    
    this.methodName = methodName;
    this.args = args;
}
protected void logResult(R result, Throwable throwable) {
    
    
    if (throwable != null) {
    
    
        boolean isBusinessError = throwable instanceof TBase || (throwable.getCause() != null && throwable
                .getCause() instanceof TBase);
        if (isBusinessError) {
    
    
            logBusinessError(throwable);
        } else if (throwable instanceof DegradeException || throwable instanceof DegradeRuntimeException) {
    
    //这里为内部rpc框架抛出的异常，使用时可以酌情修改
            if (RhinoSwitch.getBoolean("isPrintDegradeLog", false)) {
    
    
                log.error("{} degrade exception, param:{} , error:{}", methodName, args, throwable);
            }
        } else {
    
    
            log.error("{} unknown error, param:{} , error:{}", methodName, args, ExceptionUtils.extractRealException(throwable));
        }
    } else {
    
    
        if (isLogResult()) {
    
    
            log.info("{} param:{} , result:{}", methodName, args, result);
        } else {
    
    
            log.info("{} param:{}", methodName, args);
        }
    }
}
private void logBusinessError(Throwable throwable) {
    
    
    log.error("{} business error, param:{} , error:{}", methodName, args, throwable.toString(), ExceptionUtils.extractRealException(throwable));
}
private boolean isLogResult() {
    
    
      //这里是动态配置开关，用于动态控制日志打印，开源动态配置中心可以使用nacos、apollo等，如果项目没有使用配置中心则可以删除
    return RhinoSwitch.getBoolean(methodName + "_isLogResult", false);
}}

Log processing implementation class

/**
 * 发生异常时，根据是否为业务异常打印日志。
 * 跟CompletableFuture.whenComplete配合使用，不改变completableFuture的结果（正常OR异常）
 */
@Slf4j
public class LogErrorAction<R> extends AbstractLogAction<R> implements BiConsumer<R, Throwable> {
    
    
public LogErrorAction(String methodName, Object... args) {
    
    
    super(methodName, args);
}
@Override
public void accept(R result, Throwable throwable) {
    
    
    logResult(result, throwable);
}
}

print log mode

completableFuture
.whenComplete(
  new LogErrorAction<>("orderService.getOrder", params));

Exception returns default value

/**
 * 当发生异常时返回自定义的值
 */
public class DefaultValueHandle<R> extends AbstractLogAction<R> implements BiFunction<R, Throwable, R> {
    
    
    private final R defaultValue;
/**
 * 当返回值为空的时候是否替换为默认值
 */
private final boolean isNullToDefault;
/**
 * @param methodName      方法名称
 * @param defaultValue 当异常发生时自定义返回的默认值
 * @param args            方法入参
 */
  public DefaultValueHandle(String methodName, R defaultValue, Object... args) {
    
    
   super(methodName, args);
   this.defaultValue = defaultValue;
   this.isNullToDefault = false;
  }
/**
 * @param isNullToDefault
 * @param defaultValue 当异常发生时自定义返回的默认值
 * @param methodName      方法名称
 * @param args            方法入参
 */
  public DefaultValueHandle(boolean isNullToDefault, R defaultValue, String methodName, Object... args) {
    
    
   super(methodName, args);
   this.defaultValue = defaultValue;
   this.isNullToDefault = isNullToDefault;
  }
@Override
public R apply(R result, Throwable throwable) {
    
    
    logResult(result, throwable);
    if (throwable != null) {
    
    
        return defaultValue;
    }
    if (result == null && isNullToDefault) {
    
    
        return defaultValue;
    }
    return result;
}
public static <R> DefaultValueHandle.DefaultValueHandleBuilder<R> builder() {
    
    
    return new DefaultValueHandle.DefaultValueHandleBuilder<>();
}
public static class DefaultValueHandleBuilder<R> {
    
    
    private boolean isNullToDefault;
    private R defaultValue;
    private String methodName;
    private Object[] args;
    DefaultValueHandleBuilder() {
    
    
    }
    public DefaultValueHandle.DefaultValueHandleBuilder<R> isNullToDefault(final boolean isNullToDefault) {
    
    
        this.isNullToDefault = isNullToDefault;
        return this;
    }
    public DefaultValueHandle.DefaultValueHandleBuilder<R> defaultValue(final R defaultValue) {
    
    
        this.defaultValue = defaultValue;
        return this;
    }
    public DefaultValueHandle.DefaultValueHandleBuilder<R> methodName(final String methodName) {
    
    
        this.methodName = methodName;
        return this;
    }
    public DefaultValueHandle.DefaultValueHandleBuilder<R> args(final Object... args) {
    
    
        this.args = args;
        return this;
    }
    public DefaultValueHandle<R> build() {
    
    
        return new DefaultValueHandle<R>(this.isNullToDefault, this.defaultValue, this.methodName, this.args);
    }
    public String toString() {
    
    
        return "DefaultValueHandle.DefaultValueHandleBuilder(isNullToDefault=" + this.isNullToDefault + ", defaultValue=" + this.defaultValue + ", methodName=" + this.methodName + ", args=" + Arrays.deepToString(this.args) + ")";
    }
}

Application example of default return value

completableFuture.handle(new DefaultValueHandle<>("orderService.getOrder", Collections.emptyMap(), params));