background
Many students should encounter complex data interfaces involving the assembly of N tables, especially the XX list on the home page (such as the merchant list of Meituan Waimai, the list of Taobao order details, etc.). If all the codes are executed according to the usual linear
process Logic, then the response time of the interface can be said to explode directly (every few seconds), the product will not kill you, and the user will also spray you. For this reason,
I will share the optimization ideas and results with a practical chestnut.
business description
This is the schedule list on the home page. The list needs to display the list data of games and related games in the last 5 days. The
data is as follows:
This list interface needs to be composed of two major business domain data:
- (Competition Domain) Events, schedules, results, matches against associated teams, teams
- (Game Domain) Schedule-related games, game content (such as how many levels are included)
There are 7 tables in total.
Business flowchart
business analysis
Although the business description is only one sentence, the implementation is very complicated.
There are matches in the last 5 days and the list data of the associated games, disassembled (without using join):
- Query the dates with matches and cache them.
- Query the dates with games and cache them.
- According to the start time of the query condition, loop until the 5 days with intersection are found.
- Query process 3 to calculate the actual schedule and the time range of the game as a condition, query the schedule
- Assemble the schedule by day, construct a 2-dimensional array, and prepare public data
- Traverse the schedule list by day
- Go through the schedule list
- Query and assemble the data of 7 tables to generate a single detailed data
- output match list
Process 3 is the difficulty of the requirement. It is impossible to know in advance how much data needs to be fetched to get the desired result.
Basic implementation
According to the above process, do not do other optimization, and realize it with a linear process. In the case of preparing 1000 matches and 10000 games, the following results are obtained: (laughing and crying) A single request exceeds 3 seconds, and the amount of data is produced
. It will blow up for you in minutes.
Of course, the amount of data is actually more than normal, but I’m afraid of abnormal times.
optimization
The source code contains business-sensitive information, so it will not be pasted. Refer to the test chestnut below for the form.
Optimization ideas
- Starting from the fetch logic, improve the efficiency of data retrieval and reduce the return of useless data
- Optimize algorithms to reduce database access
- Cache as much data as possible that changes little
- Time-consuming tasks are executed in a thread pool
- When there is a cache, it may be more efficient to query data in a loop rather than in batches
parallelStream concurrent stream optimization (risky)
In fact, it is simply traversing the schedule day with concurrent streams.
The risk point is that the parallel streams that come with JDK are common thread pools and are not suitable for IO tasks. This will cause the execution speed of other businesses that use parallel streams to degrade to a single thread. .
Single IO thread pool optimization
Here, a special IO thread pool is constructed to process the data of the schedule day, and the schedule list of a single day is used as a task for concurrent processing.
Master-slave IO thread pool optimization
Here, two dedicated IO thread pools (master and slave) are constructed to process the data of the schedule day and the schedule list data of the corresponding day respectively. The schedule list of a single day is used as a task of the main task pool, and the schedule list of the day is used as a task of the main task pool. As a task in the subtask pool, it is processed concurrently.
Ultimate move
If the data changes infrequently, cache the game list interface directly, and then update the game list asynchronously through the schedule and game change events. In this way, the
response time of the interface can be directly reduced to less than 100ms, and another advantage is to avoid the response time jitter caused by data updates. This I won’t test it, because this solution is the outermost optimization. The ideal situation is the time-consuming of cache query, which is about 50ms by visual inspection.
If the inner layer implementation is not good enough, when the schedule and games change frequently, it takes 3 seconds to construct a single list This will still cause the service to go down.
Of course, here we can also check the time and change events at the same time. Only when the two conditions are met can the list be updated asynchronously.
The source code of the optimization tool
import lombok.extern.slf4j.Slf4j;
import org.slf4j.MDC;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
import javax.annotation.Nonnull;
import java.util.*;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Future;
/**
* IO密集任务工具
* Warning: {@link #asyncTaskQueue}, {@link #getAsyncTaskQueueResult} 依赖 ThreadLocal, 注意多线程下的问题
* Warning: 警惕嵌套使用该工具,可能会导致死锁. 如果真需要嵌套,请使用master和worker模式.
*
* @Description
* @Author LionZhou
* @Date 2023/2/21
* @Version 1.0
*/
@Slf4j
public class IOUtil {
static final ThreadPoolTaskExecutor masterThreadPoolExecutor;
static final ThreadLocal<List<Future<Object>>> masterTaskQueue;
static final ThreadPoolTaskExecutor workerThreadPoolExecutor;
static final ThreadLocal<List<Future<Object>>> workerTaskQueue;
public static <R> Future<R> async(Callable<R> func) {
return workerThreadPoolExecutor.submit(func);
}
public static <R> List<Future<R>> async(Collection<Callable<R>> funcs) {
ArrayList<Future<R>> list = new ArrayList<>(funcs.size());
for (Callable<R> func : funcs) {
list.add(workerThreadPoolExecutor.submit(func));
}
return list;
}
public static void asyncTaskQueue(Callable<Object> func) {
asyncTaskQueue(func, false);
}
/**
* 提交异步任务
*
* @param func 待执行任务
* @param isMaster 是否使用master
*/
public static void asyncTaskQueue(Callable<Object> func, boolean isMaster) {
ThreadPoolTaskExecutor threadPoolTaskExecutor = isMaster ? masterThreadPoolExecutor : workerThreadPoolExecutor;
ThreadLocal<List<Future<Object>>> taskQueue = isMaster ? masterTaskQueue : workerTaskQueue;
Future<Object> future = threadPoolTaskExecutor.submit(func);
List<Future<Object>> futures = taskQueue.get();
if (Objects.isNull(futures)) {
futures = new ArrayList<>(32);
taskQueue.set(futures);
}
futures.add(future);
}
public static <R> List<R> getAsyncTaskQueueResult(Class<R> clz) {
return getAsyncTaskQueueResult(clz, false);
}
/**
* 获取异步任务结果
*
* @param clz 返回结果需要转换的泛型类型
* @param isMaster 是否使用master
*/
public static <R> List<R> getAsyncTaskQueueResult(Class<R> clz, boolean isMaster) {
ThreadLocal<List<Future<Object>>> taskQueue = isMaster ? masterTaskQueue : workerTaskQueue;
List<Future<Object>> futures = taskQueue.get();
if (Objects.isNull(futures)) {
return Collections.emptyList();
}
ArrayList<R> rs = new ArrayList<>(futures.size());
try {
for (Future<Object> future : futures) {
rs.add((R) future.get());
}
} catch (InterruptedException | ExecutionException e) {
log.error(e.getMessage(), e);
throw new RuntimeException(e);
} finally {
taskQueue.remove();
}
return rs;
}
public static <R> Optional<R> syncWaitDone(Callable<R> func) {
try {
return Optional.of(async(func).get());
} catch (InterruptedException | ExecutionException e) {
log.error(e.getMessage(), e);
}
return Optional.empty();
}
public static <R> List<Optional<R>> syncWaitDone(Collection<Callable<R>> funcs) {
List<Future<R>> list = async(funcs);
ArrayList<Optional<R>> rsList = new ArrayList<>(funcs.size());
for (Future<R> future : list) {
try {
rsList.add(Optional.of(future.get()));
} catch (InterruptedException | ExecutionException e) {
log.error(e.getMessage(), e);
rsList.add(Optional.empty());
}
}
return rsList;
}
/**
* 自定义线程池,并注入MDC上下文,方便链路追踪
*
* @param coreSize 核心线程数
* @param queueSize 队列长度
* @return 线程池
*/
public static ThreadPoolTaskExecutor taskExecutor(int coreSize, int queueSize) {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor() {
@Override
public void execute(@Nonnull Runnable task) {
Map<String, String> copyOfContextMap = MDC.getCopyOfContextMap();
super.execute(() -> {
MDC.setContextMap(Objects.nonNull(copyOfContextMap) ? copyOfContextMap : Collections.emptyMap());
try {
task.run();
} catch (Exception e) {
log.error(e.getMessage());
} finally {
MDC.clear();
}
});
}
@Override
public Future<?> submit(@Nonnull Runnable task) {
Map<String, String> copyOfContextMap = MDC.getCopyOfContextMap();
return super.submit(() -> {
MDC.setContextMap(Objects.nonNull(copyOfContextMap) ? copyOfContextMap : Collections.emptyMap());
try {
task.run();
} catch (Exception e) {
log.error(e.getMessage());
} finally {
MDC.clear();
}
});
}
};
executor.setCorePoolSize(coreSize);
// 因为测试,暂时设置为core相同
executor.setMaxPoolSize(coreSize);
executor.setKeepAliveSeconds(120);
executor.setQueueCapacity(queueSize);
executor.initialize();
return executor;
}
static {
masterThreadPoolExecutor = taskExecutor(1, 50);
workerThreadPoolExecutor = taskExecutor(10, 300);
masterTaskQueue = new ThreadLocal<>();
workerTaskQueue = new ThreadLocal<>();
}
public static void main(String[] args) {
final Callable<Object> callB = () -> {
try {
Thread thread = Thread.currentThread();
log.info("worker {},{}", thread.getThreadGroup().getName(), thread.getId());
Thread.sleep(500);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
return true;
};
Callable<Object> runnable = () -> {
try {
Thread thread = Thread.currentThread();
for (int i = 0; i < 10; i++) {
// 提交到worker线程池
asyncTaskQueue(callB);
}
List<Boolean> async = getAsyncTaskQueueResult(Boolean.class);
log.info("master {},{}", thread.getThreadGroup().getName(), thread.getId());
} catch (Exception e) {
throw new RuntimeException(e);
}
return true;
};
for (int i = 0; i < 8; i++) {
// 提交到master线程池调用
asyncTaskQueue(runnable, true);
}
long st = System.currentTimeMillis();
List<Boolean> async = getAsyncTaskQueueResult(Boolean.class, true);
long et = System.currentTimeMillis();
log.info("{} ms", et - st);
masterThreadPoolExecutor.shutdown();
workerThreadPoolExecutor.shutdown();
}
}
Test Results
Only worker mode, master 0, worker 3
private static void onlyWorker() {
Callable<Object> runnable = () -> {
try {
for (int i = 0; i < 10; i++) {
// 提交到worker线程池
asyncTaskQueue(callB);
}
List<Boolean> async = getAsyncTaskQueueResult(Boolean.class);
Thread thread = Thread.currentThread();
log.info("master {},{}", thread.getThreadGroup().getName(), thread.getId());
} catch (Exception e) {
throw new RuntimeException(e);
}
return true;
};
for (int i = 0; i < 8; i++) {
try {
runnable.call();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
public static void main(String[] args) {
long st = System.currentTimeMillis();
onlyWorker();
long et = System.currentTimeMillis();
log.info("{} ms", et - st);
masterThreadPoolExecutor.shutdown();
workerThreadPoolExecutor.shutdown();
}
Master-slave mode, master 1, worker 2
private static void mixed() {
Callable<Object> runnable = () -> {
try {
for (int i = 0; i < 10; i++) {
// 提交到worker线程池
asyncTaskQueue(callB);
}
List<Boolean> async = getAsyncTaskQueueResult(Boolean.class);
Thread thread = Thread.currentThread();
log.info("master {},{}", thread.getThreadGroup().getName(), thread.getId());
} catch (Exception e) {
throw new RuntimeException(e);
}
return true;
};
for (int i = 0; i < 8; i++) {
// 提交到master线程池调用
asyncTaskQueue(runnable, true);
}
List<Boolean> async = getAsyncTaskQueueResult(Boolean.class, true);
}
public static void main(String[] args) {
long st = System.currentTimeMillis();
mixed();
long et = System.currentTimeMillis();
log.info("{} ms", et - st);
masterThreadPoolExecutor.shutdown();
workerThreadPoolExecutor.shutdown();
}
master only, master 3, worker 0
private static void onlyMaster() {
Callable<Object> runnable = () -> {
try {
for (int i = 0; i < 10; i++) {
// 提交到worker线程池
callB.call();
}
Thread thread = Thread.currentThread();
log.info("master {},{}", thread.getThreadGroup().getName(), thread.getId());
} catch (Exception e) {
throw new RuntimeException(e);
}
return true;
};
for (int i = 0; i < 8; i++) {
// 提交到master线程池调用
asyncTaskQueue(runnable, true);
}
List<Boolean> async = getAsyncTaskQueueResult(Boolean.class, true);
}
public static void main(String[] args) {
long st = System.currentTimeMillis();
onlyMaster();
long et = System.currentTimeMillis();
log.info("{} ms", et - st);
masterThreadPoolExecutor.shutdown();
workerThreadPoolExecutor.shutdown();
}
in conclusion
- For IO-intensive business, there is a huge gap between using thread pool and not using response time.
- The master-slave mode is not necessarily better than the same number of worker modes, especially when the number of threads is small and the master task is not time-consuming.
Proper use of the thread pool can greatly improve the response of the service.
Thinking: Why do I have to divide the master and worker to handle two kinds of tasks? Can't they be put together?