Several Realization Methods of Java Local High Performance Cache

Java caching technology can be divided into remote caching and local caching. Common solutions for remote caching include the famous redis and memcache, while representative technologies for local caching mainly include HashMap, Guava Cache, Caffeine, and Encahche. This blog post only covers local caching and focuses on high-performance local caching.

This blog post will first introduce common local caching technologies, and have a general understanding of local caching; secondly, introduce Cache, which claims to have the best performance in local caching, and discuss how good it is? How do you do it so well? Finally, through several practical examples, high-performance local caching is applied in daily work.

1. Java local cache technology introduction

1.1 Use the List collection contains method to loop through (ordered) 1.1 HashMap

Through the underlying method of Map, the objects that need to be cached are directly placed in memory.

• Advantages: simple and rude, no need to introduce third-party packages, more suitable for some relatively simple scenarios.

• Disadvantages: There is no cache elimination strategy, and the cost of customized development is high.

public class LRUCache extends LinkedHashMap {

    /**
     * 可重入读写锁，保证并发读写安全性
     */
    private ReentrantReadWriteLock readWriteLock = new ReentrantReadWriteLock();
    private Lock readLock = readWriteLock.readLock();
    private Lock writeLock = readWriteLock.writeLock();

    /**
     * 缓存大小限制
     */
    private int maxSize;

    public LRUCache(int maxSize) {
        super(maxSize + 1, 1.0f, true);
        this.maxSize = maxSize;
    }

    @Override
    public Object get(Object key) {
        readLock.lock();
        try {
            return super.get(key);
        } finally {
            readLock.unlock();
        }
    }

    @Override
    public Object put(Object key, Object value) {
        writeLock.lock();
        try {
            return super.put(key, value);
        } finally {
            writeLock.unlock();
        }
    }

    @Override
    protected boolean removeEldestEntry(Map.Entry eldest) {
        return this.size() > maxSize;
    }
}

1.2 Guava Cache

Guava Cache is a cache technology based on the LRU replacement algorithm that is open sourced by Google. However, Guava Cache was replaced by Caffeine, which will be introduced below, so it is not necessary to write sample codes. Interested readers can visit the Guava Cache homepage.

• Advantages: Support maximum capacity limit, two expired deletion strategies (insert time and access time), support simple statistical functions.

• Disadvantages: Both springboot2 and spring5 have given up support for Guava Cache.

1.3 Caffeine

Caffeine uses W-TinyLFU (combining the advantages of LUR and LFU) open source caching technology. The cache performance is close to the theoretical optimum, which is an enhanced version of Guava Cache.

public class CaffeineCacheTest {

    public static void main(String[] args) throws Exception {
        //创建guava cache
        Cache<String, String> loadingCache = Caffeine.newBuilder()
                //cache的初始容量
                .initialCapacity(5)
                //cache最大缓存数
                .maximumSize(10)
                //设置写缓存后n秒钟过期
                .expireAfterWrite(17, TimeUnit.SECONDS)
                //设置读写缓存后n秒钟过期,实际很少用到,类似于expireAfterWrite
                //.expireAfterAccess(17, TimeUnit.SECONDS)
                .build();
        String key = "key";
        // 往缓存写数据
        loadingCache.put(key, "v");

        // 获取value的值，如果key不存在，获取value后再返回
        String value = loadingCache.get(key, CaffeineCacheTest::getValueFromDB);

        // 删除key
        loadingCache.invalidate(key);
    }

    private static String getValueFromDB(String key) {
        return "v";
    }
}

1.4 Encase

Ehcache is a pure java in-process caching framework, which is fast and lean. It is hibernate's default cacheprovider.

• Advantages: Supports multiple cache elimination algorithms, including LFU, LRU, and FIFO; cache supports in-heap cache, off-heap cache, and disk cache; supports multiple cluster solutions to solve data sharing problems.

• Disadvantages: performance is worse than Caffeine

public class EncacheTest {

    public static void main(String[] args) throws Exception {
        // 声明一个cacheBuilder
        CacheManager cacheManager = CacheManagerBuilder.newCacheManagerBuilder()
                .withCache("encacheInstance", CacheConfigurationBuilder
                        //声明一个容量为20的堆内缓存
                        .newCacheConfigurationBuilder(String.class,String.class, ResourcePoolsBuilder.heap(20)))
                .build(true);
        // 获取Cache实例
        Cache<String,String> myCache =  cacheManager.getCache("encacheInstance", String.class, String.class);
        // 写缓存
        myCache.put("key","v");
        // 读缓存
        String value = myCache.get("key");
        // 移除换粗
        cacheManager.removeCache("myCache");
        cacheManager.close();
    }
}

 In the introduction of Caffeine's official website, Caffeine has advantages compared with several other solutions in terms of performance and functions. Therefore, the following will mainly discuss the performance and implementation principles of Caffeine.

2. High-performance cache Caffeine

2.1 Cache type

2.1.1 Cache

Cache<Key, Graph> cache = Caffeine.newBuilder()
    .expireAfterWrite(10, TimeUnit.MINUTES)
    .maximumSize(10_000)
    .build();

// 查找一个缓存元素， 没有查找到的时候返回null
Graph graph = cache.getIfPresent(key);
// 查找缓存，如果缓存不存在则生成缓存元素,  如果无法生成则返回null
graph = cache.get(key, k -> createExpensiveGraph(key));
// 添加或者更新一个缓存元素
cache.put(key, graph);
// 移除一个缓存元素
cache.invalidate(key);

The Cache interface provides the ability to explicitly search to find, update, and remove cached elements. When the cached element cannot be generated or an exception is thrown during the generation process and the generation of the element fails, cache.get may return null .

2.1.2 Loading Cache

LoadingCache<Key, Graph> cache = Caffeine.newBuilder()
    .maximumSize(10_000)
    .expireAfterWrite(10, TimeUnit.MINUTES)
    .build(key -> createExpensiveGraph(key));

// 查找缓存，如果缓存不存在则生成缓存元素,  如果无法生成则返回null
Graph graph = cache.get(key);
// 批量查找缓存，如果缓存不存在则生成缓存元素
Map<Key, Graph> graphs = cache.getAll(keys);

A LoadingCache is a Cache implementation with the CacheLoader capability attached.
If the cache is good, the corresponding cache element will be generated through CacheLoader.load.

2.1.3 Loading Cache 2.1.3 Async Cache

AsyncCache<Key, Graph> cache = Caffeine.newBuilder()
    .expireAfterWrite(10, TimeUnit.MINUTES)
    .maximumSize(10_000)
    .buildAsync();

// 查找一个缓存元素， 没有查找到的时候返回null
CompletableFuture<Graph> graph = cache.getIfPresent(key);
// 查找缓存元素，如果不存在，则异步生成
graph = cache.get(key, k -> createExpensiveGraph(key));
// 添加或者更新一个缓存元素
cache.put(key, graph);
// 移除一个缓存元素
cache.synchronous().invalidate(key);

AsyncCache is the asynchronous form of Cache, which provides the ability of Executor to generate cache elements and return CompletableFuture. The default thread pool implementation is ForkJoinPool.commonPool() , of course you can also customize your thread pool selection by overriding and implementing the Caffeine.executor(Executor) method.

2.1.4 Async Loading Cache

AsyncLoadingCache<Key, Graph> cache = Caffeine.newBuilder()
    .maximumSize(10_000)
    .expireAfterWrite(10, TimeUnit.MINUTES)
    // 你可以选择: 去异步的封装一段同步操作来生成缓存元素
    .buildAsync(key -> createExpensiveGraph(key));
    // 你也可以选择: 构建一个异步缓存元素操作并返回一个future
    .buildAsync((key, executor) -> createExpensiveGraphAsync(key, executor));

// 查找缓存元素，如果其不存在，将会异步进行生成
CompletableFuture<Graph> graph = cache.get(key);
// 批量查找缓存元素，如果其不存在，将会异步进行生成
CompletableFuture<Map<Key, Graph>> graphs = cache.getAll(keys);

AsyncLoadingCache is the asynchronous form of LoadingCache, which provides the function of asynchronous load to generate cache elements.

2.2 Eviction strategy

• Based on capacity

// 基于缓存内的元素个数进行驱逐
LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .maximumSize(10_000)
    .build(key -> createExpensiveGraph(key));

// 基于缓存内元素权重进行驱逐
LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .maximumWeight(10_000)
    .weigher((Key key, Graph graph) -> graph.vertices().size())
    .build(key -> createExpensiveGraph(key));

• time based

// 基于固定的过期时间驱逐策略
LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .expireAfterAccess(5, TimeUnit.MINUTES)
    .build(key -> createExpensiveGraph(key));
LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .expireAfterWrite(10, TimeUnit.MINUTES)
    .build(key -> createExpensiveGraph(key));

// 基于不同的过期驱逐策略
LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .expireAfter(new Expiry<Key, Graph>() {
      public long expireAfterCreate(Key key, Graph graph, long currentTime) {
        // Use wall clock time, rather than nanotime, if from an external resource
        long seconds = graph.creationDate().plusHours(5)
            .minus(System.currentTimeMillis(), MILLIS)
            .toEpochSecond();
        return TimeUnit.SECONDS.toNanos(seconds);
      }
      public long expireAfterUpdate(Key key, Graph graph, 
          long currentTime, long currentDuration) {
        return currentDuration;
      }
      public long expireAfterRead(Key key, Graph graph,
          long currentTime, long currentDuration) {
        return currentDuration;
      }
    })
    .build(key -> createExpensiveGraph(key));

• reference based

// 当key和缓存元素都不再存在其他强引用的时候驱逐
LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .weakKeys()
    .weakValues()
    .build(key -> createExpensiveGraph(key));

// 当进行GC的时候进行驱逐
LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .softValues()
    .build(key -> createExpensiveGraph(key));

2.3 Refresh mechanism

LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .maximumSize(10_000)
    .refreshAfterWrite(1, TimeUnit.MINUTES)
    .build(key -> createExpensiveGraph(key));

The refresh strategy can only be used in LoadingCache. Unlike eviction, if the cache element is queried during refresh, its old value will still be returned, and the refreshed new value will not be returned until the refresh of the element is completed.

2.4 Statistics

Cache<Key, Graph> graphs = Caffeine.newBuilder()
    .maximumSize(10_000)
    .recordStats()
    .build();

Data collection can be turned on by using the Caffeine.recordStats() method. The Cache.stats() method will return a CacheStats object, which will contain some statistical indicators, such as:

• hitRate(): query cache hit rate

• evictionCount(): the number of caches evicted

• averageLoadPenalty(): The average time it takes for new values ​​to be loaded

With the RESTful Controller provided by SpringBoot, it is very convenient to query the usage of Cache.

3. Caffeine's actual combat in SpringBoot

According to the description of the Caffeine Github official website document, Caffeine is a high-performance caching library based on Java8. And in Spring5 (SpringBoot2.x), Guava was officially abandoned, and Caffeine with better performance was used as the default caching solution.

SpringBoot uses Caffeine in two ways:

• Method 1: Directly import Caffeine dependencies, and then use Caffeine functions to implement caching

• Method 2: Introduce Caffeine and Spring Cache dependencies, and use the SpringCache annotation method to implement caching.
  The following two usage methods are introduced respectively.

Method 1: Use Caffeine dependencies

First introduce maven-related dependencies:

<dependency>  
  <groupId>com.github.ben-manes.caffeine</groupId>  
    <artifactId>caffeine</artifactId>  
</dependency>

Second, set the configuration options for the cache

@Configuration
public class CacheConfig {

    @Bean
    public Cache<String, Object> caffeineCache() {
        return Caffeine.newBuilder()
                // 设置最后一次写入或访问后经过固定时间过期
                .expireAfterWrite(60, TimeUnit.SECONDS)
                // 初始的缓存空间大小
                .initialCapacity(100)
                // 缓存的最大条数
                .maximumSize(1000)
                .build();
    }

}

Finally, add caching to the service

@Slf4j
@Service
public class UserInfoServiceImpl {

    /**
     * 模拟数据库存储数据
     */
    private HashMap<Integer, UserInfo> userInfoMap = new HashMap<>();

    @Autowired
    Cache<String, Object> caffeineCache;

    public void addUserInfo(UserInfo userInfo) {
        userInfoMap.put(userInfo.getId(), userInfo);
        // 加入缓存
        caffeineCache.put(String.valueOf(userInfo.getId()),userInfo);
    }

    public UserInfo getByName(Integer id) {
        // 先从缓存读取
        caffeineCache.getIfPresent(id);
        UserInfo userInfo = (UserInfo) caffeineCache.asMap().get(String.valueOf(id));
        if (userInfo != null){
            return userInfo;
        }
        // 如果缓存中不存在，则从库中查找
        userInfo = userInfoMap.get(id);
        // 如果用户信息不为空，则加入缓存
        if (userInfo != null){
            caffeineCache.put(String.valueOf(userInfo.getId()),userInfo);
        }
        return userInfo;
    }

    public UserInfo updateUserInfo(UserInfo userInfo) {
        if (!userInfoMap.containsKey(userInfo.getId())) {
            return null;
        }
        // 取旧的值
        UserInfo oldUserInfo = userInfoMap.get(userInfo.getId());
        // 替换内容
        if (!StringUtils.isEmpty(oldUserInfo.getAge())) {
            oldUserInfo.setAge(userInfo.getAge());
        }
        if (!StringUtils.isEmpty(oldUserInfo.getName())) {
            oldUserInfo.setName(userInfo.getName());
        }
        if (!StringUtils.isEmpty(oldUserInfo.getSex())) {
            oldUserInfo.setSex(userInfo.getSex());
        }
        // 将新的对象存储，更新旧对象信息
        userInfoMap.put(oldUserInfo.getId(), oldUserInfo);
        // 替换缓存中的值
        caffeineCache.put(String.valueOf(oldUserInfo.getId()),oldUserInfo);
        return oldUserInfo;
    }

    @Override
    public void deleteById(Integer id) {
        userInfoMap.remove(id);
        // 从缓存中删除
        caffeineCache.asMap().remove(String.valueOf(id));
    }

}

Method 2: Use Spring Cache annotations

First introduce maven-related dependencies

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-cache</artifactId>
</dependency>
<dependency>
    <groupId>com.github.ben-manes.caffeine</groupId>
    <artifactId>caffeine</artifactId>
</dependency>

Second, configure the cache management class

@Configuration  
public class CacheConfig {  
  
    /**  
     * 配置缓存管理器  
     *  
     * @return 缓存管理器  
     */  
    @Bean("caffeineCacheManager")  
    public CacheManager cacheManager() {  
        CaffeineCacheManager cacheManager = new CaffeineCacheManager();  
        cacheManager.setCaffeine(Caffeine.newBuilder()  
                // 设置最后一次写入或访问后经过固定时间过期  
                .expireAfterAccess(60, TimeUnit.SECONDS)  
                // 初始的缓存空间大小  
                .initialCapacity(100)  
                // 缓存的最大条数  
                .maximumSize(1000));  
        return cacheManager;  
    }  
  
}

Finally, add caching to the service

@Slf4j
@Service
@CacheConfig(cacheNames = "caffeineCacheManager")
public class UserInfoServiceImpl {

    /**
     * 模拟数据库存储数据
     */
    private HashMap<Integer, UserInfo> userInfoMap = new HashMap<>();

    @CachePut(key = "#userInfo.id")
    public void addUserInfo(UserInfo userInfo) {
        userInfoMap.put(userInfo.getId(), userInfo);
    }

    @Cacheable(key = "#id")
    public UserInfo getByName(Integer id) {
        return userInfoMap.get(id);
    }

    @CachePut(key = "#userInfo.id")
    public UserInfo updateUserInfo(UserInfo userInfo) {
        if (!userInfoMap.containsKey(userInfo.getId())) {
            return null;
        }
        // 取旧的值
        UserInfo oldUserInfo = userInfoMap.get(userInfo.getId());
        // 替换内容
        if (!StringUtils.isEmpty(oldUserInfo.getAge())) {
            oldUserInfo.setAge(userInfo.getAge());
        }
        if (!StringUtils.isEmpty(oldUserInfo.getName())) {
            oldUserInfo.setName(userInfo.getName());
        }
        if (!StringUtils.isEmpty(oldUserInfo.getSex())) {
            oldUserInfo.setSex(userInfo.getSex());
        }
        // 将新的对象存储，更新旧对象信息
        userInfoMap.put(oldUserInfo.getId(), oldUserInfo);
        // 返回新对象信息
        return oldUserInfo;
    }

    @CacheEvict(key = "#id")
    public void deleteById(Integer id) {
        userInfoMap.remove(id);
    }

}