在呼叫外部介面時,為提升介面訪問的安全性,會對呼叫的介面帶上ticket標識,從而實現“船票”方式訪問,但是ticket在獲取時有些限制條件,如何在符合條件內實現一個ticket訪問的工具方法呢?這裡採取的實現為二級快取的方式
一、前置條件:有效期:3小時
保護期:考慮新舊切換10分鐘之內新舊都可以訪問
頻率:每分鐘1000
二、實現方案:實現形式:二級快取,使用spring官方提供的cache管理器進行自定義實現
使用方式:引入Spring Boot Starter、使用cache相關注解實現
快取設定:使用兩級快取設定
本地快取(有效期3小時),使用caffeine來實現,也可以使用其他本地快取分散式快取(有效期3小時10分鐘,避免臨界時間3小時時ticker不可用),使用redis實現,也可以使用其他分散式快取重新整理機制:
分散式快取重新整理
同步重新整理:未獲取到快取(本地和分散式)時觸發,呼叫統一重新整理方法,使用本地鎖控制多執行緒併發,使用分散式鎖防止多例項競爭重新整理
非同步重新整理:job每間隔10分鐘執行檢測(通常小於保護期10分鐘),判斷當前ticket分散式快取有效期是否小於10分鐘,是就重新整理,不是就忽略,臨界情況可以在10分鐘新舊交替期間完成重新整理任務,間隔設定過大會導致分散式快取已失效但是job還未重新整理情況(實際上會呼叫同步重新整理兜底-此時產生阻塞)
本地快取重新整理
釋出-訂閱模式(也稱之觀察者模式)傳送分佈快取變更的事件,本地快取監聽這個事件進行清除操作
三、Spring Boot Starter編寫//快取管理器類public class SecondCacheManager implements CacheManager { private ConcurrentMap<String, Cache> cacheMap = new ConcurrentHashMap<String, Cache>(); private SecondCacheProperties secondCacheProperties; private RedisTemplate<Object, Object> redisTemplate; private RedisLockRegistry redisLockRegistry; public SecondCacheManager(SecondCacheProperties secondCacheProperties, RedisTemplate<Object, Object> redisTemplate, RedisLockRegistry redisLockRegistry) { this.secondCacheProperties = secondCacheProperties; this.redisTemplate = redisTemplate; this.redisLockRegistry = redisLockRegistry; } @Override public Cache getCache(String name) { Cache cache = cacheMap.get(name); if (cache != null) { return cache; } if (!secondCacheProperties.isDynamic() && !cacheMap.keySet().contains(name)) { return cache; } cache = new SecondCache(name, redisTemplate, caffeineCache(), secondCacheProperties, redisLockRegistry); Cache oldCache = cacheMap.putIfAbsent(name, cache); return oldCache == null ? cache : oldCache; } @Override public Collection<String> getCacheNames() { return this.cacheMap.keySet(); } private com.github.benmanes.caffeine.cache.Cache<Object, Object> caffeineCache() { Caffeine<Object, Object> cacheBuilder = Caffeine.newBuilder(); if (secondCacheProperties.getCaffeine().getExpireAfterAccess() > 0) { cacheBuilder.expireAfterAccess(secondCacheProperties.getCaffeine().getExpireAfterAccess(), TimeUnit.MILLISECONDS); } if (secondCacheProperties.getCaffeine().getExpireAfterWrite() > 0) { cacheBuilder.expireAfterWrite(secondCacheProperties.getCaffeine().getExpireAfterWrite(), TimeUnit.MILLISECONDS); } if (secondCacheProperties.getCaffeine().getInitialCapacity() > 0) { cacheBuilder.initialCapacity(secondCacheProperties.getCaffeine().getInitialCapacity()); } if (secondCacheProperties.getCaffeine().getMaximumSize() > 0) { cacheBuilder.maximumSize(secondCacheProperties.getCaffeine().getMaximumSize()); } if (secondCacheProperties.getCaffeine().getRefreshAfterWrite() > 0) { cacheBuilder.refreshAfterWrite(secondCacheProperties.getCaffeine().getRefreshAfterWrite(), TimeUnit.MILLISECONDS); } return cacheBuilder.build(); } public void clearLocal(String cacheName, Object key) { Cache cache = cacheMap.get(cacheName); if (cache == null) { return; } //清除本地快取 SecondCache secondCache = (SecondCache) cache; secondCache.clearLocal(key); }}
//自定義二級快取public class SecondCache extends AbstractValueAdaptingCache { private String name; private RedisTemplate<Object, Object> redisTemplate; private Cache<Object, Object> caffeineCache; private SecondCacheProperties secondCacheProperties; private RedisLockRegistry redisLockRegistry; protected SecondCache(boolean allowNullValues) { super(allowNullValues); } public SecondCache(String name, RedisTemplate<Object, Object> redisTemplate, Cache<Object, Object> caffeineCache, SecondCacheProperties secondCacheProperties, RedisLockRegistry redisLockRegistry) { super(secondCacheProperties.isCacheNullValues()); this.name = name; this.redisTemplate = redisTemplate; this.caffeineCache = caffeineCache; this.secondCacheProperties = secondCacheProperties; this.redisLockRegistry = redisLockRegistry; } @Override public String getName() { return this.name; } @Override public Object getNativeCache() { return this; } /** * sync=false走以下方法,預設規則 * * @param key * @return */ @Override public ValueWrapper get(Object key) { Object value = lookup(key); return toValueWrapper(value); } /** * sync=true走以下方法 * * @param key * @param valueLoader * @param <T> * @return */ @Override public <T> T get(Object key, Callable<T> valueLoader) { Object value = lookup(key); if (value != null) { return (T) value; } //重新整理快取 return refresh(key, valueLoader); } @SneakyThrows private <T> T refresh(Object key, Callable<T> valueLoader) { Object value = null; Lock lock = null; boolean flag = false; try { //獲取本地鎖和分散式鎖 lock = redisLockRegistry.obtain(key); //重試時間內,每100ms不斷重試 flag = lock.tryLock(secondCacheProperties.getRetryTime(), TimeUnit.MILLISECONDS); //再次獲取,或許被其他機器設定了新值 value = lookup(key); if (value != null) { return (T) value; } if (!flag) { return null; } //未設定呼叫業務方法獲取value value = valueLoader.call(); Object storeValue = toStoreValue(value); //更新快取 put(key, storeValue); return (T) value; } catch (Exception e) { log.error("error",e); throw e; } finally { if (Objects.nonNull(lock) && flag) { lock.unlock(); } } } @Override public void put(Object key, Object value) { if (!super.isAllowNullValues() && value == null) { this.evict(key); return; } long expire = getExpire(); if (expire > 0) { redisTemplate.opsForValue().set(getKey(key), toStoreValue(value), expire, TimeUnit.MILLISECONDS); } else { redisTemplate.opsForValue().set(getKey(key), toStoreValue(value)); } //通知各端刪除本地快取 push(new SecondCacheMessage(this.name, key)); //重新整理本地快取 caffeineCache.put(key, value); } @Override public void evict(Object key) { // 先清除redis中快取資料,然後清除caffeine中的快取,避免短時間內如果先清除caffeine快取後其他請求會再從redis里加載到caffeine中 redisTemplate.delete(getKey(key)); push(new SecondCacheMessage(this.name, key)); caffeineCache.invalidate(key); } @Override public void clear() { // 先清除redis中快取資料,然後清除caffeine中的快取,避免短時間內如果先清除caffeine快取後其他請求會再從redis里加載到caffeine中 Set<Object> keys = redisTemplate.keys(this.name.concat(":")); for (Object key : keys) { redisTemplate.delete(key); } push(new SecondCacheMessage(this.name, null)); caffeineCache.invalidateAll(); } @Override protected Object lookup(Object key) { Object cacheKey = getKey(key); Object value = caffeineCache.getIfPresent(key); if (value != null) { log.debug("get cache from caffeine, the key is : {}", cacheKey); return value; } value = redisTemplate.opsForValue().get(cacheKey); if (value != null) { log.debug("get cache from redis and put in caffeine, the key is : {}", cacheKey); caffeineCache.put(key, value); } return value; } private Object getKey(Object key) { return this.name.concat(":").concat(StringUtils.isEmpty(secondCacheProperties.getCachePrefix()) ? key.toString() : secondCacheProperties.getCachePrefix().concat(":").concat(key.toString())); } private long getExpire() { long expire = secondCacheProperties.getRedis().getDefaultExpiration(); Long cacheNameExpire = secondCacheProperties.getRedis().getExpires().get(this.name); return cacheNameExpire == null ? expire : cacheNameExpire.longValue(); } /** * @param message * @description 快取變更時通知其他節點清理本地快取 */ private void push(SecondCacheMessage message) { redisTemplate.convertAndSend(secondCacheProperties.getRedis().getTopic(), message); } /** * @param key * @description 清理本地快取 */ public void clearLocal(Object key) { if (key == null) { caffeineCache.invalidateAll(); } else { caffeineCache.invalidate(key); } }}
//快取訊息bean@Data@AllArgsConstructor@NoArgsConstructorpublic class SecondCacheMessage implements Serializable { private String cacheName; private Object key;}
//監聽redis訊息,清除本地快取public class SecondCacheMessageListener implements MessageListener { private RedisTemplate<Object, Object> redisTemplate; private SecondCacheManager secondCacheManager; public SecondCacheMessageListener(RedisTemplate<Object, Object> redisTemplate, SecondCacheManager secondCacheManager) { super(); this.redisTemplate = redisTemplate; this.secondCacheManager = secondCacheManager; } @Override public void onMessage(Message message, byte[] pattern) { SecondCacheMessage secondCacheMessage = (SecondCacheMessage) redisTemplate.getValueSerializer().deserialize(message.getBody()); secondCacheManager.clearLocal(secondCacheMessage.getCacheName(), secondCacheMessage.getKey()); }}
//自動裝配類@Configuration@AutoConfigureAfter(RedisAutoConfiguration.class)@EnableConfigurationProperties(SecondCacheProperties.class)@ConditionalOnClass({RedisTemplate.class})public class SecondCacheAutoConfiguration { @Bean(name = "secondCacheManager") @ConditionalOnMissingBean public SecondCacheManager cacheManager(RedisTemplate<Object, Object> redisTemplate, RedisLockRegistry redisLockRegistry, SecondCacheProperties secondCacheProperties) { return new SecondCacheManager(secondCacheProperties, redisTemplate, redisLockRegistry); } @Bean @ConditionalOnMissingBean public RedisMessageListenerContainer redisMessageListenerContainer(RedisTemplate<Object, Object> redisTemplate, SecondCacheManager secondCacheManager, SecondCacheProperties secondCacheProperties) { RedisMessageListenerContainer redisMessageListenerContainer = new RedisMessageListenerContainer(); redisMessageListenerContainer.setConnectionFactory(redisTemplate.getConnectionFactory()); SecondCacheMessageListener secondCacheMessageListener = new SecondCacheMessageListener(redisTemplate, secondCacheManager); redisMessageListenerContainer.addMessageListener(secondCacheMessageListener, new ChannelTopic(secondCacheProperties.getRedis().getTopic())); return redisMessageListenerContainer; } @Bean @ConditionalOnMissingBean public RedisLockRegistry redisLockRegistry(RedisConnectionFactory connectionFactory) { return new RedisLockRegistry(connectionFactory, "project-name", 60000L); }}
//註解方式使用 //註解的作用請參考spring cache的相關資料@Cacheable(value = "customCache", key = "'getTicket'", sync = true, cacheManager = "secondCacheManager") public String getTicket() { //獲取ticket邏輯}@CachePut(key = "customCache", value = "getTicket", cacheManager = "secondCacheManager")public Boolean update(UserVO userVO) { //新增邏輯} @CacheEvict(key = "customCache", value = "getTicket", cacheManager = "secondCacheManager")public void delete(long id) { //刪除邏輯}
四、總結:
1、兩級快取設定(本地快取-Caffeine、分散式快取-redis)
2、Spring Cache註解無縫整合 (省去自定義註解+AOP攔截)
3、使用簡單,引入Spring Boot的Starter元件可實現自動裝配,注入即可使用
4、分散式鎖(多機器之間)+本地鎖(多執行緒之間)聯合控制併發讀取資料
5、同步重新整理+非同步重新整理保證資料實時準確有效
6、釋出-訂閱模式重新整理快取(本地快取)
7、快取穿透問題防禦