In this post, I describe how to use MemoryCache and track lifetime of entities in .NET applications.
Caching is one of famous ways to improve a performance of an application and to decrease a load on a data provider. In .NET there are packages Microsoft.Extensions.Caching to use different storage : Redis, SqlServer, Cosmos, Memory. Last one is the simplest method. It just saves a data in RAM. You can use it just installing Microsoft.Extensions.Caching.Memory from NuGet.
To keep data, it uses inner
ConcurrentDictionary. So operations set/get are thread-safe. An interaction with MemoryCache is described in
IMemoryCache. This interface has several methods to set, remove and get a value. The usual algorithm to work with a cache is:
1. Try to get a value from the cache.
2. If you didn’t get the value from the cache, request to a data source and save received value in the cache.
We should to be careful in using memory and prevent from uncontrolled allocation. Therefore, in the above code we limit cache size, lifetime of entities by
The one important part of caching is an update of data in a cache store. In the above code, we check existence of entity during receiving a value. If it doesn’t exist, we get value from data provider and then save it in the cache. Also for this purpose
IMemoryCache extension has method
GetOrCreate<TItem>(Object key, Func<ICacheEntry,TItem> factory).
I should mention work expiration mechanism in
MemoryCache. When we try to get a value from a cache. The last one receives a value from an inner dictionary and check if it expired. If yes, the entity is marked as expired and then removed. If no, then the cache just returns the value. In both cases, the cache starts an invalidation process for all values in the inner dictionary in a separate thread. The period of this checking you can set up in
ExpirationScanFrequency filed of
MemoryCacheOptions (by default, 1 minute). Also, MemoryCache has compacting logic. It removes entities in the following order:
1. All expired items.
2. Items by priority. Lowest priority items are removed first.
3. Least recently used objects.
4. Items with the earliest absolute expiration.
5. Items with the earliest sliding expiration.
Pinned items with priority NeverRemove are never removed. Due to these reasons, we have to track values in a cache and update it. The simplest way is checking value by
TryGetValue method during receiving it from a cache.
Issue: updating value in MemoryCache
We can see in the above code, it is possible to call
GetOrUpdate from multiple threads and a cache value will be updated multiple times in these threads — race condition. There are different ways to avoid it:
Using synchronization primitives. For example, you can use
Also, we can use
Interlocked methods like here. There is a lock per key approach. Anyway, this get or update logic breaks a single responsibility principle. You get value and sometimes set new value in a single method.
Lazy<T> approach. Put in
Lazy<T> instance instead of
T in a cache. This way is described in StackOverflow. There is implementation of LazyCache also. The lazy approach is better than the previous. It allows keeping the single responsibility principle and use an instance from cache when it is actually necessary.
3. Using Callbacks. Official documentation mentions eviction callbacks. Let’s see what it is. Each entity in
MemoryCache has callbacks collection of
PostEvictionCallbackRegistration type, that are invoked after eviction value from the cache in the separate thread. You can add your delegate by
RegisterPostEvictionCallback method in
MemoryCacheEntryOptions argument of
Eviction reason sets in
EvictionReason.Removed after removing and
EvictionReason.Expiredafter checking lifetime. When cache entity was updated, eviction reason will be
EvictionReason.Replaced In case removing due to capacity overflow, it is
EvictionReason.CapacityA simple example:
It allows starting background update evicted values and split get/set logic — single responsibility principle. But you need to remember about race condition.
CacheEntity allows tracking changes by using
IChangeToken. I omit this topic, since it has the similar approach and I don’t pretend on comprehensive tutorial (Definitely, it would be a long read). You can read here about using change token.
It this post, I touched some important issues of using memory cache:
- You have to limit size and entities lifetime. It allows reducing memory usage.
- Cache invalidation is triggered during an attempt to get a value. It uses background thread to check lifetime and clean expired entities in a cache.
- MemoryCache is thread-safe, but doesn’t prevent race condition for
Set method. So I described several approaches to solve this problem. Each has pros and cons, choose the best for your case. Also, I saw an interesting async approach based on
TaskCompletionsSource. Learn more you can get from the list below.
P.S. Read a source code. Thanks to Open Source policy, it is a good way to learn more than from documentation.