2

I have an API that currently does not use any caching. I do have one piece of Middleware that I am using that generates cache headers (Cache-Control, Expires, ETag, Last-Modified - using the https://github.com/KevinDockx/HttpCacheHeaders library). It does not store anything as it only generates the headers.

When an If-None-Match header is passed to the API request, the middleware checks the Etag value passed in vs the current generated value and if they match, sends a 304 not modified as the response (httpContext.Response.StatusCode = StatusCodes.Status304NotModified;)

I'm using a Redis cache and I'm not sure how to implement cache invalidation. I used the Microsoft.Extensions.Caching.Redis package in my project. I installed Redis locally and used it in my controller as below:

[AllowAnonymous]
[ProducesResponseType(200)]
[Produces("application/json", "application/xml")]
public async Task<IActionResult> GetEvents([FromQuery] ParameterModel model)
        {
            var cachedEvents = await _cache.GetStringAsync("events");
            IEnumerable<Event> events = null;

            if (!string.IsNullOrEmpty(cachedEvents))
            {
                events = JsonConvert.DeserializeObject<IEnumerable<Event>>(cachedEvents);
            }
            else
            {
                events = await _eventRepository.GetEventsAsync(model);
                string item = JsonConvert.SerializeObject(events, new JsonSerializerSettings()
                {
                    ReferenceLoopHandling = ReferenceLoopHandling.Ignore
                });
                await _cache.SetStringAsync("events", item);
            }

            var eventsToReturn = _mapper.Map<IEnumerable<EventViewDto>>(events);
            return Ok(eventsToReturn);
        }

Note that _cache here is using IDistributedCache. This works as the second time the request is hitting the cache. But when the Events I am fetching are modified, it does not take the modified values into account. It serves up the same value without doing any validation.

My middlware is setup as: Cache Header Middleware -> MVC. So the cache headers pipeline will first compare the Etag value sent by the client and either decides to forward the request to MVC or short circuits it with a 304 not modified response.

My plan was to add a piece of middleware prior to the cache header one (i.e. My Middleware -> Cache Header Middleware -> MVC) and wait for a response back from the cache header middleware and check if the response is a 304. If 304, go to the cache and retrieve the response. Otherwise update the response in the cache.

Is this the ideal way of doing cache invalidation? Is there a better way of doing it? With above method, I'll have to inspect each 304 response, determine the route, and have some sort of logic to verify what cache key to use. Not sure if that is the best approach.

If you can provide some guidelines and documentation/tutorials on cache invalidation, that would be really helpful.

Improve this question
3
  • cache invalidation to me means that you are able to detect changes at your origin and when they occur some process looks in your cache to remove/refresh the items. I would look for something along those lines and see if your publishing systems offers a notification system so you can react to changed items. Commented Dec 11, 2018 at 21:11
  • @NoRefundsNoReturns Do you mean to say whenever an update take place on an object, go ahead and update the cached value with that response? Not sure how this would work as some actions might return a collection of items and a user might be updating a single one. There would have to be a lot of tracking done on every single object in the system and in the cache to do it that way. Commented Dec 11, 2018 at 21:25
  • Whatever your "key" is to get data from your origin, I'm saying that whatever changes that data in your origin may have some kind of change notification that you can subscribe to. I support an invalidation service that listens to the publishing systems and removes items in cache following changes. Subsequent requests suffer a cache miss that itself subsequently goes to origin and subsequently reload the new item in the cache. Commented Dec 11, 2018 at 21:29

1 Answer 1

Reset to default
3

Here is a guideline based on how a service I support uses cache invalidation on a CQRS system.

The command system receives create, update, delete requests from clients. The request is applied to Origin. The request is broadcast to listeners.

A separate invalidation service exists and subscribes to the change list. When a command event is received, the configured distributed caches are examined for the item in the event. A couple of different actions are taken based on the particular system.

The first option is the Invalidation service removes the item from a distributed cache. Subsequently consumers of the services sharing the distributed cache will eventually suffer a cache miss, retrieve the item from storage and add the latest version of the item to the distributed cache. In this scenario there is a race condition between all of the discreet machines in the services and Origin may receive multiple requests for the same item in a short window. If the item is expensive to retrieve this can strain Origin. But the Invalidation scenario is very simple.

The second option is the Invalidation service makes a request to one of the services using the same distributed cache and asks the service to ignore cache and get the latest version of the item from Origin. This addresses the potential spike for multiple discreet machines calling Origin. But it means the Invalidation service is more tightly coupled to the other related services. And the service now has an API that allows a caller to bypass its caching strategy. Access to the uncached API would need to be secured to just the Invalidation service and other authorized callers.

In either case, all of the discreet machines that use the same redis database also subscribe to the command change list. Any individual machine just processes changes locally by removing items from its local cache. No error exists if the item is not present. The item will be refreshed from redis or Origin on the next request. For hot items, this means multiple requests to Origin could still be received from any machine that has removed the hot item and redis has not yet been updated. It can be beneficial for the discreet machines to locally "cache" and "item being retrieved" task that all subsequent request can await rather than calling Origin.

In addition to the discreet machines and a shared redis, the Invalidation logic also extends to Akamai and similar content distribution networks. Once the redis cache has been invalidated, the Invalidation routine uses the CDN APIs to flush the item. Akamai is fairly well-behaved and if configured correctly makes a relatively small number of calls to Origin for the updated item. Ideally the service has already retrieved the item and copies exist in both discreet machines local caches and the shared redis. CDN invalidation can be another source of spikes of requests if not anticipated and designed correctly.

In redis, from the discreet machines sharing it, a design that uses redis to indicate an item is being refreshed can also shield origin from multiple requests for the same item. A simple counter whose key is based on the item ID and the current time interval rounded to the nearest minute, 30 seconds, etc. can use the redis INCR command the machines that gets the count of 1 access Origin while all others wait.

Finally, for hot items, it can be helpful to have a Time To Refresh value attached to the item. If all payloads have a wrapper similar to below, then when an item is retrieved and its refresh time has passed, the called performs a background refresh of the item. For hot items this means they will be refreshed from cache before their expiration. For a system with heavy reads and low volumes of writes, caching items for an hour with a refresh time of something less than an hour means the hot items will generally stay in redis.

Here is a sample wrapper for cached items. In all cases it is assumed that the caller knows type T based on the item key being requested. The actual payload written to redis is assumed to be a byte array serialized and possibly gzip-ed. The SchemaVersion provide a hint to how the redis string is created.

interface CacheItem<T> {
  String Key {get; }
  DateTimeOffset ExpirationTime {get; }
  DateTimeOffset TimeToRefresh {get; }
  Int SchemaVersion {get;}
  T Item {get; }
}

When storing var redisString = Gzip.Compress(NetDataContractSerializer.Serialize(cacheItem))

When retrieving the item is recreated by the complementary uncompress and deserialize methods.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Thank you. I'll have to look at this as I'm not too familiar with implementing this.That explanation gives me a direction to follow and verify if it will work. My use case is relatively simple so I might make it a bit less complex
This is the design for a service that gets 20K RPS 24x7x365 with updates running 500 RPS. Pick and choose. If the Time To Refresh piece works for you and you can live with a 15 minute delay between publishing and it appearing in your system that would be the easiest place to start. Set your local cache timeout to 5 minutes and the items will disappear. Refresh from distributed cache at TTR.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.