Caching

What is a cache?

A cache is a component that stores portions of data that would otherwise either take a long time to calculate/process. Caches may also originate from another underlying backend system, where caching is used to prevent additional requests for round trips for frequently used data. Caching can be used to improve performance or reduce application latencies in both these cases.

A cache hit means the data required for the request is in the cache and is served from the cache.

A cache miss means the data required for the request is not in the cache and would need to be computed by the backend systems. 

Why we should cache??

1) Reduce network calls. (Store data in cache and return that data to user instead to hit the DB)

2) Avoid repeated computations. (Suppose we want average age of employees, its a costly operation, we should store it in cache)

3) Reduce DB load.

4) Increase the response time.

Cache Policy - When I have to add data ad when we have to evict the data.

1) We can't store all the requests in Cache, we have to predict which requests we have to save and they can come again in future.

2) For eviction we can use any cache eviction policy according to our need.

Cache Eviction Algorithms

1) Least Recently Used (LRU)

2) Least Frequently Used (LFU)

3) First In First Out (FIFO) (Sliding Window)

LRU

This is the default and is a variation on Least Frequently Used. The oldest element is the Less Recently Used (LRU) element. The last used timestamp is updated when an element is put into the cache or an element is retrieved from the cache with a get call.

LFU

For each get call on the element the number of hits is updated. When a put call is made for a new element (and assuming that the max limit is reached) the element with least number of hits, the Least Frequently Used element, is evicted.

FIFO

Elements are evicted in the same order as they come in. When a put call is made for a new element (and assuming that the max limit is reached for the memory store) the element that was placed first (First-In) in the store is the candidate for eviction (First-Out).

Cache issues if we don't use the right things

1) Choose right eviction policy.

2) The cache store should not be very small as it can increase the overhead. Suppose X user queries something and hit the DB and store it in the Cache and now Y user request some queries, hit the DB and store it in the cache by replacing X result as we can store only 1 element. It is called as thrashing.

3) Data inconsistency, suppose we have updated some result in the DB and cache is not updated. So cache will return the old data.

Caching Strategies

1) Cache Aside Strategy

1. When the request is received by the server/app, the app checks for the data in the cache. If the data is there in the cache, it’s a cache hit and served from the cache.
2. If the data is not available in the cache, then it’s a cache miss.
3. The app fetches the data from the database and updates the cache with the fetched data.

2) Write through Cache

In this strategy, the cache has a writer component that can write to database.

1. The application receives request to write data to database.
2. The application writes the data to the cache.
3. The cache invokes the writer to write to database as well as update the cache.

3) Read through Cache

 

The cache is configured with a loader component that loads data from the database.

1. When the application receives the read request, it asks the cache for data associated with the key.
2. If there is a cache hit, then the data is served from the cache.
3. If there is a cache miss, then the cache invokes the loader that fetches the data from the database, updates the cache, and serves the fetched data.
4. The next time there is a read request for the same data, it’s served from the cache.

4) Write back cache

1. The app writes data to the cache and receives an immediate acknowledgement that the write is successful.
2. The cache internally maintains a buffer to save the writes.
3. The cache asynchronously writes the data from the buffer to the database at a later point in time.