educative.io

Educative

Benefit from the Write-back cache by updating all counters and timestamps in cache only

Our system can get huge benefits from caching recent active users. Application servers can quickly check if the cache has the desired record before hitting backend servers. Our rate limiter can significantly benefit from the Write-back cache by updating all counters and timestamps in cache only. The write to the permanent storage can be done at fixed intervals. This way we can ensure minimum latency added to the user’s requests by the rate limiter. The reads can always hit the cache first; which will be extremely useful once the user has hit their maximum limit and the rate limiter will only be reading data without any updates.

isn’t write back cache inefficient? all data is put in cache, not data that is likely to be used. is the benefit of higher hit rate enough to offset the drawback of greater memory requirement?

I think you got it wrong. Not all data will be put in cache. We will still apply the Least Recently Used Policy to gradually clear clean up some space in the cache storage. Furthermore I think the process of gradually writing the cache data to persistent data should happen right before you clear out some cache data based on LRU policy that way, before our data will be consistent. It just matter of time that we updated it to the persistent database. In conclusion we are using the write-back cache policy so that client can get a faster response time for request (low latency).

@Stephen_Thajeb - But the write that you put into the cache may never be read again. Assuming that most writes are read very often, the write-back seems inefficient.

On that point, I agree with you so I thinik it’s is just a trade off between you want the “read” or “write” process to be faster. If you want the write to be faster then you could use the write-back cache which will potentially caused a lot of cache miss, but if you want to read faster you should use other caching policy. In conclusion, I think System Design interview is an open ended question. There is no exact answer, you can alwasy assume things and as long as you provide a good reason for your assumption you are good to go.