educative.io

Why should we use request count instead of url count to estimate memory?

Courses
,
#1

Hi,

Since we have 20K requests per second, we will be getting 1.7 billion requests per day:
20K * 3600 seconds * 24 hours = ~1.7 billion
To cache 20% of these requests, we will need 170GB of memory.
0.2 * 1.7 billion * 500 bytes = ~170GB

I’m a little confused with the 20K being used.
20K is 200 URLs/s * 100 read requests, which means the unique URLs should be 200 URLs/s.
I think what we should cache is the unique URLs, not the requests. Do I get this right?

If so, 1.7GB would be the memory needed.

URLs: 200 URLs/s * 3600 seconds * 24 hours = ~17 million
cache memory: 0.2 * 17 million * 500 bytes = 1.7 GB

Please help me out about this calculation. Thanks.

4 Likes
#2

Both are valid ways to do caching. Please check documentation online for various caching strategies.
The technique mentioned in the article is called “read-through cache” and is typically used in read-heavy applications. When the first read comes in, it results in a cache miss, and the data is retrieved from storage and updated in the cache for use the next time. From them on, the cache hit makes returning the same tinyurl very fast.
What you mention is a “write-through” cache where some percentage of writes are cached. The issue here is that every write requires an update of both cache and storage (which increases latency for writes). Also, it gets tricky because during write we do not know how much read traffic is going to come to the data, so the cache size needed can be close to the size of the entire dataset).
A combination of both these types of caches will yield significant results in read-write heavy applications, but write-through will help more when a larger percentage of data is read.

1 Like