In this example, we estimated and average size of 100 bytes per message.
We later estimated a server can handle 50k concurrent connections, and to handle 500 millions connections we’d need 10K servers.
if the user is always on a specific server, why not cache their message history there? The memory use would be minimal even if we stored a large history there
100 bytes per message * 50K users * last 100 messages per user = 500MB
and caching the last 100 messages even sound extreme to me.
hi John,
We have looked at your problem and working on it. We will get back to you when this is resolved.
Thank you.
Yes this seems reasonable. There are many good answers for caching problems, depending upon the scenario.
So if we want to cache latest 10 conversation of every user, with last 10 messages for every conversation, we will need half a GB cache, as you calculated. Which is not too much. But, as we estimated, if we have 10K servers, we will need 10K*0.5GB => 5TB overall cache. Would you still go with this solution? Or will you decide to have dedicated cache servers? Every solution has pros/cons. Always remember to discuss them during the interview.
@Design_Gurus
Whats the cons of having application level cache… I mean having cache on each server?
So for latest 10 latest message each user will be : 50K * 10 * 100 Bytes = 50 GB each server. Do you think 50GB is not good number to be on each server ?