Number of servers calc seems wrong

[Anton_van_der_Bijl]

The estimate for number of servers doesn’t make sense to me. It uses number of daily active users (500M) divided by request per second capacity of a server (8000) to get to 62500 servers. However this is dividing days by seconds without any conversion and ignores how many requests the daily active users perform on average (if it’s 1000 per day the server requirements should be 1000 times more than if it’s 1 per day).

Can somebody explain this to me? Why is it considered a reasonable estimate despite the above?

[Isra_Javaid]

Hi Anton_van_der_Bijl,
Can you please provide a link to the lesson and course for better understanding?
Thank you

[Anton_van_der_Bijl]

Examples of Resource Estimation - Grokking Modern System Design for Software...

[Isra_Javaid]

Hi Anton_van_der_Bijl,
Have you seen the detailed Calculation? It shows how they calculate the whole procedure. It is not directly calculated with seconds. First, it multiplies the request per user per day by the total users per day, then divides it into a single server.
It is fine for me, and I hope you get the point. If not, then you can ask freely again.

[Anton_van_der_Bijl]

Yes, I’ve seen the detailed calculation. That isn’t the problem. The end result of the detailed calculation is that 15 servers are required.

Then in the next paragraph they basically say this isn’t a great estimate followed by this:

Therefore, we approximate the number of servers by depicting how many clients a server handles on a given day:

That formula gives a result of 62500, but it doesn’t make sense to me for reasons as explained in my original post and it doesn’t agree at all to the detailed calculation (which appears at face value to be a much better one, even though the result feels intuitively too low).

[Isra_Javaid]

Hi Anton_van_der_Bijl,
The “Number of servers required” section starts by estimating the number of required servers through a simple calculation: (Number of requests per second)/(RPS of a server). Because we described that this calculation performed does not lead to a realistic number of servers, we emphasized WHY this estimation will not work. In reality, estimating the number of servers is NOT a straightforward calculation that can be done through a simple formula. We, therefore, devised a realistic approach (which, of course, doesn’t seem optimal as we wrote in the lesson). But we wanted a reference that would give a realistic enough estimation to be used in the rest of the course.
In a realistic world, the correct estimation would require consideration of many more parameters, some of which we described in the lesson. However, going through all of those is beyond the scope of the lesson, and therefore, we settled for something primitive.
I hope now your point is cleared. Let me know for further clarification.
Thank you

[Alena_Charnova]

+1; how did we jump from 15 servers to 62.5K servers? the above answer isn’t going to fly in any interview the 8K RPS per server is also not even close to any realistic RPS I’ve seen in the wild. It assumes that server would run hot 100% usage, you don’t want to get close to that. then you shouldn’t plan against your average QPS, you should look at peak QPS. you want to have redundancy, likely servers spread at least 2 data centers ideally surviving a complete loss of at least one. you want to be able to deploy, meaning take out percentage of your fleet for deployments. and then there’ll be more than one service likely at least several for any of the typical problems. this all can be easily explained and put into a formula

[Isra_Javaid]

Hello Alena_Charnova,
We think for a mix of CPU and memory-bound queries, 8K RPS is not unrealistic on a commodity server. Indeed for a specific workload, these numbers can change drastically. Further, using the 8K RPS number, we assumed that all 500 M user requests arrive simultaneously. It is because we want to prepare for the worst case. In reality, requests follow some arrival distribution (for example, diurnal patterns for day and night are different), and real services that use the cloud are elastic so that they can shed excess servers when the load is low and engage more when needed. If we know (possibly from our data) that at most “x” concurrent requests can come in on average, “y” request comes in per second, we can accordingly change the calculation. However, we keep things simple because these are back-of-the-envelope calculations.
I hope now your point is cleared. Let me know for further clarification.
Thank you

[xuan.nguyen.swe]

I understand the way to estimate number of servers in this chapter. But I would like to ask what could happen if I use it in the interview. How to convince the interviewer if s/he doesn’t agree with this approach?

[Educative_Team]

Hello xuan.nguyen.swe,
We have provided two approaches towards the estimation problem. The first one is technically sound but less plausible, while the second one seems practical. We preferred using the second approach because it is plausible as it is based on back-of-the-envelope calculations, which are considered rough. As far as Interviews are considered, it is a highly subjective debate. Interviewers can ask a variety of questions, and candidates can only debate on them. Regarding this very point, just like we said before, it will take more parameters and a lot more calculations, simulations, and the practical infrastructure to find accurate answers. There may be no rule of thumb here because different tech giants may use different approaches to meet their needs. At the end of the day, we have to make assumptions that can lead to plausible and reasonable answers, which is just the case here. Also, in interviews, evaluators assess candidates for all the sections of the interview and not just one portion.

I hope now your point is cleared. Let me know for further clarification.
Thank you

[xuan.nguyen.swe]

Thanks for your answer! This is a good answer in an interview, I think. We basically admit that this estimation is just one approach and it’s not accurate, but still more reasonable than calculating for a simple single layer of servers. We could recalculate and improve later when we have more details about the design, which could have multiple layers and complicated connection between servers.

[Dima]

We, therefore, devised a realistic approach

You have just picked an arbitrary value that seemed “realistic” to you and decided to use it throughout the course. That’s totally okay for this particular purpose, but you should have just said that instead of introducing a dubious formula.