educative.io

Concerns in design

Courses
,
#1

Hi @Design_Gurus

Please clarify following doubts:

  1. Why do we need to maintain sorted set in reddis?Why cant we just maintain a list.We dont need actual request timestamp. Let us say we receive three request, now client may have send as A B C in order but we might have received as C B A. But does it matter if we process CB and reject A in case limit is 2 request? What i am trying to say we can use the timestamp at which we insert in redis list instead of the time stamp at which request was created by sender.In this way we dont need sorted set.Take a List and tail has new records and front has old.It will be way faster.

  2. I think in figure we should do load balancing at Load balancer and not webserver. The DDOS attack can send so many requests to we server that it is always busy in calling rate limiter.This should be done before request hits webserver probably at load balancer or api gateway

1 Like
#2

Hi @rahul9,

Hope you are doing good.

Let me try to answer your questions:

  1. If the client sends three requests A, B and C but we ignore client’s timestamp and take into account only the timestamp of when the requests where inside the system we can be unfair with the client. Let’s think about a scenario when the requests where sent at proper time distance (A and B within the same minute and C after a minute of B). If your system receive all three at same time, we gonna drop one of them. Or, we can have a different scenario, where the timestamps arrive into the system showing a timestamp greater than a minute but actually, the client sent all within a minute. Moreover, if your system is a system that process things sequentially, so, you need to process the requests in order, you can broke it (or imagine a system where the client do sequential processing… for example, request C depends on the output of request B, which depends on output of request A. Saying that, I think that keep the requests order and their timestamps are crucial to API Rate Limiter design

  2. Agree. Actually, this design can be improved. Once the request of a given user is dropped, we can have a cache where we keep a list of dropped users and a TTL. Before send a request to the API Rate Limiter, we check this cache and drop it right away if the user is still under a “dropped state”.

Hope that helps!

Regards,
Artur Baruchi
@abaruchi at twitter

2 Likes
#3

Thanks for answers.For point 1, in case we need sequential access, then i agree, but otherwise i dont see reason to maintain time stamp,Client may be hitting from different geographical locations and same ,so ideally speaking, client time stamp is not useful?You get a request from Client and you take or reject base on how many requests of client already are there with us in last second(or so based on design),until unless its a hard requirement,we can get away with this and have a simple list in which data in inserted in order of arrival and use system timestamp.The advantage is that it will improve speed up as insertion is 0(1) now ,no need for sorted insertion and rate limiter should be as fast as possible.What are you thoughts?

1 Like
#4

Hi @rahul9,

You do have a point. Probably keeping the TS of the system (i.e. when the request arrived to us) would cover several scenarios (probably, major part). I think that it is always a good practice to discuss this with the interviewer. Presenting Pros and Cons of each solution based on assumptions you both acknowledged will make you shine and give a very good impression about you. This discussion was a pretty good example of how the scope is important. Keeping the solution under some boundaries is extremely important.

Thanks!

Artur Baruchi

1 Like
#5

when we want to operationalize this, there are many challenges, for every request, it may not make sense to check the rate limiting service, becomes too chaty, rather, each frontdoor must have a ratelimiting library that should get its client side of cache updated periodically, and update the service periodically, may be for throttling payment failure txs, we require strict tracking, however, 99% of the usecases, eventual consistancy(soft throttling) is ok., based on the api ratelimiting preference, the library should decide how to impose the limit
This means, there must be mechanism to also client side caching the rate limiting preference.
making rate limiting against DOS attacks at IP level/user level/app/api level etc., again the preference configuration must be maintained at client library
Redis support the incr kind of api that must be used directly, we should not get into concurrency discussion for this problem space as we are purely focused on solving rate limiting problem.
rolling window solution to store each timestamp in set/list against user might become more expensive, must be made this option available purely based on preference.
Better to cover how some of the vendors like apigee/kong had built such solution - Designing API Rate Limiting Algorithm and Distributed Systems | Kong Inc.
The problem space is broad, discussed ratelimiting at different layers of stack in length here - Rate-limiting strategies and techniques  |  Cloud Architecture Center  |  Google Cloud
deciding the key - for multitenacity - better to make the key as hash(tenant,appid,method,userid)