educative.io

Educative

Getting an error message whenever the defined threshold is crossed within a single server

  1. The APIs are accessible through a cluster, so the rate limit should be considered across different servers. The user should get an error message whenever the defined threshold is crossed within a single server or across a combination of servers.

does it make sense to get an error message whenever the defined threshold is crossed within a single server? shouldn’t it always be by cluster? if we are unlucky are all requests go to a single server by chance, because the load balancing was not evenly distributed, should we still have a rate limit error?

As the API requests are routed through a load balancer, one can expect that if one is seeing the error message, it is because the whole cluster of servers is busy. In any case, clearing the cache and reattempting the request is always a good idea.