Consider the below scenario,
LB sent 10 requests to server A, server A caters first 5 requests and becomes unavailable. Now, how the remaining 5 requests are handled?
How to handle the above scenario where the nature of the request is
Hi Gobi.
If LB is working at IP layer, it will act as a NAT proxy, forwarding client’s packets to a chosen back-end server. In that case, a connection will be between the end client and a back-end server (the LB will silently do the NAT operations). So if this server was asked to handle 10 customers’ requests, and it failed at some point. That will results either in a RESET packet at TCP level or something will time-out at the protocol or application level. Effectively that will tell the respective clients that the connection is broken and they will retry.
If LB is working at the TCP level, then SSL will terminate at the LB, and LB will make a new TCP connection with a backend server, and will forward clients’ data there. Now if the back-end server fails, LB will know it (its session will terminate), and LB will most probably send an RST packet to the client to terminate the connection. Client will need to retry. And now LB will forward to a healthy server.
For mission critical requests, we will probably have dedicated servers whose sole tasks will be to wait until work comes, and we will never overload such infrastructure. Also the application layer of such work will be extra vigilant regarding any latency increases etc.
If you have any further questions, feel free to reach out to us.
Thank you.