Isn’t this a bad hashing mechanism. If one thread takes a responsibility of downloading files from a single web server, then this can to bad load balancing. For example a thread responsible for downloading content from Wikipedia.
1 Like
Hi @ashok2
There is always a point of diminishing returns on response time versus the number of threads in a system. If you have too many threads running concurrently, the system’s overall response time gets worse. The operating-system thread scheduler (or Java-system thread scheduler, if you’re not using OS threads) spends more and more time managing threads, and this overhead takes up the CPU time rather than allowing the threads to run.
You also need to consider whether the queue object handles the responses (collecting them from the request processes and handing them back to the clients) or whether the request-processor objects can hand the responses back directly. The former design has the advantage that the client cannot get any direct access to the request-processor objects, but the disadvantage is that you are introducing an unnecessary bottleneck in processing terms. The latter option (handing responses back directly), of course, has the opposite characteristics: no extra bottleneck, but access to client objects is enabled.
Feel free to discuss in case of any confusion.