The article says URL frontier crawls a url on each thread. I am trying to understand if this means fetching the actual content of the file.
We have another module, HTML Fetcher, to download the content. What is the actual difference exactly?
I would say the URL Frontier store/provides the URL that needs to be visited, whereas the Fetch does nothing but fetches the HTML content from a URL that URL Frontier provides.