educative.io

Seems you use redundant functions in examples with Tenacity

Courses
,
#1

Both in Designing for Failure chapter and Crawler project it seems you introduce functions that are not necessary. The best example in the Crawler project:

 def do_something(url_to_crawl):
    dic = {}
    r = requests.get(url = url_to_crawl)
    if r.status_code != 200:
        raise RuntimeError
    text = r.text
    dic['data'] = text
    dic['status_code'] = r.status_code
    return dic

@app.task()
def getURL(url_to_crawl):
    @tenacity.retry(wait=tenacity.wait_fixed(5), stop=tenacity.stop_after_attempt(3))
    def do_something_and_retry(url_to_crawl):
        return do_something(url_to_crawl)
    return do_something_and_retry(url_to_crawl)

Why not just something like:

@app.task()
@tenacity.retry(
    wait=tenacity.wait_exponential(multiplier=0.5, max=30, exp_base=2),
    stop=tenacity.stop_after_delay(60),
    retry=tenacity.retry_if_exception_type(ValueError)
)
def get_url(url):
    response = requests.get(url)
    if response.status_code >= 300:
        raise ValueError(f"Bad response code: {response.status_code}")
    return response.status_code

Course: https://www.educative.io/courses/hackers-guide-scaling-python
Lesson: Project Solution Review - The Hacker's Guide to Scaling Python