Hi @Dmitry_Polovinkin !!
In Apache Airflow, task heartbeats are a mechanism to ensure that the scheduler and other components can monitor the progress of a running task. When a task is running, it sends periodic heartbeats to the metadata database to indicate that it is still alive and making progress. These heartbeats help the scheduler and other components track the health and progress of tasks.
To address your specific question, changing the status of a DAG run to “Failed” does not directly stop the task heartbeats. The heartbeat mechanism operates independently of the overall DAG run status. Even if a DAG run is marked as “Failed,” the individual tasks within that run can still be actively running and sending heartbeats.
The scheduler regularly checks the database for these heartbeats to determine if a task is making progress. If the scheduler detects that a task has not sent a heartbeat within a specified time interval, it may consider the task as stalled or failed, and it can take actions such as marking the task as failed, restarting it, or killing the corresponding process.
The heartbeat mechanism is essential for handling scenarios where tasks may run for an extended period, and it helps the scheduler manage and monitor task execution. It ensures that even if a task is not completing quickly, the scheduler can still detect if it is making progress or has encountered issues.
In summary, changing the overall status of a DAG run does not directly impact the task heartbeats. The scheduler relies on these heartbeats to make informed decisions about the state of individual tasks and take appropriate actions based on their progress or lack thereof.
| hope it helps. Happy Learning