Skip to main content

Guide

Monitor an Apache Airflow DAG

A paused DAG is not scheduled by Airflow. Manual triggers still work, which can hide the problem during an investigation. A heartbeat after the final task succeeds catches the missing scheduled completion.

Start your 14-day free trial No card required. The trial uses Pro limits.
By Paul , published , last verified

The scheduled run that never exists

Airflow does not schedule a paused DAG. You can still trigger the DAG manually in the UI, so one successful manual run does not prove the timetable is active. The scheduler can produce no scheduled run and no task failure for your normal alerting to read.

Place a heartbeat task after the work you care about. It sends only when upstream tasks succeed. When the DAG is paused or never reaches its final task, the expected PostDeploy ping does not arrive.

Add a final heartbeat task

Store the full heartbeat URL in an Airflow Connection, Variable or secret backend. This example uses a Variable to keep the DAG file free of the token.

Example shell
airflow variables set postdeploy_heartbeat_url 'https://ingest.postdeploy.dev/h/hb_YOUR_TOKEN'

from airflow.models import Variable
from airflow.operators.bash import BashOperator
from airflow.sdk import DAG
from pendulum import datetime

with DAG(
    dag_id="nightly_report",
    schedule="17 3 * * *",
    start_date=datetime(2026, 1, 1, tz="UTC"),
    catchup=False,
) as dag:
    report = BashOperator(task_id="report", bash_command="/opt/report/bin/run")
    heartbeat = BashOperator(
        task_id="postdeploy_heartbeat",
        bash_command=f"curl -fsS {Variable.get('postdeploy_heartbeat_url')}",
    )
    report >> heartbeat

Verify scheduled execution

Check that the DAG is unpaused and has a next run. Trigger one manual run to verify the token and network path, then wait for or observe one scheduled run. These are separate checks because a paused DAG can still accept a manual trigger.

Example shell
airflow dags unpause nightly_report
airflow dags list-runs -d nightly_report
airflow dags trigger nightly_report

Troubleshoot a late heartbeat

First check whether the DAG is paused. Then check the DAG run and its upstream tasks. If a manual run works but scheduled runs do not appear, investigate the scheduler and timetable rather than the heartbeat task.

Questions

Does a paused Airflow DAG still run on its schedule?

No. Airflow does not schedule a paused DAG. You can still trigger it manually, which is why a manual success does not prove scheduled execution works.

Where should the heartbeat task go in an Airflow DAG?

Put it after the task or task group that represents successful completion. That makes a missing heartbeat meaningful when any required upstream work fails or does not start.

What PostDeploy does not do here

  • PostDeploy does not read Airflow task logs or scheduler logs
  • PostDeploy does not unpause a DAG
  • PostDeploy does not trigger or retry an Airflow DAG run

Sources

Links are the platform's own documentation for the failure mode described above.

Monitor this DAG with PostDeploy

14 days at Pro limits. No card required.

Start your 14-day free trial