Exploring DAG runs
DAG
Stamps
Initial conditions
Categories:
Google Cloud
Make sure that you're on the DAGs tab in the Airflow web interface. It takes several minutes for this process to complete. Refresh your browser to make sure you're looking at the latest information.
...In Airflow, click composer_hadoop_tutorial to open the DAG details page. This page includes several representations of the workflow tasks and dependencies.
In the toolbar, click Graph. Mouseover the graphic for each task to see its status. Note that the border around each task also indicates the status (green border = running; red = failed, etc.).
...Click the "Refresh" link to make sure you're looking at the most recent information. The borders of the processes change color as the state of the process changes Note: If your Dataproc cluster alre
...Once the status for create_dataproc_cluster has changed to "running", go to Navigation menu > Dataproc, then click on: Clusters to monitor cluster creation and deletion. The cluster created by
...Once Dataproc gets to a state of "Running", return to Airflow and click Refresh to see that the cluster is complete. When the run_dataproc_hadoop process is complete, go to Navigation menu > C
...Once all the steps are complete in the DAG, each step has a dark green border. Additionally the Dataproc cluster that was created is now deleted.