Software Engineering Daily

Software Engineering Daily


Apache Airflow with Maxime Beauchemin, Vikram Koka, and Ash Berlin-Taylor

June 10, 2020

Apache Airflow was released in 2015, introducing the first popular open source solution to data pipeline orchestration. Since that time, Airflow has been widely adopted for dependency-based data workflows. A developer might orchestrate a pipeline with hundreds of tasks, with dependencies between jobs in Spark, Hadoop, and Snowflake. Since Airflow’s creation, it has powered the