Apache Airflow: The Hands-On Guide
Learn Apache Airflow! Hands-on videos cover Airflow, AWS, Kubernetes, Docker & more. Master workflow automation.
Course Content
Apache Airflow: The Hands-On Guide is an all-levels Udemy course taught by Marc Lamberti. It runs 10h 50m, is taught in English, and includes a certificate of completion. Learners rate it 4.5 out of 5 from 5,270 reviews, with 42,570 students enrolled. It is a paid course, last updated May 2025.
Master Apache Airflow: Data Pipeline Orchestration & Workflow Automation
Unlock the power of programmatic workflow management with Apache Airflow, the leading open-source platform for authoring, scheduling, and monitoring data pipelines. This comprehensive course equips you with the skills to become proficient in Airflow, a highly sought-after skill for data professionals.
What You'll Discover:
- Airflow Fundamentals: Understand the core concepts of Airflow, including the roles of the scheduler and web server. Learn what Airflow is and how it works.
- Hands-on with Data Pipelines: Dive into the Forex Data Pipeline project, a practical exercise showcasing a wide range of Airflow operators. Work with popular technologies like Slack, Spark, and Hadoop.
- DAG Mastery: Learn to design, build, and maintain robust Directed Acyclic Graphs (DAGs). You'll gain expertise in:
- Timezone Handling: Properly manage timezones within your DAGs.
- Unit Testing: Ensure the reliability of your DAGs through unit testing.
- DAG Folder Structure: Organize your DAGs for maintainability.
- Scalable Airflow: Explore different executors to scale your Airflow deployments:
- Local Executor: Ideal for development and testing.
- Celery Executor: For distributed task execution.
- Kubernetes Executor: Leverage the power of Kubernetes for scalable and resilient pipelines. Learn how to specialize workers, add new workers, and handle node failures.
- Kubernetes Integration: Set up a Kubernetes cluster (using Rancher) and integrate it with Airflow using the Kubernetes Executor to run your data pipelines.
- Advanced Airflow Concepts: Expand your knowledge with practical examples of:
- DAG Templating: Create reusable DAGs with templating.
- DAG Dependencies: Establish dependencies between DAGs for complex workflows.
- SubDAGs: Organize complex workflows using SubDAGs.
- Deadlock Prevention: Understand and avoid deadlocks in your DAGs.
- Monitoring & Alerting: Implement comprehensive monitoring using Elasticsearch and Grafana to track pipeline performance and identify issues.
- Airflow Security: Secure your Airflow instance with best practices:
- RBAC (Role-Based Access Control): Manage user permissions effectively.
- Authentication & Password Protection: Secure access to the Airflow UI.
- Data Encryption: Protect sensitive data.
- Practical Application: Benefit from numerous practical exercises to reinforce your learning.
- Best Practices: Learn and apply industry best practices for optimal Airflow usage.
- Knowledge Assessment: Test your understanding with quizzes at the end of each section.
This course provides dedicated support to answer your questions quickly and efficiently. We've crafted this content with care to provide you with the best possible learning experience. By the end of this course, you'll have the confidence to effectively use Apache Airflow for all your data pipeline orchestration needs.
Hire the developers behind TutorialSearch.io
We built this site — full-stack web apps, AI integration & automation. Tell us what you need and we'll build yours.