Udemyallen

Apache Airflow on AWS EKS: The Hands-On Guide

Deploy Apache Airflow on AWS EKS with Kubernetes Executor. Build production-ready workflows! Learn setup & best practices.

4.4(409)
5,490students
7h 29m
Certificate
Marc Lamberti
Marc LambertiApache Airflow Expert, Data Engineer

Course Content

Apache Airflow on AWS EKS: The Hands-On Guide is an all-levels Udemy course taught by Marc Lamberti. It runs 7h 29m, is taught in English, and includes a certificate of completion. Learners rate it 4.4 out of 5 from 409 reviews, with 5,490 students enrolled. It is a paid course, last updated March 2023.

Looking to deploy Apache Airflow on AWS EKS? This course provides a comprehensive guide to building a production-ready Airflow architecture on Amazon Elastic Kubernetes Service (EKS). Addressing the common challenges of using the official Helm chart, this training is designed for users who already have a foundational understanding of Airflow.

Here's what you'll learn:

  • EKS Cluster Configuration: Establish a robust and secure EKS cluster using best practices for optimal performance and reliability.
  • GitOps Automation: Implement GitOps principles for automated deployments and seamless updates to your Airflow environment.
  • Helm for Airflow Deployment: Leverage Helm to simplify the configuration and deployment of Airflow on Kubernetes.
  • Kubernetes Executor & Features: Configure the Airflow Helm chart to utilize the Kubernetes Executor and unlock a wide range of powerful features.
  • DAG Deployment with Git-Sync & EFS: Automate DAG deployment using Git-Sync and AWS EFS for version control and data persistence.
  • CI/CD Pipelines for Airflow: Implement CI/CD pipelines with AWS CodePipeline to streamline Airflow deployments and updates.
  • DAG Testing: Learn how to automatically test your Airflow DAGs to ensure data quality and pipeline reliability.
  • Secret Management: Securely store and manage credentials and sensitive data using a Secret Backend.
  • Remote Logging: Enable remote logging to AWS S3 for centralized log management and analysis.
  • Environment Management: Create and manage multiple environments – dev, staging, and production – for testing and deployment.
  • Scalability & High Availability: Design a scalable and highly available production environment for your Airflow on EKS setup.

Important Note: This course assumes you have existing knowledge of Airflow, Kubernetes, Docker, and AWS. This course focuses on the architectural setup, not on developing DAGs or interacting with AWS services within your DAGs. It is not eligible for AWS Free Tier due to the use of multiple AWS services required for a real-world deployment.

$ whoami — the devs behind tutorialsearch.io

Hire the developers behind TutorialSearch.io

We built this site — full-stack web apps, AI integration & automation. Tell us what you need and we'll build yours.

5.0(173)
1h 27m
all
skillshare
5.0(68)
1h 18m
all
skillshare
5.0
11m
beginner
pluralsight

AWS DynamoDB Deep Dive

by Ivan Mushketyk

5.0
6h 8m
intermediate
pluralsight
5.0
18m
intermediate
pluralsight