Udemyintermediateen

Master AWS Lambda Functions for Data Engineers using Python

Learn Python Lambda functions! Deploy with Docker & layers, validate data with Glue & Athena. Serverless development skills.

4.3(266)
9,623students
13h 19m
Certificate
Durga Viswanatha Raju Gadiraju
Durga Viswanatha Raju GadirajuCEO at ITVersity and CTO at Analytiqs, Inc

Course Content

Master AWS Lambda Functions for Data Engineers using Python is an intermediate-level Udemy course taught by Durga Viswanatha Raju Gadiraju. It runs 13h 19m, is taught in English, and includes a certificate of completion. Learners rate it 4.3 out of 5 from 266 reviews, with 9,623 students enrolled. It is a paid course, last updated December 2024.

Learn to build powerful, end-to-end data pipelines with AWS Lambda Functions using Python! This comprehensive course guides you through the entire process, from data ingestion to analysis, leveraging key AWS services.

What you'll master:

  • AWS Lambda Function Development & Deployment: Develop and deploy Lambda functions using various methods:
    • Zip Files: Learn the standard approach for deploying Lambda functions.
    • Custom Docker Images: Build and deploy Lambda functions with custom dependencies and environments using Docker.
    • Lambda Layers: Utilize layers to share code and dependencies across multiple Lambda functions, promoting code reusability.
  • Data Pipeline Architecture: Design and implement robust data pipelines using a combination of AWS services.
  • Python & AWS Services Integration: Develop Python applications that seamlessly interact with a wide range of AWS services.
  • Essential AWS Services: Gain hands-on experience with:
    • Boto3: The AWS SDK for Python, enabling programmatic access to AWS services.
    • Amazon S3: Object storage for data lakes and data warehousing.
    • Amazon DynamoDB: A NoSQL database for storing metadata and job details.
    • Amazon ECR (Elastic Container Registry): A fully managed container registry for storing and managing Docker images.
    • Amazon CloudWatch: Monitoring and logging for your Lambda functions.
    • AWS Glue Catalog: A central metadata repository for your data assets.
    • Amazon Athena: Serverless interactive query service for analyzing data in S3.
    • Amazon EventBridge: Serverless event bus for building event-driven applications.
    • AWS Step Functions: Orchestrate complex workflows and coordinate multiple Lambda functions.
  • ETL Data Pipelines: Build Extract, Transform, Load (ETL) pipelines to process and transform data.
  • Data Transformation with Pandas: Utilize Pandas for efficient data manipulation and transformation, including converting large datasets to Parquet format.
  • S3 Event Notifications: Trigger Lambda functions automatically based on S3 events.
  • Scheduled Lambda Functions: Schedule Lambda functions using AWS EventBridge for automated data processing.
  • Data Validation with Athena: Validate data quality and insights using SQL queries against data stored in S3 via Athena.

Getting Started & Environment Setup:

  • AWS Account Setup: Create an AWS account and configure the AWS CLI.
  • Development Environment: Set up a development environment on Windows and Mac using:
    • WSL (Windows Subsystem for Linux): Run a Linux environment on Windows.
    • Docker Desktop: Containerization for consistent development environments.
    • Visual Studio Code: A powerful code editor with extensions for AWS development.
    • Remote Development Extension Kit: Seamlessly connect to your WSL environment from VS Code.
  • Project Setup: Establish a project structure for developing Python applications with AWS services.

Key Takeaways:

  • Deploy Python Applications as Lambda Functions: Develop and deploy Lambda functions using both zip files and custom Docker images.
  • Monitor & Troubleshoot with CloudWatch: Effectively monitor and troubleshoot Lambda function performance using CloudWatch logs.
  • Complete Codebase: Access the complete application code and notebooks used in the demonstration.
  • Broad AWS Service Proficiency: Gain the ability to build solutions leveraging a wide range of AWS services.
$ whoami — the devs behind tutorialsearch.io

Need a web app built? Hire the team behind TutorialSearch.io

We ship fast, production-grade full-stack apps (Next.js, React, edge infra) — the same stack that powers this site.

5.0(173)
1h 27m
all
skillshare
5.0(122)
3h 38m
all
skillshare
5.0(68)
1h 18m
all
skillshare
5.0
11m
beginner
pluralsight