Building Data Pipelines with Luigi 3 and Python
Build robust data pipelines with Python and Luigi. Avoid messy scripts & maintenance headaches. Learn to create reliable, scalable data workflows.
Course Content
Building Data Pipelines with Luigi 3 and Python is an intermediate-level Pluralsight course taught by Dan Tofan. It runs 1h 34m, is taught in English, and includes a certificate of completion. Learners rate it 4.8 out of 5 from 21 reviews. It is a paid course, first published June 2024.
Data from diverse sources often requires significant processing before it's useful. It's a common pitfall to try and build your own data pipeline library for batch processing, but this often leads to complex and hard-to-maintain systems.
This course, Building Data Pipelines with Luigi and Python, provides a practical path to creating robust and manageable data workflows. You'll gain hands-on experience with a powerful open-source tool.
Here's what you'll learn:
- Building Data Pipelines: Start by constructing your first data pipelines using Luigi and Python. We'll cover the core concepts and practical implementation.
- Luigi Pipeline Configuration: Master the art of configuring your Luigi pipelines for optimal performance and flexibility. Learn how to define tasks, dependencies, and execution strategies.
- Running Luigi Pipelines: Discover the different ways to run your pipelines, from simple command-line execution to more advanced scheduling and monitoring techniques.
By the end of this course, you'll possess the essential Luigi skills and knowledge to design, build, and maintain data pipelines with ease. Say goodbye to fragile, difficult-to-debug systems and hello to scalable, reliable data processing! Learn how to streamline your batch processing workflows.
Want AI in your product? Hire the team behind TutorialSearch.io
LLM features, RAG, agents and automation wired into a real product — built by the devs who shipped this site.