Pluralsightintermediateen

Building Pipelines for Workflow Orchestration Using Google Composer

Learn Cloud Composer: build, schedule & monitor data pipelines with shell & Python. Master workflow orchestration for data processing. Fully managed service.

4.3(19)
1h 58m
Certificate
Vitthal Srinivasan
Vitthal SrinivasanInstructor

Course Content

Building Pipelines for Workflow Orchestration Using Google Composer is an intermediate-level Pluralsight course taught by Vitthal Srinivasan. It runs 1h 58m, is taught in English, and includes a certificate of completion. Learners rate it 4.3 out of 5 from 19 reviews. It is a paid course, first published August 2024.

Unlock the power of workflow automation with Google Cloud Composer, a robust pipeline orchestration service built on the widely adopted Apache Airflow API. Launched in May 2018, Composer is rapidly becoming the go-to solution for building and executing complex system pipelines on Google Cloud Platform (GCP).

This course, Building Pipelines for Workflow Orchestration Using Google Composer, provides a comprehensive understanding of how to leverage Composer to quickly create pipelines with interconnected tasks. You'll gain clarity on Composer's role within the broader GCP ecosystem and understand how it stacks up against other pipeline services like Dataflow.

Here's what you'll learn:

  • Understanding Google Cloud Composer: Discover what a Composer environment is and how to define and run your pipelines within it.
  • Airflow Operators for Pipeline Building: Master the use of Airflow operators to execute a variety of tasks, including running shell scripts, executing custom Python code, and implementing sophisticated control flow within your pipelines.
  • GCP-Specific Operators: Learn to utilize Airflow's specialized GCP operators to seamlessly integrate with key services:
    • Email Integration: Automate notifications and alerts.
    • BigQuery Interaction: Process and analyze data stored in BigQuery.
    • Dataproc Cluster Management: Instantiate and manage Dataproc clusters for big data processing.

By the end of this course, you'll be equipped with the skills and knowledge to confidently build and deploy complex pipelines using the Apache Airflow API and the capabilities of Google Cloud Composer. Take your data processing and workflow automation to the next level with this powerful GCP service.

$ whoami — the devs behind tutorialsearch.io

Need a web app built? Hire the team behind TutorialSearch.io

We ship fast, production-grade full-stack apps (Next.js, React, edge infra) — the same stack that powers this site.

5.0
2h 42m
intermediate
pluralsight
5.0(28)
1h 31m
all
skillshare
5.0
1h 28m
beginner
pluralsight