Pluralsightexperten

Handling Streaming Data with Azure Databricks Using Spark Structured Streaming

Learn Spark Structured Streaming with PySpark on Azure Databricks. Build reliable, end-to-end streaming pipelines. Deep dive into features and practical applications.

Related:Spark SQL
4.7(33)
2h 28m
Certificate
Mohit Batra
Mohit BatraInstructor

Course Content

Handling Streaming Data with Azure Databricks Using Spark Structured Streaming is an advanced Pluralsight course taught by Mohit Batra. It runs 2h 28m, is taught in English, and includes a certificate of completion. Learners rate it 4.7 out of 5 from 33 reviews. It is a paid course, first published November 2020.

Build Real-Time Data Pipelines with Azure Databricks

This course teaches you how to harness the power of Spark Structured Streaming on Azure Databricks to create robust, real-time data pipelines. Learn to handle streaming data from multiple sources and continuously generate results.

What you'll learn:

  • Spark Structured Streaming Fundamentals: Get a quick overview of the Spark Structured Streaming processing model and understand its core concepts.
  • Environment Setup: Learn to configure your environment for building streaming applications on the Microsoft Azure platform.
  • Data Source & Sink Configuration: Master the process of configuring data sources and sinks for your streaming pipelines.
  • End-to-End Pipeline Development: Build complete streaming pipelines by extracting data from diverse sources, transforming it, and loading it into destinations like Azure Data Lake, Azure Event Hubs, and Azure SQL.
  • Timestamp Handling: Understand and utilize different timestamps associated with events for accurate data processing.
  • Windowing Operations: Learn how to effectively aggregate data using Windows in Spark Structured Streaming.
  • Stream Enrichment: Combine streaming data with static or historical datasets to enrich your insights.
  • Multiple Stream Processing: Discover how to effectively combine multiple streams into a single, cohesive pipeline.
  • Production Deployment: Build production-ready pipelines and schedule them as jobs within Databricks.
  • Databricks CLI Management: Learn to manage your pipelines using the Databricks CLI for efficient monitoring and control.

By the end of this course, you'll be proficient in developing complex streaming pipelines on Azure Databricks to address a wide range of business challenges. Gain the skills to process streaming data in real-time and unlock valuable insights from your data.

$ whoami — the devs behind tutorialsearch.io

Want AI in your product? Hire the team behind TutorialSearch.io

LLM features, RAG, agents and automation wired into a real product — built by the devs who shipped this site.

5.0
53m
beginner
pluralsight
5.0
1h 29m
beginner
pluralsight
5.0(766)
2h 58m
intermediate
skillshare