Google Cloud Professional Data Engineer Course [2019 Update]

Master GCP for Data Engineering: Exam Prep & Practical Skills

This comprehensive course equips you with the essential knowledge and practical skills to pass the Google Cloud Certified Data Engineer exam. Designed for efficiency, this streamlined curriculum covers core GCP components used by big data engineers, focusing on real-world applications. Learn the why behind GCP services, crucial for exam success.

Data Storage Solutions: Choose the Right Fit

Understand GCP's diverse storage solutions, including:

• Cloud Storage: Scalable object storage for various data types.
• Cloud Datastore: NoSQL document database for flexible data models.
• Cloud Spanner: Globally distributed, scalable, strongly consistent database.
• Cloud SQL: Managed relational databases (MySQL, PostgreSQL, SQL Server).
• BigQuery: Serverless, highly scalable data warehouse for analytics.
• BigTable: NoSQL wide-column store optimized for time-series data.

We'll compare these solutions, exploring ideal use cases for each.

BigQuery: Data Warehousing & Analysis

• BigQuery Pricing: Understand the pricing structure and strategies for cost optimization.
• Partitioned & Wildcard Tables: Learn to optimize query performance.
• Authorized Views: Securely share data with controlled access.
• BigQuery SQL: Practical exercises using Standard SQL.
• Temporary Tables: Utilize temporary tables for efficient data manipulation.
• Schema Types: Explore the different schema types supported by BigQuery.

Cloud Dataflow: Stream & Batch Data Processing

• Dataflow Fundamentals: Learn the Dataflow model and its role in GCP data pipelines.
• Deploying Dataflow Jobs: Practical guidance on deploying and monitoring Dataflow jobs.
• Dataflow Monitoring: Utilize the Cloud Dataflow Monitoring Interface and gcloud commands.
• Streaming Dataflow: Update streaming jobs on the fly.
• Dataflow Logging: Implement effective logging for troubleshooting.
• Local & Cloud Execution: Run Dataflow jobs locally and on the Dataflow service.

Hadoop & Dataproc: Big Data Processing

• Hadoop Overview: Understand the core components of Hadoop and its relevance to GCP.
• Dataproc: Managed Hadoop and Spark service for scalable data processing.
• Apache Spark Jobs: Utilize Spark for fast data processing.

Cloud Dataflow Practical: Local & Cloud Execution

Gain hands-on experience running Dataflow jobs both locally and leveraging the Dataflow service for scalable processing.

Stackdriver: Monitoring, Logging & Debugging

• Stackdriver Functions: Leverage Stackdriver for debugging, error reporting, monitoring, alerting, and tracing.
• Logging: Centralized logging for comprehensive system insights.
• Export Logs to BigQuery: Analyze logs for deeper insights using BigQuery.

Machine Learning Solutions: AI with GCP

• Cloud Machine Learning Engine: Build and deploy custom machine learning models.
• BigQuery ML: Create and execute machine learning models directly within BigQuery.
• Kubeflow & Spark ML: Leverage open-source ML frameworks.
• Cloud AutoML: Automated machine learning for vision (AutoML Vision) and natural language (AutoML Natural Language).
• Dialogflow: Build conversational chatbots.
• Tensor Processing Unit (TPU): Understand Google's specialized hardware for accelerating machine learning workloads.
• Machine Learning Terminology: Learn key concepts like features, labels, models, linear/logistic regression, classification, clustering, and supervised/unsupervised learning.

Migration to GCP: Data Transfer

• Transfer Appliance: Bulk data transfer to GCP.
• Storage Transfer Service: Automated data transfer from various sources.

Dataprep: Data Preparation

• What is Dataprep? Learn about Dataprep and its role in data preparation workflows.
• Hands-on Dataprep: Run Dataflow jobs in the background for efficient data transformation.

Security on GCP: Best Practices

• Cloud Security: Implement best practices for securing your GCP environment.
• Cloud Storage Security: Securely interact with Cloud Storage.
• Penetration Testing: Understand penetration testing methodologies.
• Bastion/Jumphost: Secure access to your GCP resources.
• Encryption: Protect data at rest and in transit.
• Data Loss Prevention (DLP): Prevent sensitive data leakage.
• Live Migration: Understand live migration techniques.

Cloud Composer: Workflow Orchestration