Prophecy Data Transformation Copilot for Data Engineering

Unlock the Power of Cloud Data Lakes with Prophecy!

This comprehensive course empowers data engineers and data analysts to master the creation and deployment of a robust cloud data lakehouse architecture. Built with Prophecy's Data Transformation Copilot, this hands-on training is your ideal starting point for a successful data engineering journey leveraging Spark and Prophecy. (Developed by Prophecy Data.)

We'll guide you through the entire process, beginning with staging data from diverse sources, including Salesforce, operational databases with CDC (Change Data Capture), and machine-generated data like logs and metrics. Learn how to clean and normalize ingested tables to build a complete, efficient, and high-quality data model.

Through four practical projects, you'll gain expertise in building consumption applications for real-world scenarios:

• Spreadsheet Export for Finance: Develop a data model and transformation pipeline tailored for financial reporting. Crucially, you'll learn to implement unit tests and integration tests to ensure data quality and maintain data integrity.
• Operational Alerting System: Build an alerting system for operational support, focusing on orchestration best practices to proactively ensure customer success.
• Sales Data Upload to Salesforce: Explore advanced extensibility concepts to create a seamless data pipeline for uploading sales data back into Salesforce, adhering to standardized practices.
• Live Usage Dashboard on Databricks: Create an interactive dashboard on Databricks to monitor live product usage. Deep dive into observability and data quality monitoring techniques.

The best part? All code developed throughout the course is completely open-source and readily accessible for your own projects. Apply your newly acquired skills directly to your real-world data challenges.

Learn from a team of seasoned data engineers and data architects with extensive experience from industry leaders like Salesforce, Databricks, and Instagram. We'll provide step-by-step guidance to build these valuable use cases.