Data Engineering on Microsoft Azure Cloud Platform

Course Overview: This hands-on course provides in-depth training in data engineering using Microsoft Azure. You'll master key tools and technologies such as SQL, Python, Azure Data Factory, Azure Databricks, Spark, and PySpark. The course is designed around project-driven learning, ensuring you are well-prepared for real-world data engineering roles.

Course Structure:

• Duration:
  1.5-hour sessions on Saturdays and Sundays
  Weekly Doubt-Clearing Session: 1.5 hours on Fridays

• Total Sessions:
  14 sessions + 1.5-hour weekly doubt-clearing session on Fridays

• Project-Based Learning:
  Each session includes practical, project-driven activities to build your skills.

Modules:

1. SQL & Python (Basic to Advanced) – 4 Sessions

   • Learn foundational and advanced SQL queries.
   • Develop Python programming skills tailored for data engineering.

2. Azure & Big Data Services – 1 Session

   • Introduction to Azure and key Big Data services:
     • Azure Data Lake
     • Azure Data Factory
     • Azure Databricks

3. Azure DevOps Setup – 1 Session

   • Overview of Azure DevOps tools for CI/CD and project management.

4. Azure Data Factory (ADF) – 2 Sessions

   • Learn how to set up ADF pipelines, data flows, and triggers for ETL tasks.

5. Azure Databricks – 2 Sessions

   • Set up and use Databricks for advanced data processing and analytics.

6. Spark Architecture & APIs – 1 Session

   • Understand Spark architecture and APIs for large-scale data processing.

7. PySpark – 4 Sessions

   • Work with PySpark, including RDDs, DataFrames, and optimization techniques.

8. Final Project & Interview Readiness – 2 Sessions

   1. Complete a capstone project applying all concepts learned.
   2. Prepare for data engineering interviews with tips, best practices, and mock interviews.

Additional Details:

• Doubt-Clearing Sessions: Weekly 1.5-hour sessions on Fridays for addressing student questions and clarifying concepts.

• Azure Subscription: Students will need their own Azure subscription for hands-on labs (costs are borne by the student).

• Project-Driven Learning: From the second session onward, all activities are focused on practical, real-world projects to ensure skill-building.

Certification: Upon successful completion of the course, students will receive a Certificate of Completion, validating their Azure data engineering skills.