DP-203: Data Engineering on Microsoft Azure

Curriculum For This Course

Introduction

6Lectures

Time 00:25:00

Design and implement data storage - Basics

17Lectures

Time 01:52:00

Design and implement data storage - Overview on Transact-SQL

12Lectures

Time 00:34:00

Design and implement data storage - Azure Synapse Analytics

53Lectures

Time 04:42:00

Design and Develop Data Processing - Azure Data Factory

36Lectures

Time 04:23:00

Design and Develop Data Processing - Azure Event Hubs and Stream Analytics

27Lectures

Time 02:58:00

Design and Develop Data Processing - Scala, Notebooks and Spark

30Lectures

Time 01:59:00

Design and Develop Data Processing - Azure Databricks

34Lectures

Time 02:27:00

Design and Implement Data Security

20Lectures

Time 01:49:00

Monitor and optimize data storage and data processing

27Lectures

Time 02:11:00

Introduction
	1. IMPORTANT - How we are going to approach the exam objectives	3:00
	2. OPTIONAL - Overview of Azure	2:00
	3. OPTIONAL - Concepts in Azure	4:00
	4. Azure Free Account	5:00
	5. Creating an Azure Free Account	5:00
	6. OPTIONAL - Quick tour of the Azure Portal	6:00

Design and implement data storage - Basics
	1. Section Introduction	2:00
	2. Understanding data	4:00
	3. Example of data storage	2:00
	4. Lab - Azure Storage accounts	6:00
	5. Lab - Azure SQL databases	15:00
	6. A quick note when it comes to the Azure Free Account	4:00
	7. Lab - Application connecting to Azure Storage and SQL database	11:00
	8. Different file formats	7:00
	9. Azure Data Lake Gen-2 storage accounts	3:00
	10. Lab - Creating an Azure Data Lake Gen-2 storage account	9:00
	11. Using PowerBI to view your data	7:00
	12. Lab - Authorizing to Azure Data Lake Gen 2 - Access Keys - Storage Explorer	6:00
	13. Lab - Authorizing to Azure Data Lake Gen 2 - Shared Access Signatures	8:00
	14. Azure Storage Account - Redundancy	11:00
	15. Azure Storage Account - Access tiers	9:00
	16. Azure Storage Account - Lifecycle policy	3:00
	17. Note on Costing	5:00

Design and implement data storage - Overview on Transact-SQL
	1. Section Introduction	2:00
	2. The internals of a database engine	4:00
	3. Lab - Setting up a new Azure SQL database	3:00
	4. Lab - T-SQL - SELECT clause	3:00
	5. Lab - T-SQL - WHERE clause	3:00
	6. Lab - T-SQL - ORDER BY clause	1:00
	7. Lab - T-SQL - Aggregate Functions	1:00
	8. Lab - T-SQL - GROUP BY clause	4:00
	9. Lab - T-SQL - HAVING clause	1:00
	10. Quick Review on Primary and Foreign Keys	4:00
	11. Lab - T-SQL - Creating Tables with Keys	3:00
	12. Lab - T-SQL - Table Joins	5:00

Design and implement data storage - Azure Synapse Analytics
	1. Section Introduction	2:00
	2. Why do we need a data warehouse	10:00
	3. Welcome to Azure Synapse Analytics	2:00
	4. Lab - Let's create a Azure Synapse workspace	3:00
	5. Azure Synapse - Compute options	3:00
	6. Using External tables	4:00
	7. Lab - Using External tables - Part 1	9:00
	8. Lab - Using External tables - Part 2	12:00
	9. Lab - Creating a SQL pool	7:00
	10. Lab - SQL Pool - External Tables - CSV	9:00
	11. Data Cleansing	4:00
	12. Lab - SQL Pool - External Tables - CSV with formatted data	3:00
	13. Lab - SQL Pool - External Tables - Parquet - Part 1	4:00
	14. Lab - SQL Pool - External Tables - Parquet - Part 2	7:00
	15. Loading data into the Dedicated SQL Pool	2:00
	16. Lab - Loading data into a table - COPY Command - CSV	11:00
	17. Lab - Loading data into a table - COPY Command - Parquet	3:00
	18. Pausing the Dedicated SQL pool	3:00
	19. Lab - Loading data using PolyBase	5:00
	20. Lab - BULK INSERT from Azure Synapse	6:00
	21. My own experience	6:00
	22. Designing a data warehouse	11:00
	23. More on dimension tables	5:00
	24. Lab - Building a data warehouse - Setting up the database	6:00
	25. Lab - Building a Fact Table	8:00
	26. Lab - Building a dimension table	6:00
	27. Lab - Transfer data to our SQL Pool	15:00
	28. Other points in the copy activity	2:00
	29. Lab - Using Power BI for Star Schema	6:00
	30. Understanding Azure Synapse Architecture	7:00
	31. Understanding table types	7:00
	32. Understanding Round-Robin tables	5:00
	33. Lab - Creating Hash-distributed Tables	5:00
	34. Note on creating replicated tables	1:00
	35. Designing your tables	4:00
	36. Designing tables - Review	4:00
	37. Lab - Example when using the right distributions for your tables	10:00
	38. Points on tables in Azure Synapse	2:00
	39. Lab - Windowing Functions	4:00
	40. Lab - Reading JSON files	5:00
	41. Lab - Surrogate keys for dimension tables	6:00
	42. Slowly Changing dimensions	4:00
	43. Type 3 Slowly Dimension dimension	2:00
	44. Creating a heap table	3:00
	45. Snowflake schema	1:00
	46. Lab - CASE statement	6:00
	47. Partitions in Azure Synapse	2:00
	48. Lab - Creating a table with partitions	11:00
	49. Lab - Switching partitions	7:00
	50. Indexes	6:00
	51. Quick Note - Modern Data Warehouse Architecture	2:00
	52. Quick Note on what we are taking forward to the next sections	2:00
	53. What about the Spark Pool	2:00

Design and Develop Data Processing - Azure Data Factory
	1. Section Introduction	1:00
	2. Extract, Transform and Load	2:00
	3. What is Azure Data Factory	5:00
	4. Starting with Azure Data Factory	2:00
	5. Lab - Azure Data Lake to Azure Synapse - Log.csv file	13:00
	6. Lab - Azure Data Lake to Azure Synapse - Parquet files	13:00
	7. Lab - The case with escape characters	8:00
	8. Review on what has been done so far	6:00
	9. Lab - Generating a Parquet file	5:00
	10. Lab - What about using a query for data transfer	6:00
	11. Deleting artefacts in Azure Data Factory	3:00
	12. Mapping Data Flow	5:00
	13. Lab - Mapping Data Flow - Fact Table	14:00
	14. Lab - Mapping Data Flow - Dimension Table - DimCustomer	15:00
	15. Lab - Mapping Data Flow - Dimension Table - DimProduct	10:00
	16. Lab - Surrogate Keys - Dimension tables	4:00
	17. Lab - Using Cache sink	9:00
	18. Lab - Handling Duplicate rows	8:00
	19. Note - What happens if we don't have any data in our DimProduct table	4:00
	20. Changing connection details	1:00
	21. Lab - Changing the Time column data in our Log.csv file	8:00
	22. Lab - Convert Parquet to JSON	5:00
	23. Lab - Loading JSON into SQL Pool	5:00
	24. Self-Hosted Integration Runtime	3:00
	25. Lab - Self-Hosted Runtime - Setting up nginx	9:00
	26. Lab - Self-Hosted Runtime - Setting up the runtime	7:00
	27. Lab - Self-Hosted Runtime - Copy Activity	7:00
	28. Lab - Self-Hosted Runtime - Mapping Data Flow	16:00
	29. Lab - Processing JSON Arrays	8:00
	30. Lab - Processing JSON Objects	6:00
	31. Lab - Conditional Split	6:00
	32. Lab - Schema Drift	12:00
	33. Lab - Metadata activity	14:00
	34. Lab - Azure DevOps - Git configuration	11:00
	35. Lab - Azure DevOps - Release configuration	11:00
	36. What resources are we taking forward	1:00

Design and Develop Data Processing - Azure Event Hubs and Stream Analytics
	1. Batch and Real-Time Processing	5:00
	2. What are Azure Event Hubs	5:00
	3. Lab - Creating an instance of Event hub	7:00
	4. Lab - Sending and Receiving Events	10:00
	5. What is Azure Stream Analytics	2:00
	6. Lab - Creating a Stream Analytics job	4:00
	7. Lab - Azure Stream Analytics - Defining the job	10:00
	8. Review on what we have seen so far	8:00
	9. Lab - Reading database diagnostic data - Setup	4:00
	10. Lab - Reading data from a JSON file - Setup	6:00
	11. Lab - Reading data from a JSON file - Implementation	5:00
	12. Lab - Reading data from the Event Hub - Setup	7:00
	13. Lab - Reading data from the Event Hub - Implementation	8:00
	14. Lab - Timing windows	10:00
	15. Lab - Adding multiple outputs	4:00
	16. Lab - Reference data	5:00
	17. Lab - OVER clause	8:00
	18. Lab - Power BI Output	10:00
	19. Lab - Reading Network Security Group Logs - Server Setup	3:00
	20. Lab - Reading Network Security Group Logs - Enabling NSG Flow Logs	8:00
	21. Lab - Reading Network Security Group Logs - Processing the data	13:00
	22. Lab - User Defined Functions	9:00
	23. Custom Serialization Formats	3:00
	24. Lab - Azure Event Hubs - Capture Feature	7:00
	25. Lab - Azure Data Factory - Incremental Data Copy	11:00
	26. Demo on Azure IoT Devkit	5:00
	27. What resources are we taking forward	1:00

Design and Develop Data Processing - Scala, Notebooks and Spark
	1. Section Introduction	2:00
	2. Introduction to Scala	2:00
	3. Installing Scala	6:00
	4. Scala - Playing with values	3:00
	5. Scala - Installing IntelliJ IDE	5:00
	6. Scala - If construct	3:00
	7. Scala - for construct	1:00
	8. Scala - while construct	1:00
	9. Scala - case construct	1:00
	10. Scala - Functions	2:00
	11. Scala - List collection	4:00
	12. Starting with Python	3:00
	13. Python - A simple program	2:00
	14. Python - If construct	1:00
	15. Python - while construct	1:00
	16. Python - List collection	2:00
	17. Python - Functions	2:00
	18. Quick look at Jupyter Notebook	4:00
	19. Lab - Azure Synapse - Creating a Spark pool	8:00
	20. Lab - Spark Pool - Starting out with Notebooks	9:00
	21. Lab - Spark Pool - Spark DataFrames	4:00
	22. Lab - Spark Pool - Sorting data	6:00
	23. Lab - Spark Pool - Load data	8:00
	24. Lab - Spark Pool - Removing NULL values	8:00
	25. Lab - Spark Pool - Using SQL statements	3:00
	26. Lab - Spark Pool - Write data to Azure Synapse	11:00
	27. Spark Pool - Combined Power	2:00
	28. Lab - Spark Pool - Sharing tables	4:00
	29. Lab - Spark Pool - Creating tables	5:00
	30. Lab - Spark Pool - JSON files	6:00

Design and Develop Data Processing - Azure Databricks
	1. What is Azure Databricks	4:00
	2. Clusters in Azure Databricks	6:00
	3. Lab - Creating a workspace	3:00
	4. Lab - Creating a cluster	14:00
	5. Lab - Simple notebook	3:00
	6. Lab - Using DataFrames	4:00
	7. Lab - Reading a CSV file	4:00
	8. Databricks File System	2:00
	9. Lab - The SQL Data Frame	3:00
	10. Visualizations	1:00
	11. Lab - Few functions on dates	2:00
	12. Lab - Filtering on NULL values	2:00
	13. Lab - Parquet-based files	2:00
	14. Lab - JSON-based files	3:00
	15. Lab - Structured Streaming - Let's first understand our data	3:00
	16. Lab - Structured Streaming - Streaming from Azure Event Hubs - Initial steps	8:00
	17. Lab - Structured Streaming - Streaming from Azure Event Hubs - Implementation	10:00
	18. Lab - Getting data from Azure Data Lake - Setup	7:00
	19. Lab - Getting data from Azure Data Lake - Implementation	5:00
	20. Lab - Writing data to Azure Synapse SQL Dedicated Pool	5:00
	21. Lab - Stream and write to Azure Synapse SQL Dedicated Pool	5:00
	22. Lab - Azure Data Lake Storage Credential Passthrough	10:00
	23. Lab - Running an automated job	6:00
	24. Autoscaling a cluster	2:00
	25. Lab - Removing duplicate rows	3:00
	26. Lab - Using the PIVOT command	4:00
	27. Lab - Azure Databricks Table	5:00
	28. Lab - Azure Data Factory - Running a notebook	6:00
	29. Delta Lake Introduction	2:00
	30. Lab - Creating a Delta Table	5:00
	31. Lab - Streaming data into the table	3:00
	32. Lab - Time Travel	2:00
	33. Quick note on the deciding between Azure Synapse and Azure Databricks	2:00
	34. What resources are we taking forward	1:00

Design and Implement Data Security
	1. Section Introduction	1:00
	2. What is the Azure Key Vault service	5:00
	3. Azure Data Factory - Encryption	5:00
	4. Azure Synapse - Customer Managed Keys	3:00
	5. Azure Dedicated SQL Pool - Transparent Data Encryption	2:00
	6. Lab - Azure Synapse - Data Masking	10:00
	7. Lab - Azure Synapse - Auditing	6:00
	8. Azure Synapse - Data Discovery and Classification	4:00
	9. Azure Synapse - Azure AD Authentication	3:00
	10. Lab - Azure Synapse - Azure AD Authentication - Setting the admin	4:00
	11. Lab - Azure Synapse - Azure AD Authentication - Creating a user	8:00
	12. Lab - Azure Synapse - Row-Level Security	7:00
	13. Lab - Azure Synapse - Column-Level Security	4:00
	14. Lab - Azure Data Lake - Role Based Access Control	7:00
	15. Lab - Azure Data Lake - Access Control Lists	7:00
	16. Lab - Azure Synapse - External Tables Authorization via Managed Identity	8:00
	17. Lab - Azure Synapse - External Tables Authorization via Azure AD Authentication	5:00
	18. Lab - Azure Synapse - Firewall	7:00
	19. Lab - Azure Data Lake - Virtual Network Service Endpoint	7:00
	20. Lab - Azure Data Lake - Managed Identity - Data Factory	6:00

Monitor and optimize data storage and data processing
	1. Best practices for structing files in your data lake	3:00
	2. Azure Storage accounts - Query acceleration	2:00
	3. View on Azure Monitor	7:00
	4. Azure Monitor - Alerts	8:00
	5. Azure Synapse - System Views	2:00
	6. Azure Synapse - Result set caching	6:00
	7. Azure Synapse - Workload Management	4:00
	8. Azure Synapse - Retention points	2:00
	9. Lab - Azure Data Factory - Monitoring	7:00
	10. Azure Data Factory - Monitoring - Alerts and Metrics	4:00
	11. Lab - Azure Data Factory - Annotations	3:00
	12. Azure Data Factory - Integration Runtime - Note	7:00
	13. Azure Data Factory - Pipeline Failures	3:00
	14. Azure Key Vault - High Availability	2:00
	15. Azure Stream Analytics - Metrics	3:00
	16. Azure Stream Analytics - Streaming Units	2:00
	17. Azure Stream Analytics - An example on monitoring the stream analytics job	11:00
	18. Azure Stream Analytics - The importance of time	7:00
	19. Azure Stream Analytics - More on the time aspect	6:00
	20. Azure Event Hubs and Stream Analytics - Partitions	5:00
	21. Azure Stream Analytics - An example on multiple partitions	7:00
	22. Azure Stream Analytics - More on partitions	4:00
	23. Azure Stream Analytics - An example on diagnosing errors	4:00
	24. Azure Stream Analytics - Diagnostics setting	6:00
	25. Azure Databricks - Monitoring	7:00
	26. Azure Databricks - Sending logs to Azure Monitor	3:00
	27. Azure Event Hubs - High Availability	6:00

Free DP-203 Exam Questions & Microsoft DP-203 Dumps
	Downloads: 99	Size: 2.59 MB
	Downloads: 173	Size: 2.59 MB
Microsoft.test4prep.dp-203.v2022-04-23.by.emil.64q.ete	Downloads: 1009	Size: 1.73 MB
Microsoft.examlabs.dp-203.v2021-05-04.by.alexander.36q.ete	Downloads: 1360	Size: 1.3 MB