Beskrivelse
Vi gennemgår praktisk implementering af data storage muligheder i Azure. Samtidig går vi i dybden med sikkerhed og værktøjet Azure Data Factory, som kan få alle tjenesterne til at tale sammen ved at migrere og transformere data igennem en pipeline. Kurset er bygget op omkring en case, hvor man afdækker storage behov samt krav og identificerer en mulig implementeringsløsning i Azure. Følgende Azure storage services og analyseværktøjer er blandt andre indeholdt i kurset:
- Til filbaseret ustruktureret data er Azure Storage account et godt valgt til blob storage/datalake.
- CosmosDB er Microsofts lynhurtige noSQL database med 5 forskellige APIer til semistruktureret data og med mulighed for global replikering.
- Klassisk relationel data kan gemmes i en Azure SQL database, og du lærer hvad der adskiller den fra en traditionel on-premise løsning.
- Azure Synapse Analytics (tidligere SQL DW) er et Parallelt DW i skyen, men som navnet antyder er analyse-delen nu dybt integreret i produktet.
- Live data kan sendes direkte til Azure Event Hub og analyseres med window funktioner i Stream Analytics.
...%%%...
- Via Azure Databricks (Apache Spark baseret analyse platform) sætter du et Spark cluster op og ser hvordan man kan analysere data fra datakilderne i en notebook.
- Azure Data Factory er værktøjet som kan få alle tjenesterne til at tale sammen ved at migrere og transformere data igennem en pipeline.
Indhold
Module 1: Azure for the Data Engineer
- This module explores how the world of data has evolved and how cloud data platform technologies are providing new opportunities for business to explore their data in different ways. The student will gain an overview of the various data platform technologies that are available, and how a Data Engineers role and responsibilities has evolved to work in this new world to an organization benefit
- Lessons
- Explain the evolving world of data
- Survey the services in the Azure Data Platform
- Identify the tasks that are performed by a Data Engineer
- Describe the use cases for the cloud in a Case Study
- Lab : Azure for the Data Engineer
- Identify the evolving world of data
- Determine the Azure Data Platform Services
- Identify tasks to be performed by a Data Engineer
- Finalize the data engineering deliverables Module 2: Working with Data Storage
- This module teaches the variety of ways to store data in Azure. The Student will learn the basics of storage management in Azure, how to create a Storage Account, and how to choose the right model for the data you want to store in the cloud. They will also understand how data lake storage can be created to support a wide variety of big data analytics solutions with minimal effort.
- Lessons
- Choose a data storage approach in Azure
- Create an Azure Storage Account
- Explain Azure Data Lake storage
- Upload data into Azure Data Lake
- Lab : Working with Data Storage
- Choose a data storage approach in Azure
- Create a Storage Account
- Explain Data Lake Storage
- Upload data into Data Lake Store Module 3: Enabling Team Based Data Science with Azure Databricks
- This module introduces students to Azure Databricks and how a Data Engineer works with it to enable an organization to perform Team Data Science projects. They will learn the fundamentals of Azure Databricks and Apache Spark notebooks; how to provision the service and workspaces and learn how to perform data preparation task that can contribute to the data science project.
- Lessons
- Explain Azure Databricks
- Work with Azure Databricks
- Read data with Azure Databricks
- Perform transformations with Azure Databricks
- Lab : Enabling Team Based Data Science with Azure Databricks
- Explain Azure Databricks
- Work with Azure Databricks
- Read data with Azure Databricks
- Perform transformations with Azure Databricks Module 4: Building Globally Distributed Databases with Cosmos DB
- In this module, students will learn how to work with NoSQL data using Azure Cosmos DB. They will learn how to provision the service, and how they can load and interrogate data in the service using Visual Studio Code extensions, and the Azure Cosmos DB .NET Core SDK. They will also learn how to configure the availability options so that users are able to access the data from anywhere in the world.
- Lessons
- Create an Azure Cosmos DB database built to scale
- Insert and query data in your Azure Cosmos DB database
- Build a .NET Core app for Cosmos DB in Visual Studio Code
- Distribute your data globally with Azure Cosmos DB
- Lab : Building Globally Distributed Databases with Cosmos DB
- Create an Azure Cosmos DB
- Insert and query data in Azure Cosmos DB
- Build a .Net Core App for Azure Cosmos DB using VS Code
- Distribute data globally with Azure Cosmos DB Module 5: Working with Relational Data Stores in the Cloud
- In this module, students will explore the Azure relational data platform options including SQL Database and SQL Data Warehouse. The student will be able explain why they would choose one service over another, and how to provision, connect and manage each of the services.
- Lessons
- Use Azure SQL Database
- Describe Azure SQL Data Warehouse
- Creating and Querying an Azure SQL Data Warehouse
- Use PolyBase to Load Data into Azure SQL Data Warehouse
- Lab : Working with Relational Data Stores in the Cloud
- Use Azure SQL Database
- Describe Azure SQL Data Warehouse
- Creating and Querying an Azure SQL Data Warehouse
- Use PolyBase to Load Data into Azure SQL Data Warehouse Module 6: Performing Real-Time Analytics with Stream Analytics
- In this module, students will learn the concepts of event processing and streaming data and how this applies to Events Hubs and Azure Stream Analytics. The students will then set up a stream analytics job to stream data and learn how to query the incoming data to perform analysis of the data. Finally, you will learn how to manage and monitor running jobs.
- Lessons
- Explain data streams and event processing
- Data Ingestion with Event Hubs
- Processing Data with Stream Analytics Jobs
- Lab : Performing Real-Time Analytics with Stream Analytics
- Explain data streams and event processing
- Data Ingestion with Event Hubs
- Processing Data with Stream Analytics Jobs Module 7: Orchestrating Data Movement with Azure Data Factory
- In this module, students will learn how Azure Data factory can be used to orchestrate the data movement and transformation from a wide range of data platform technologies. They will be able to explain the capabilities of the technology and be able to set up an end to end data pipeline that ingests and transforms data.
- Lessons
- Explain how Azure Data Factory works
- Azure Data Factory Components
- Azure Data Factory and Databricks
- Lab : Orchestrating Data Movement with Azure Data Factory
- Explain how Data Factory Works
- Azure Data Factory Components
- Azure Data Factory and Databricks Module 8: Securing Azure Data Platforms
- In this module, students will learn how Azure provides a multi-layered security model to protect your data. The students will explore how security can range from setting up secure networks and access keys, to defining permission through to monitoring across a range of data stores.
- Lessons
- An introduction to security
- Key security components
- Securing Storage Accounts and Data Lake Storage
- Securing Data Stores
- Securing Streaming Data
- Lab : Securing Azure Data Platforms
- An introduction to security
- Key security components
- Securing Storage Accounts and Data Lake Storage
- Securing Data Stores
- Securing Streaming Data Module 9: Monitoring and Troubleshooting Data Storage and Processing
- In this module, the student will get an overview of the range of monitoring capabilities that are available to provide operational support should there be issue with a data platform architecture. They will explore the common data storage and data processing issues. Finally, disaster recovery options are revealed to ensure business continuity.
- Lessons
- Explain the monitoring capabilities that are available
- Troubleshoot common data storage issues
- Troubleshoot common data processing issues
- Manage disaster recovery
- Lab : Monitoring and Troubleshooting Data Storage and Processing
- Explain the monitoring capabilities that are available
- Troubleshoot common data storage issues
- Troubleshoot common data processing issues
- Manage disaster recovery
