Subcribe and Access : 5200+ FREE Videos and 21+ Subjects Like CRT, SoftSkills, JAVA, Hadoop, Microsoft .NET, Testing Tools etc..

Batch Date: May 20th @7:00AM

Faculty: Real Time Expert

Duration: 1 Month

Venue :
DURGA SOFTWARE SOLUTIONS at Maitrivanam
Plot No : 202, IInd Floor ,
HUDA Maitrivanam,
Ameerpet, Hyderabad-500038.

Ph.No: +91- 9246212143, 80 96 96 96 96

Syllabus:

AZURE Data Engineering

Azure Basics

1. Introduction to Cloud Computing
2. IaaS Vs PaaS vs SaaS and Serverless
3. Understand Azure Management Portal

Data Engineer – Roles & Responsibilities

1. Overview of the Roles and Responsibilities of Data Engineers
2. Role of DevOps for automation of Data and other components
3. Understand the Data Engineering Processes

Introduction to Azure Data Services

1. Relation Databases
2. NoSQL Databases
3. Storage Services
4. ETL Services
5. Big Data Services
6. Stream Analytics

Database in the Cloud

1. Introduction to various options for hosting SQL server on Azure
2. Create Logical SQL Server and Sql Database
3. Learn about various Pricing Tiers (DTU vs vCores)
4. Learn about Backup & Restore methods (Point-in-Time and Long Term Backup Retention)
5. Understand how to scale-up and scale-down the capacity of SQL Database
6. Configure Geo-Replication for disaster recovery.
7. Configure Firewall to whitelist the required IP Address for secure access at both Server level and Database Level.
8. Manage Sensitive Data with Dynamic Data Masking and Data Encryption

Azure Storage

1. Create a Storage Account
2. Create Containers and Blobs
3. Install & Explore Azure Storage Explorer
4. Create SAS tokens and understand the benefits of SAS tokens

Azure Data Lake Gen 2

1. Overview of Azure Data Lake Service ( ADLS ) Gen2
2. Comparison with Azure Storage Account.
3. Quick History of ADLS Service
4. Lab: Creating an Azure Data Lake Store Gen2 with Portal
5. Convert Raw JSON files into Parquet files and store into ADLS
6. Understand Modern Data Analytics Solution Architecture
7. Lab: Azure Data Factory integration with ADLS

Azure NoSQL – Cosmos DB

1. Overview of Cosmos DB
2. Understand the Cosmos DB architecture & Hierarchy
3. Create Cosmos DB Account & Database
4. Create Cosmos DB Containers and explore Partitions & Indexes
5. Working with Cosmos DB Capacity (Request Units) and Cost Estimations
6. Learn how to query data from Cosmos DB

Azure – Deployment as Code using Bicep

1. Overview of Bicep
2. Benefits of Bicep
3. Provision Azure Resources using Bicep
4. Integrate Bicep Scripts to Azure DevOps

Azure Data Factory

• Overview of Azure Data Factory
  • Real-time use cases of Azure Data Factory
  • Lab – Create Azure Data Factory Instance
  • Azure Data Factory Core Concepts (Pipelines, Linked Services, Datasets, Activities, Triggers, Integration Runtime (Azure, Self-Hosted & SSIS IR)
  • Lab – A tour of Azure Data Factory User Experience & Interface (Pipeline UI Components, Data Flow UI Components, Monitor UI components, Debug Components, Trigger UI Components, Management Hub)
  • Quiz
• Explore ADF
  • Lab – Create the first ADF Pipeline
  • Understand the Real-Time Use Case for a Fictions Company and it’s Data
  • Lab – Copy data from Storage Account to Azure SQL Database
  • Lab – Learn different ways of Executing a Stored Procedure (Lookup & Stored Procedure)
  • Lab – Filter the data using Filter Activity
  • Lab – Retrieve the Storage Blobs using GetMetaData Activity
  • Lab – Loop Activities – Iterate through the results of executing a Stored Procedure
  • Lab – Set Variable – Learn how to create, configure and use the Variables in the Pipeline
  • Lab – If Condition Activity – Process records based on a Flag
  • Lab – Fail – Learn how to throw an exception in the Pipeline intentionally
  • Lab – Logic Apps – Send Email using Logic App with Outlook action
  • Lab – Logic Apps – Validate the Logic App Send mail Functionality
  • Lab – Web Activity – Integrate and Invoke Logic Apps for sending E-mails
  • Lab – Azure Data Factory – Learn how to dynamically pass values using Parameters
  • Lab – Execute Pipeline Activity – Learn how to invoke nested Pipeline
    Scheduled Jobs
  • Understand the differences between Tumbling Schedule and Schedule
  • Lab – Execute Data Factory Pipeline based on Blob Creation Event
  • Lab – Execute Data Factory Pipeline based on Custom Event
  • Lab – Perform Data Transformations using Data Flows
  • Quiz
• Data Transformations using Data Flows
  • Introduction to Data Flows & Power Query
  • Explore various types of Data Flow Transformations
  • Lab – Remove Nulls
  • Lab – Error Row Handling
• Data Transformations using Power Query
• Production Grade pipelines
  • Lab – Copy data from multiple files (stored in Storage Account) to Azure SQL Database
  • Lab- Optimize the performance of
  • Lab – Promote ADF Pipelines to multiple Environments

Azure Data Bricks

1. Overview of Azure Data Bricks
2. Create an Azure Data Bricks service.
3. Create a Spark cluster in Azure Data Bricks.
4. Learn transformation of data in Azure Data Bricks using Scala
5. Create Notebook to develop ETL using Data Bricks
6. Understand how to parameterize Data Bricks pipelines
7. Invoke Data Bricks pipelines with Azure Data Factory.

Azure Synapse Analytics

1. Overview of Synapse Analytics

• Traditional approach to Data Analytics
• Overview of Data Lakehouse
• Overview of Synapse Analytics
• Understand Synapse Analytics Components (Dedicated, Serverless, Spark Pools)
• Lab: Create Synapse Analytics Workspace
• Lab: Explore Synapse Components: Compute Options, Storage, RBAC Permissions
• Understand Synapse SQL Architecture – Control Node, Compute Node, Data Movement Service
• Quiz

2. Analyze data with SQL Serverless

• Overview of SQL Serverless Pools
• Lab: Working with Database & External Tables with SQL Serverless Pools
• Lab: Explore & Analyze the data from the SQL Serverless Pool
• Quiz

3. Analyze data with dedicated SQL pools

• Overview of SQL Dedicated Pools
• Lab: Create a dedicated SQL Pool
• Lab: Load the data into SQL Pool using COPY command
• Lab: Explore & Analyze the data from the SQL Pool
• Lab: Pause & Resume SQL Dedicated Pool
• Lab: Load the data into SQL Pool using Polybase
• Quiz

4. Analyze data with Apache Spark Pools

• Overview of Apache Spark Pools and it’s environment
• Lab: Create an Apache Spark Pool
• Lab: Load the data into Apache Spark Pool
• Lab: Explore & Analyze the data using Spark and Notebooks
• Quiz

5. Overview of Synapse Pipelines

• Overview of Synapse Pipelines
• Understand the components of the Synapse Pipelines
• Lab: Transform Data using mapping Data Flows
• Lab: Orchestrating, running and Monitor the Pipelines
• Quiz