Subcribe and Access : 5200+ FREE Videos and 21+ Subjects Like CRT, SoftSkills, JAVA, Hadoop, Microsoft .NET, Testing Tools etc..
Batch
Date: Nov 6th @6:30AM
Faculty: Mr. Sameer (10+ Yrs of Exp,.. & Real Time Expert)
(Leading Faculty in Twin Cities)
Duration : 2 Months
Venue
:
DURGA SOFTWARE SOLUTIONS,
Flat No : 202,
2nd Floor,
HUDA Maitrivanam,
Ameerpet, Hyderabad - 500038
Ph.No: +91 - 9246212143, 80 96 96 96 96
Syllabus:
ADB with PYSPARK
Module 1: Cloud Computing Concepts
- What is the "Cloud" ?
- Why cloud services
- Types of cloud models
- Deployment Models
- private Cloud deployment model
- public Cloud deployment model
- hybrid cloud deployment model
- Microsoft Azure,
- Amazon Web Services,
- Google Cloud Platform
- characteristics of cloud computing
- On-demand self-service
- Broad network access
- Multi-tenancy and resource pooling
- Rapid elasticity and scalability
- Measured service
- Cloud Data Warehouse Architecture
- Shared Memory architecture
- Shared Disk architecture
- Shared Nothing architecture
Module 2: Core Azure Services
- Core Azure Architectural components
- Core Azure Services and Products
- Azure solutions
- Azure management tools
Module 3: Security, Privacy, Compliance
- Securing network connectivity
- Core Azure identity services
- Security tools and features
- Azure Governance methodologies
- Monitoring and reportings
- Privacy, compliance, and data protection standards
Module 4: Azure Pricing and Support
- Azure subscriptions
- Planning and managing costs
- Azure support options
- Azure Service Level Agreements (SLAs)
- Service Lifecycle in Azure
Module 5: Introduction to Azure Databricks
- Introduction to Databricks
- Azure Databricks Architecture
- Azure Databricks Main Concepts
Module 6: Azure Databricks Account Creation
- Azure Free Account
- Free Subscription for Azure Databricks
- Create Databricks Community Edition Account
Module 7: Databricks Cluster Types and Notebook Options
- Creating and configuring clusters
- create Notebook
- quick tour on notebook options
Module 8: Databricks Utilities and Notebook Parameters
- Dbutils commands on files, directories
- Notebooks and libraries
- Databricks Variables
- Widget Types
- Databricks notebook parameters
Module 9: Databricks CLI
- Azure Databricks CLI Installation
- Databricks CLI - DBFS, Libraries and Jobs
Module 10: Databricks Integration with Azure Blob Storage
- Read data from Blob Storage and Creating Blob mount point
Module 11: Databricks Integration with Azure Data Lake Storage Gen2
- Reading files from Azure Data Lake Storage Gen2
Module 12: Databricks Integration with Azure Data Lake Storage Gen1
- Reading Files from data lake storage Gen1
Module 13: Reading and Writing CSV files in Databricks
- Read CSV Files
- Read TSV Files and PIPE Seperated CSV Files
- Read CSV Files with multiple delimiter in spark 2 and spark 3
- Reading different position Multidelimiter CSV files
Module 14: Reading and Writing Parquet files in Databricks
- Read Parquet files from Data Lake Storage Gen2
- Reading and Creating Partition files in Spark
Module 16: Parsing Complex Json FilesL
- Reading and Writing JSON Files
- Reading, Transforming and Writing Complex JSON files
Module 17: Reading and Writing ORC and Avro Files
- Reading and Writing ORC and Avro Files
Module 19: Databricks Integration with Azure Synapse
- Reading and Writing Azure Synapse data from Azure Databricks
Module 20: Databricks Integration with Amazon Redshift(Redshift)
- Read and Write data from Redshift using databricks
Module 21: Databricks Integration with Snowflake
- Reading and Writing data from Snowflake
Module 22: Databricks Integration with CosmosDB SQL API
- Reading and Writing data from Azure CosmosDB Account
Module 23: Python Introduction
- Python Introduction
- Installation and setup
- Python Data Types for Azure Databricks
Module 24: Python Data Types
- Deep dive into String Data Types in Python for Azure Databricks
- Deep dive into python collection list and tuple
- Deep dive on set and dict data types in python
Module 25: Python Functions and Arguments
- Python Functions and Arguments
- Lambda Functions
Module 26: Python Modules and Packages
- Python Modules and Packages
Module 27: Python Flow Control
- Python Flow Control
- For-Each
- While
Module 28: Python File Handling
Module 29: Python Logging Module
Module 30: Python Exception Handling
- Python Exception Handlings
Module 31: Pyspark Introduction
- Pyspark Introduction
- Pyspark Components and Features
Module 32: Spark Architecture and Internals
- Apache Spark Internal architecture
- jobs stages and tasks
- Spark Cluster Architecture Explained
Module 33: Spark RDD
- Different Ways to create RDD in Databricks
- Spark Lazy Evaluation Internals & Word Count Program
- RDD Transformations in Databricks & coalesce vs repartition
- RDD Transformation and Use Cases
Module 34: Spark SQL
- Spark SQL Introduction
- Different ways to create DataFrames
Module 35: Spark SQL Intenals
- Catalyst Optimizer and Spark SQL Execution Plan
- Deep dive on Sparksession vs sparkcontext
- spark SQL Basics part-1
- RDD Transformation and Use Cases
Module 36: Spark SQL Basics
- Spark SQL Basics Part-2
- Joins in Spark SQL
Module 37: Spark SQL Functions and UDFs
- Spark SQL Functions part-1
- Spark SQL Functions part-2
- Spark SQL Functions Part-3
- Spark SQL UDFs
- Spark SQL Temp tables and Joins
Module 38: Databricks Delta and Implementing Dimensions SCD1 and SCD2
- Implementing SCD Type1 and Apache Spark Databricks Delta
- Delta Lake in Azure Databricks
- Implementing SCD Type with and without Databricks Delta
Module 39: Databricks Integration with Azure Data Factory
- Azure Data Factory Integration with Azure Databricks
Module 40: Databricks Streaming
- Delta Streaming in Azure Databricks
- Data Ingestion with Auto Loader in Azure Databricks
Module 41: Azure Databricks Projects
- Azure Databricks Project-1
- Azure Databricks Project-2
Module 42: Databricks Integration with Azure Devops
- Azure Databricks CICD Pipelines
