Courses Offered: SCJP SCWCD Design patterns EJB CORE JAVA AJAX Adv. Java XML STRUTS Web services SPRING HIBERNATE  

       

HADOOP Course Details
 

Subcribe and Access : 5200+ FREE Videos and 21+ Subjects Like CRT, SoftSkills, JAVA, Hadoop, Microsoft .NET, Testing Tools etc..

Batch Date: Oct 27th @ 10:30AM

Faculty: Real Time
Expert

Duration: 45 Days

Venue :
DURGA SOFTWARE SOLUTIONS at Maitrivanam
Plot No : 202, IInd Floor ,
HUDA Maitrivanam,
Ameerpet, Hyderabad-500038.

Ph.No: +91 - 9246212143, 80 96 96 96 96



Syllabus:

HADOOP

1. INTRODUCTION

• What is Big Data?
• What is Hadoop?
• Need of Hadoop
• Sources and Types of Data
• Comparison with Other Technologies
• Challenges with Big Data

i. Storage
ii. Processing

• RDBMS vs Hadoop
• Advantages of Hadoop
• Hadoop Echo System components
• RDBMS vs Hadoop
• Advantages of Hadoop
• Hadoop Echo System components

2. HDFS (Hadoop Distributed File System)

• Features of HDFS
• Name node ,Data node ,Blocks
• Configuring Block size,
• HDFS Architecture( 5 Daemons)

i. Name Node
ii. Data Node
iii. Secondary Name node
iv. Job Tracker
v. Task Tracker

• Metadata management
• Storage and processing
• Replication in Hadoop
• Configuring Custom Replication
• Fault Tolerance in Hadoop
• HDFS Commands

3. MAP REDUCE

• Map Reduce Architecture
• Processing Daemons of Hadoop

>> Job Tracker (Roles and Responsibilities)
>> Task Tracker(Roles and Responsibilities)

Phases of Map Reduce

i. Mapper phase
ii. Reducer phase

• Input split
• Input split vs Block size
• Partitioner in Map Reduce
• Groupings and Aggregations
• Data Types in Map Reduce

Map Reduce Programming Model

>> Driver Code
>> Mapper Code
>> Reducer Code

• Programming examples
• File input formats
• File output formats
• Merging in Map Reduce
• Speculative Execution Model
• Speculative Job

4. SQOOP (SQL + HADOOP)

• Introduction to Sqoop
• SQOOP Import
• SQOOP Export
• Importing Data From RDBMS to HDFS
• Importing Data From RDBMS to HIVE
• Importing Data From RDBMS to HBASE
• Exporting From HASE to RDBMS
• Exporting From HBASE to RDBMS
• Exporting From HIVE to RDBMS
• Exporting From HDFS to RDBMS
• Transformations While Importing / Exporting
• Filtering data while importing
• Vertical and Horizontal merging while import
• Working with delimiters while importing
• Groupings and Aggregations while import
• Incremental import
• Examples and operations
• Defining SQOOP Jobs

5. YARN

• Introduction
• Speculative Execution ,Speculative job and Speculative Task.
• Comparision of Hadoop1.xx with Hadoop2.xx
• Comparision with previous versions
• YARN Architecture Componets

i. Resource Manager
ii. Application Master
iii. Node Manager
iv. Application Manager
v. Resource Scheduler
vi. Job History Server
vii. Container

6. PIG

• Introduction to pig
• Pig Advantages
• Pig Latin Script
• Pig Terminologies
• Pig Relations, Bags, Tuples, Fields
• Pig Console / Grunt Shell
• Pig Execution Modes
• Pig Data Types
• Nulls
• Constants
• Expressions
• Schemas
• Parameter Substitution
• Arithmetic Operators
• Comparison Operators
• Null Operators
• Boolean Operators
• Sign Operators
• Flatten Operators
• Transformations in Pig
• Generating New Fields
• Type cating
• Filter Transformation
• Eliminating nulls and duplicates
• Data Merging
• UNION, JOINS in pig

Relational Operators in Pig

• COGROUP
• CROSS
• DISTINCT
• FILTER
• FOREACH
• GROUP
• JOIN (INNER)
• JOIN (OUTER)
• LIMIT
• LOAD
• ORDER
• SAMPLE
• SPILT
• STORE
• UNION

Diagnostic Operators in Pig

• Describe
• Dump
• Explain
• Illustrate

Eval Functions in Pig

• AVG
• CONCAT
• COUNT
• DIFF
• IS EMPTY
• MAX
• MIN
• SIZE
• SUM
• TOKENIZE
• Submitting Pig Scripts
• writing Custom UDFS in Pig

7. HIVE

• Introduction
• Hive Architecture
• Hive Metastore
• Hive Query Launguage
• Difference between HQL and SQL
• Hive Built in Functions
• Loading Data From Local Files To Hive Tables
• Loading Data FromHdfs Files To Hive Tables
• Tables Types
• Inner Tables
• External Tables
• Hive Working with unstructured data
• Hive Working With Xml Data
• Hive Working With Json Data
• Hive Working WithUrls And Weblog Data
• Hive Unions
• Hive Joins
• Multi Table / File Inserts
• Inserting Into Local Files
• Inserting IntoHdfs Files
• Hive UDF (user defined functions)
• Hive UDAF (user defined Aggregated functions)
• Hive UDTF (user defined table Generated functions
• Partitioned Tables
• Non – Partitioned Tables
• Multi-column Partitioning
• Dynamic Partitions In Hive
• Performance Tuning mechanism
• Bucketing in hive
• Indexing in Hive
• Hive Examples
• Hive &Hbase Integration

8. NOSQL

• What is “Not only SQL”
• NOSQL Advantages
• What is problem with RDBMS for Large
• Data Scaling Systems
• Types of NOSQL & Purposes
• Key Value Store
• Columer Store
• Document Store
• Graph Store
• Introduction to cassandra – NOSQL Database
• Introduction to MongoDB and CouchDB Database
• Intergration of NOSQL Databases with Hadoop

9. HBASE

• Introduction to big table
• What is NOSQL and colummer store Database
• HBASE Introduction
• Hbase use cases
• Hbase basics
• Column families
• Scans
• Hbase Architecture
• Map Reduce Over Hbase
• Hbase data Modeling
• Hbase Schema design
• Hbase CRUD operators
• Hive &Hbaseinteragation
• Hbase storage handlers

10. FLUME

• Introduction to FLUME
• What is the streaming data
• Big Data Applications
• Various streaming Technologies
• FLUME Architecture
• FLUME Nodes & FLUME Manager
• FLUME Local & Physical Node
• FLUME Agents & FLUME Collector
• Disadvantages of Flume-old
• FLUME new Generation
• Flume-ng Architecture
• Sources,Channels and Sinks
• Agent Chaining
• Fault Tolerance Mechanism
• Advantages of Flume-ng
• Interceptors and Channel selectors
• Configurations
• Hands on Examples

11. KAFKA

• Introduction to KAFKA
• Advanages of KAFKA
• KAFKA Architecture
• Kafka components
• BROKER
• Topics
• Producers
• Consumers
• Single-Node single Broker Architecture
• Single-Node multi Broker Architecture
• Multi-Node Multi Broker Architecture
• Acknowledgement options
• Configurations
• Hands on Examples