Training and Placement in Big Data and Hadoop

Big Data and Hadoop Course Outline

The Big Data and Hadoop training is designed to provide skills, knack and knowledge to become the finest and expertise Hadoop developer. It provides the learner with in-depth and thorough knowledge of concepts such as HDFS, Hadoop Architecture, HIVE, Flume, HBase, Map-Reduce, Oozie, Big Insights, Sqoop, PIG, Impala, JAQL, Administering Hadoop Cluster etc. At the end of the training the learner would get prone to practical exposure and could gain hands-on experience in the topics covered.

1. Understanding Big Data and Hadoop

Learning Objectives - In this module, learner will understand the Big Data, how Hadoop could solve the Big Data problems, the limitations, Hadoop Architecture, the common Hadoop ecosystem components, Anatomy of File, HDFS, Rack Awareness, Write and Read etc..
Topics

• Big Data
• Limitations and Solutions of existing Data Analytics Architecture
• Hadoop
• Hadoop Features
• Hadoop Ecosystem
• Hadoop 2.x core components
• Hadoop Storage: HDFS
• Hadoop Processing: MapReduce Framework,
• Anatomy of File Write and Read,
• Rack Awareness.

2. Hadoop Architecture and HDFS

Learning Objectives - In this module, the learner will come across the Hadoop Cluster Architecture, Data Loading Techniques and Important Configuration files in the Hadoop Cluster,
Topics

• Hadoop 2.x Cluster Architecture
• Hadoop Cluster Modes,
• Federation and High Availability
• Hadoop 2.x Configuration Files
• A Typical Production Hadoop Cluster
• MapReduce Job Execution
• Data Loading Techniques: Hadoop Copy Commands
• Common Hadoop Shell Commands
• Password-Less SSH
• FLUME
• SQOOP.

3. Hadoop MapReduce Framework - I

Learning Objectives - In this module, the learner will understand the Hadoop MapReduce framework where the data is been stored in HDFS and even the YARN concepts in MapReduce.
Topics

• Traditional way Vs MapReduce way
• Why MapReduce
• MapReduce Use Cases
• Hadoop 2.x (MapReduce )Architecture
• Hadoop 2.x (MapReduce ) Components
• YARN MR Application Execution Flow
• YARN Workflow
• Demo on MapReduce.
• Anatomy of MapReduce Program

4. Hadoop MapReduce Framework - II

Learning Objectives - In this section learner will understand the concepts Combiner & Partitioner, Input Splits in MapReduce, Demos on MapReduce by using different data sets.
Topics

• Input Splits
• Relation between HDFS Blocks and Input Splits
• Submission Flow of Map Reduce Job
• Demo of Input Splits
• Map Reduce:
• Combiner & Partitioner,
• Demo on de-identifying the Health Care Data set
• Demo on Weather Data set.

5. Advance Map Reduce

Learning Objectives - In this module, the learner do understand the Advance MapReduce concepts such as Distributed Cache, Counters, MR unit, Custom Input Format, Reduce Join, Sequence Input Format and to deal with the complex Map Reduce programs.
Topics

• Distributed Cache
• Counters,
• MRunit
• Custom Input Format
• Reduce Join
• Sequence Input Format.

6. Pig

Learning Objectives - In this module, the learner would learn Pig, types of use cases - Pig, tight coupling between Map Reduce & Pig and the Pig Latin scripting.
Topics

• About Pig,
• MapReduce Vs Pig,
• Pig Use Cases,
• Pig components,
• Programming Structure in Pig,
• Pig Execution,
• Pig Running Modes,
• Data Models in Pig,
• Pig Data Types.
• Pig Latin Program,
• Pig Latin
• Relational Operators
• Group Operator
• File Loaders
• COGROUP Operator
• Union
• Joins and COGROUP
• Diagnostic Operators
• Pig UDF
• Pig Demo - Healthcare Data set.

7. Hive

Learning Objectives - This module helps in understanding the Hive concepts, Loading, Querying the Data in Hive and Hive UDF.
Topics

• About Hive
• Hive Architecture and Components
• Hive Data Types and Data Models
• Hive Use Case
• Hive Background
• Metastore in Hive
• Limitations of Hive
• Hive Vs Pig
• Partitions and Buckets
• Comparison with Traditional Database
• Hive Tables (Managed Tables and External Tables)
• Querying Data
• Importing Data
• Managing Outputs
• Hive UDF
• Hive Script
• Hive Demo -Healthcare Data set.

8. Advance Hive and HBase

Learning Objectives - In this module, the learner will understand the Advance Hive concepts such as dynamic Partitioning and UDF, what is HBase, Hbase Architecture and all its components.
Topics

• Joining Tables
• Custom Map/Reduce Scripts,
• Dynamic Partitioning
• Hive:
• User Defined Functions.
• Thrift Server,
• HBase:
• Introduction to NoSQL Databases HBase,
• HBase v/s RDBMS
• HBase Cluster Deployment.
• HBase Components
• HBase Architecture

9. Advanced HBase

Learning Objectives - This module helps to cover the advanced HBase concepts, Filters and Bulk Loading. And also what the Zookeeper is all about and how it helps to monitor the cluster etc...
Topics:

• HBase Data Model
• HBase Client API
• HBase Shell
• ZooKeeper Data Model
• Zookeeper
• Zookeeper Service
• Data Loading Techniques
• Getting and Inserting Data
• Demos on Bulk Loading
• Filters in HBase.

10. Oozie and Hadoop Project

Learning Objectives: In this module, the learner could understand the working of multiple Hadoop ecosystem components altogether in the implementation of Hadoop to solve Big Data problems. One could discuss the multiple data sets and its project significance. This module also helps to cover Sqoop and Flume demo and even the Apache Oozie Workflow Scheduler - Hadoop Jobs.
Topics

• Flume and Sqoop Demo
• Oozie
• Oozie Workflow
• Scheduling with Oozie
• Demo on Oozie Workflow
• Oozie Commands
• Hadoop Project Demo.
• Oozie Components
• Oozie Co-ordinator
• Oozie Web Console