Big Data And Hadoop Development

Big data is a collection of large datasets that cannot be processed using traditional computing techniques. It is not a single technique or a tool, rather it has become a complete subject, which involves various tools, techniques and frameworks.

Requirements:

• Programming
• Quantitative Skills
• Multiple Technologies
• Understanding of Business & Outcomes
• Interpretation of Data

Content:

• Big Data & Hadoop Development
• Big Data and Hadoop
• Limitation of existing solution for Big data problem
• How hadoop Solves Big data problem
• Hadoop Eco-System Component
• Hadoop Architecture
• Hadoop Distributed File System
• Concept of Hadoop Distributed file system(HDFS)
• Design of HDFS
• Common challenges
• Best practices for scaling with your data
• Configuring HDFS
• Interacting with HDFS
• HDFS permission and Security
• Additional HDFS Tasks
• Data Flow (Anatomy of a File Read, Anatomy of a File Write, Coherency Model)
• Advance Map Reduce, YARN
• What is Map Reduce?
• Data Types used in Hadoop
• Concept of Mappers
• Concept of Reducers
• The Execution Framework architecture
• Concept of Partioners
• Concept of Combiners
• Hadoop Cluster Architecture
• MapReduce types
• Input Formats (Input Splits and Records, Text Input, Binary Input, Multiple Inputs)
• OutPut Formats (TextOutput, BinaryOutPut, Multiple Output)
• Writing Programs for MapReduce
• Hadoop Installation
• Installation of Hadoop
• Getting Started
• Running a sample program
• HDFS & Pseudo Cluster Environment
• Storage HDFS
• Name Node HA & Node Manager
• Cluster specification
• Hadoop Configuration (Environment Settings, Hadoop Daemon- Properties, Addresses and Ports)
• Basic Linux and HDFS Commands
• Setup a Hadoop Cluster
• What is PIG?
• Installing and Running Pig
• Grunt
• Pig’s Data Model
• Pig Latin
• Developing & Testing Pig Latin Scripts
• Writing Evaluation
• Filter
• Loads & Store Functions
• What is HIVE?
• What is HIVE ?
• Hive Architecture
• Running Hive
• Pig’s Data Model
• Comparison with Traditional Database (Schema on Read versus Write, Updates, Transactions and Indexes)
• HiveQL (Data Types, Operators and Functions)
• Tables (Managed and External Tables, Partitions and Buckets, Storage Formats, Importing Data)
• Altering Tables, Dropping Tables
• Querying Data (Sorting And Aggregating, Map Reduce Scripts, Joins & Subqueries & Views
• Map and Reduce site Join to optimize Query
• User Defined Functions
• Appending Data into existing Hive Table
• Custom Map/Reduce in Hive
• Perform Data Analytics using Pig and Hive
• What is HBASE?
• What is HBASE?
• Client API- Basics
• Client API- Advanced Features
• Client API – Administrative Features
• Available Client
• Architecture
• MapReduce Integration
• Advanced Usage
• Advanced Indexing
• Impelment HBASE
• What is SQOOP?
• What is SQOOP?
• Database Imports
• Importing Large Objects
• Performing Exports
• Exports- A Deeper look
• What is ZooKeeper?
• What is ZooKeeper?
• The Zookeeper Service (Data Modal, Operations, Implementation,Consistency, Sessions, States)
• Building Applications with Zookeeper (Zookeeper in Production)
• What is Oozie?
• What is Oozie?
• OOZIE Installation
• Running an OOZIE EXAMPLE
• OOZIE WEBCONSOLE
• Expression Language Funtions
• OOZIE WORKFLOW EXAMPLE(Java Code,PIG,Hive)
• Control Flow nodes
• Action Node Properties(Map Reduce,Hive,Pig,java)
• What is Ambari?
• What is Ambari?
• Why Ambari is needed?
• What is Work Labs?
• Hands on with examples
• Hadoop Admin Content
• Spark with Scala