Big Data For Security Analytics

Almost all of the major breaches in the past have shown that the SOC/SIEM did not do their job well and alert the client to the breach. What are SOC/SIEMs missing and why? And how do fix this problem? The search for the proverbial needle in the haystack has become a daunting task as the haystack is now phenomenally big. How can Big Data help determine security intelligence?

This training workshop will explore addressing this problem using big data analytics and leveraging Hadoop, ElasticSearch, Logstash, Kibana, and Beats along with a layer of Machine Learning on top of this data to determine active compromises in your network, possible fraud leads and visualize all this information for better perception by the human eye.

As a completely hands-on workshop participants will be taken through the steps of implementing ELK and Hadoop as well as installing the Apache Metron project that is built on the same principles. Logs from real-world scenarios will be shared and analytics run on these to see the real value that a Big Data setup can deliver.

Table of Contents:

• Day 1
• Session 1:
• Introduction to Big Data
• Generic use cases
• What it is? What it is not?
• Session 2:
• Demystification of jargon and products
• Hadoop
• MapReduce
• YARN
• HDFS
• Spark
• Pig
• Storm
• ELK
• ElasticSearch
• Logstash
• Kibana
• Beats
• Session 3:
• ELK quick install and demo
• Vulnerability Dashboard
• Session 4:
• Introduction
• Security Visualization & Analytics
• Active Threat Hunting
• Machine Learning
• Day 2
• Session 5:
• Hadoop fundamentals
• Session 6:
• Hadoop installation and small demo
• Session 7:
• ELK Concepts
• Index
• Documents
• Shards
• Filters
• Query
• Aggregations
• ELK Hands-on
• Full Installation steps on Linux
• Creating basic logstash file
• Grok Patterns
• Apache log
• Firewall
• Geo-IP
• Session 8:
• ELK Hands-on (contd…
• Windows Logon Events Monitoring
• Top logon fails
• Single user logon to multiple systems
• Day 3
• Session 9:
• Active Threat Hunting
• Concepts and Fundamentals
• Why traditional SIEM fails
• Why big-data
• Threat-Intel Translation configuration
• ELK based demo
• Ingesting logs using Python
• Creating VM dashboards
• Log Metrics using ELK
• Session 10:
• Security Visualization Concepts
• Outliers
• Good vs Bad visualization
• Lab exercises
• Finding anomaly using Kibana Visualization
• Session 11:
• Machine Learning
• Code development theory
• Demo use-case
• Session 12:
• Apache Metron introduction
• Background
• Features
• Demo