Private: HDP Certified Spark Developer (HDPCSD) – Scala

Overview of Big Data eco system
8 Topics
Overview about Ambari and HDFS
HDFS Architecture and Commands
Details about HDFS Commands
Overview about YARN
Yet Another Resource Negotiator (YARN)
YARN Scheduling and Hive Overview
Overview of Spark using Scala and Python
Overview of Pig and Data ingestion tools
Basics of programming using Scala
10 Topics
Getting Started – Scala
Basic Programming Constructs – Scala
Developing Functions and Arguments
Functions and Standard Operators
Functions Continued and Overview Of Collections
Processing Collections using Loops
Processing Collections using Loops and Map Reduce
Processing Collections using Map Reduce APIs
Developing Scala applications using IDE
Externalizing Properties in Scala Applications
Apache Spark 2 – Core APIs
5 Topics
Write a Spark Core application in Scala
Initialize a Spark application
Quick Recap of Scala
Spark 2.3.x Overview
Spark Development and Execution life cycle
Apache Spark 2 – Data Frames and Spark SQL
8 Topics
Spark Data Frames and Data Sets – Getting Started
Spark Data Frames and Data Sets – Overview of APIs
Data Frame Operations – Select Clause and Functions
Data Frame Operations – Basic Transformations
Data Frame Operations – Analytic Functions
Data Frames – Integration with Hive and JDBC
Spark SQL – Basic Transformations
Spark SQL – Analytics or Windowing Functions
Apache Spark 2 – Streaming
2 Topics
Spark Streaming (legacy) – Overview
Spark Structured Streaming – Overview
Revision for Exam
Next Lesson

Overview of Big Data eco system

Private: HDP Certified Spark Developer (HDPCSD) – Scala Overview of Big Data eco system

As part of this lesson, we will be primarily focusing on getting started with Hadoop and Spark.

  • Overview of Big Data cluster
  • HDFS – Hadoop Distributed File System
  • Detailed HDFS Commands (Self Paced)
  • YARN, Map Reduce and Spark Overview

Map Reduce and Spark are two alternative Distributed Processing Frameworks. We will be covering Spark in detail.

Live Sessions for this Lesson:

Lesson Content
0% Complete 0/8 Steps
Back to Course
Next Lesson