Spark Monitoring and Tuning Overview/Description Target Audience Prerequisites Expected Duration Lesson Objectives Course Number Expertise Level Overview/Description
In this course, you will learn about various ways to monitor Spark applications such as web UIs, metrics, and other monitoring tools. You will also learn about memory tuning.
Target Audience
Programmers and Developers wishing to perform big data development using Apache Spark 2.2
Prerequisites
None
Expected Duration (hours)
1.0
Lesson Objectives Spark Monitoring and Tuning
start the course
access the web user interface
use the Spark environment configuration parameters
use JSON to query monitoring tools for Spark
set JVM fractional memory amounts for Spark
modify speculation controls for Spark tasks
describe data serialization and the role it plays in the performance of Spark applications
describe memory management and consumption
determine executor memory allocation
describe garbage collection tuning
set the level of parallelism
use the broadcast functionality
use query execution plan explainer
implement data compression on parquet storage
monitor Spark applications
Course Number: df_apsk_a04_it_enus
Expertise Level
Intermediate