Structured Streaming


Overview/Description
Target Audience
Prerequisites
Expected Duration
Lesson Objectives
Course Number
Expertise Level


Overview/Description
In this course, you will learn about the concepts of Structured Steaming such as Windowing, DataFrame, and SQL Operations. You will also learn about File Sinks, Deduplication, and Checkpointing.

Target Audience
Programmers and Developers wishing to perform big data development using Apache Spark 2.2

Prerequisites
None

Expected Duration (hours)
1.2

Lesson Objectives

Structured Streaming

  • start the course
  • describe Structured Streaming
  • read stream input using readStream
  • write stream data using writeStream
  • apply window operations on event time
  • describe continuous applications in terms of structured streaming
  • implement deduplication with and without watermarking
  • store stream output to a directory using a file sink
  • use streaming query objects
  • manage streaming queries
  • enable checkpointing in structured streaming
  • use structured streaming to implement a word count on a text stream
  • describe the basics of Spark Streaming

    • Course Number:
    df_apsk_a03_it_enus

    Expertise Level
    Intermediate