SRE Data Pipelines & Integrity: Pipeline Design


Overview/Description
Expected Duration
Lesson Objectives
Course Number
Expertise Level



Overview/Description

Site reliability engineers (SREs) encounter numerous and varied pipeline technologies and frameworks in their work. When building a pipeline, SREs need to invest considerable time during the design phase to ensure the results work best for the specific case.

In this course, you'll explore the numerous features of a pipeline, such as latency, high availability, development, and operations. You'll also examine the two different pipeline mutations: idempotent and two-phase, as well as the checkpointing technique and various code patterns.

You'll then investigate the five core characteristics of the pipeline maturity matrix and outline how they should be used to design the pipeline technology. You'll then identify potential failure modes, outage causes, and different prevention and response techniques. Finally, you'll outline event delivery system design and operations and how to plan for customer integration and support.



Expected Duration (hours)
1.1

Lesson Objectives

SRE Data Pipelines & Integrity: Pipeline Design

  • discover the key concepts covered in this course
  • differentiate between the various pipeline features
  • differentiate between idempotent and two-phase mutations
  • describe what the checkpointing technique is used for, in what situations it's used, and the advantages of its use
  • list common code patterns, such as reusing code and the microservice approach
  • describe the purpose and characteristics of the maturity matrix
  • list example milestones for the failure tolerance maturity matrix characteristic
  • list examples of the scalability maturity matrix characteristic
  • list examples of the monitoring and debugging maturity matrix characteristic
  • list examples of the implementation maturity matrix characteristic
  • list examples of the testing maturity matrix characteristic
  • differentiate between the different potential failure modes
  • list potential causes of an outage
  • recognize event delivery system design and architecture considerations
  • outline how event delivery system operations are used using practical examples
  • describe considerations when planning for customer integration and support
  • summarize the key concepts covered in this course
  • Course Number:
    it_sredpindj_02_enus

    Expertise Level
    Intermediate