Google Cloud logo

Data Integration with Cloud Data Fusion

This instructor led live training introduces learners to Google Cloud’s data integration capability using Cloud Data Fusion. In this course, we discuss challenges with data integration and the need for a data integration platform (middleware). We then discuss how Cloud Data Fusion can help to effectively integrate data from a variety of sources and formats and generate insights. We take a look at Cloud Data Fusion’s main components and how they work, how to process batch data and real time streaming data with visual pipeline design, rich tracking of metadata and data lineage, and how to deploy data pipelines on various execution engines.

What you Will learn

What's Included?

Instructor Live Training

An instructor will answer your questions

OFFICIAL GOOGLE CLOUD CONTENT

Course content reflects the latest google cloud class

hands on labs

Real world hands on labs provided by Qwiklabs and supported by instructor

CertIficate of completion

Receive official certificate on completion of 80% of labs

Who's this course for?

Level

Language

Duration

Prerequisites

products

Course Content

Topics

  • Data integration: what, why, challenges
  • Data integration tools used in industry
  • User personas
  • Introduction to Cloud Data Fusion
  • Data integration critical capabilities
  • Cloud Data Fusion UI components

Objectives

  • Understand the need for data integration,
  • List the situations/cases where data integration can help businesses,
  • List the available data integration platforms and tools,
  • Identify the challenges with data integration
  • Understand the use of Cloud Data Fusion as a data integration platform
  • Create a Cloud Data Fusion instance,
  • Familiarize with core framework and major components in Cloud Data Fusion

Activities

  • Graded lab, quiz, discussion activity

Topics

  • Cloud Data Fusion architecture
  • Core concepts
  • Data pipelines and directed acyclic graphs (DAG)
  • Pipeline Lifecycle
  • Designing pipelines in Pipeline Studio

Objectives

  • Understand Cloud Data Fusion architecture
  • Define what a data pipeline is
  • Understand the DAG representation of a data pipeline,
  • Learn to use Pipeline Studio and its components
  • Design a simple pipeline using Pipeline Studio,
  • Deploy and execute a pipeline

Activities

  • Graded lab and quiz

Topics

  • Branching, Merging and Joining
  • Actions and Notifications
  • Error handling and Macros
  • Pipeline Configurations, Scheduling, Import and Export

Objectives

  • Perform branching, merging, and join operations.
  • Execute pipeline with runtime arguments using macros.
  • Work with error handlers.
  • Execute pre- and post-pipeline executions with help of actions and notifications.
  • Schedule pipelines for execution.
  • Import and export existing pipelines.

Activities

  • Graded labs and quiz

Topics

  • Schedules and triggers
  • Execution environment: Compute profile and provisioners
  • Monitoring pipelines

Objectives

  • Understand the composition of an execution environment.
  • Configure your pipeline’s execution environment, logging, and metrics. Understand concepts like compute profile and provisioner.
  • Create a compute profile.
  • Create pipeline alerts.
  • Monitor the pipeline under execution.

Activities

  • Quiz

Topics

  • Wrangler
  • Directives
  • User-defined directives

Objectives

  • Understand the use of Wrangler and its main components.
  • Transform data using Wrangler UI.
  • Transform data using directives/CLI methods.
  • Create and use user-defined directives.

Activities

  • Graded lab and quiz

Topics

  • Understand the data integration architecture.
  • List various connectors.
  • Use the Cloud Data Loss Prevention (DLP) API.
  • Understand the reference architecture of streaming pipelines.
  • Build and execute a streaming pipeline.

Objectives

  • Connectors
  • DLP
  • Reference architecture for streaming applications
  • Building streaming pipelines

Activities

  • Graded lab, quiz, discussion activity

Topics

  • Metadata
  • Data lineage

Objectives

  • List types of metadata.
  • Differentiate between business, technical, and operational metadata.
  • Understand what data lineage is.
  • Understand the importance of maintaining data lineage
  • Differentiate between metadata and data lineage.

Activities

  • Graded lab and quiz

Topics

  • Course Summary

Objectives

  • Review the course objectives & concepts

sign up to be notified for upcoming classes

Have Questions?

No worries. Send us a quick message and we’ll be happy to answer any questions you have.

Upcoming Schedule

Ref: T-DICDF-I-01