Associate Data Practitioner

By Google · Question Mode

QuestionQ35

Data Pipeline Orchestration

Your company is adopting BigQuery as its data warehouse platform. Your team includes experienced Python developers. You need to recommend a fully managed tool for building batch ETL processes that extract data from various source systems, transform the data by using a variety of Google services, and load the transformed data into BigQuery. You want the tool to take advantage of your team’s Python skills. What should you do?

Explanation

Dataflow is a fully managed batch and streaming data-processing service. Apache Beam lets developers define Dataflow pipelines in Python, including source connectors, transformations, and BigQuery sinks; Dataflow manages the underlying execution resources. Prebuilt Dataflow templates can also accelerate common pipeline deployments.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!