Associate Data Practitioner

By Google · Question Mode

QuestionQ40

Data Pipeline Orchestration

You are developing a batch data pipeline to process 100 GB of structured data from multiple sources for daily reporting. You must transform and standardize the data before loading it so that it is stored in one dataset. You want a low-code solution that is easy to build and manage. What should you do?

Explanation

Cloud Data Fusion provides a managed visual interface for building scalable data-integration pipelines, including ingesting from varied sources, cleansing and transforming records, and writing the result to destination systems such as BigQuery. BigQuery provides the centralized analytics dataset suited to daily reporting. This meets the low-code ETL requirement without requiring custom Cloud Run function code.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!