Associate Data Practitioner

By Google · Question Mode

QuestionQ50

Data Pipeline Orchestration

You need to process and load a daily sales CSV file in Cloud Storage into BigQuery for downstream reporting. You must quickly build a scalable data pipeline that transforms the data and provides insights into data-quality issues. What should you do?

Explanation

The Cloud Storage CSV files to BigQuery Dataflow batch template reads CSV files from Cloud Storage, transforms and loads them into BigQuery at scale, and requires a bad-records BigQuery table. That table stores rejected CSV records together with error messages, providing direct visibility into data-quality problems.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!