Professional Data Engineer

By Google · Question Mode

QuestionQ32

Ingesting and processing the data

A data analyst on your team needs to analyze data using BigQuery. The analyst wants to create a data pipeline that loads 200 CSV files, averaging 15 MB each, from a Cloud Storage bucket into BigQuery every day. The data must be ingested and transformed before it is available for analysis in BigQuery. You need to recommend a fully managed, no-code solution for the analyst. What should you do?

Explanation

BigQuery pipelines provide a fully managed scheduled workflow that can include a data-preparation task. BigQuery data preparation supports loading files from Cloud Storage and applying visual cleansing and transformation steps before writing the resulting table for analysis. This keeps ingestion and transformation in one scheduled no-code workflow. BigQuery Data Transfer Service supports the CSV load, but a separate BigQuery SQL job requires writing and maintaining SQL rather than using a no-code pipeline.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!