Associate Data Practitioner

By Google · Question Mode

QuestionQ18

Data Pipeline Orchestration

You are building a data-ingestion pipeline that loads small CSV files from Cloud Storage into BigQuery. You want each file loaded when it arrives to minimize data latency, while keeping cost and maintenance to a minimum. What should you do?

Explanation

Cloud Run functions can be triggered by Cloud Storage object events through Eventarc, including when a new object is finalized. A function can initiate a BigQuery load job for each uploaded CSV, providing event-driven ingestion without operating scheduled orchestration or a cluster.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!