QuestionQ101

VerifiedDesigning data processing systems

You are choosing services to write and transform JSON messages from Cloud Pub/Sub to BigQuery for a Google Cloud data pipeline. You want to minimize service costs while monitoring and accommodating input-data volume that varies in size with minimal manual intervention. What should you do?

Explanation

Cloud Dataflow runs managed streaming transformations and horizontal autoscaling can add or remove worker instances as workload demand changes. Cloud Monitoring exposes system-lag metrics for streaming Dataflow jobs, enabling monitoring while autoscaling handles fluctuating input volume without manual cluster resizing.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!