QuestionQ90

VerifiedData Pipeline Orchestration

Your company is developing a near-real-time streaming pipeline that processes JSON telemetry data from small appliances. You must process messages that arrive at a Pub/Sub topic, capitalize the letters in the serial number field, and write the output to BigQuery. You want a managed service and need to write minimal code for the underlying transformations. What should you do?

Explanation

The Pub/Sub to BigQuery Dataflow template is a managed streaming pipeline that reads JSON-formatted Pub/Sub messages and writes them to a BigQuery table. Its optional user-defined function processes each incoming message, making it suitable for a small field-level transformation such as capitalizing a serial number without implementing a custom streaming service.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!