QuestionQ111

VerifiedIngesting and processing the data

You need to build a machine learning model with BigQuery ML and create a Vertex AI endpoint to host the model. This must support near-real-time processing of continuous streaming data from multiple vendors. The data can include invalid values. What should you do?

Explanation

Pub/Sub decouples multiple streaming data producers from consumers, and Dataflow provides a managed streaming pipeline for transforming, validating, and sanitizing records before they are written to BigQuery. This ensures invalid data can be handled before it is used by the BigQuery ML workflow. Google’s Pub/Sub-to-BigQuery Dataflow patterns support streaming ingestion and message transformation, including field validation.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!