QuestionQ69

VerifiedIngesting and processing the data

Your company is carrying out data preprocessing for a learning algorithm in Google Cloud Dataflow. Numerous data logs are generated during this step, and the team wants to analyze them. Because the campaign is dynamic, the data grows exponentially each hour.

The data scientists wrote the following code to read data for new key features in the logs.

Question Image

You want to improve the performance of this data read. What should you do?

Explanation

Reading only the required columns reduces the volume of data that Dataflow ingests from BigQuery. A .fromQuery read can select those specific fields, avoiding an unnecessary full-table, all-column read as the log table grows.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!