QuestionQ83

VerifiedData Pipeline Orchestration

You work for a retail company that gathers customer data from several sources:

  • Online transactions: Stored in a MySQL database
  • Customer feedback: Stored as text files on a company server
  • Social media activity: Streamed in real time from social media platforms

You must design a data pipeline to extract and load this data into the appropriate Google Cloud storage system(s) for further analysis and ML model training. What should you do?

Explanation

BigQuery is a fully managed analytics platform with built-in machine-learning capabilities, making it suitable for structured transactional data and streamed social-media records intended for analysis and model training. Cloud Storage is appropriate for retaining customer-feedback text files as object data. Google Cloud provides a Dataflow streaming pattern that reads events from Pub/Sub and writes them to BigQuery.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!