Professional Data Engineer

By Google · Question Mode

QuestionQ1

Ingesting and processing the data

You analyze user clickstream data to personalize content recommendations. The data arrives continuously and must be processed with low latency, including transformations such as sessionization (grouping clicks by user within a time window) and aggregation of user activity. You need to identify a scalable solution that can handle millions of events per second and remain resilient to late-arriving data. What should you do?

Explanation

Pub/Sub provides scalable, reliable ingestion for continuous event streams. Dataflow with Apache Beam performs low-latency stateful stream processing, including session windows and aggregations, and uses event-time watermarks, triggers, and allowed-lateness settings to handle out-of-order and late data. BigQuery is a suitable analytical store for the processed clickstream data.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!