Professional Data Engineer

You analyze user clickstream data to personalize content recommendations. The data arrives continuously and must be processed with low latency, including transformations such as sessionization (grouping clicks by user within a time window) and aggregation of user activity. You need to identify a scalable solution that can handle millions of events per second and remain resilient to late-arriving data. What should you do?

A web server publishes click events as messages to a Pub/Sub topic. The server includes an eventTimestamp attribute in each message that records when the click occurred. You have a Dataflow streaming job that reads this Pub/Sub topic through a subscription, performs some transformations, and writes the results to another Pub/Sub topic for the advertising department.

The advertising department must receive every message within 30 seconds of its corresponding click, but reports that messages arrive late. Your Dataflow job has system lag of about 5 seconds and data freshness of about 40 seconds. Inspection of several messages shows no more than 1 second of lag between eventTimestamp and publishTime. What is the issue, and what should you do?

You run a logistics company and want to make event delivery from vehicle-based sensors more reliable. You operate small data centers worldwide to capture these events, but the leased lines connecting your event-collection infrastructure to your event-processing infrastructure are unreliable and have unpredictable latency. You want to resolve this in the most cost-effective manner. What should you do?

You have a BigQuery table that receives data directly from a Pub/Sub subscription. The ingested data is encrypted using a Google-managed encryption key. You must comply with a new organization policy requiring keys from a centralized Cloud Key Management Service (Cloud KMS) project to encrypt data at rest. What should you do?

You monitor and optimize your team’s BigQuery instance. A particular daily report that uses a large JOIN operation is consistently slow. You want to inspect the query’s execution plan to identify potential performance bottlenecks within the JOIN as quickly as possible. What should you do?