AWS Certified Data Engineer - Associate DEA-C01Demo

By Amazon · Question Mode

QuestionQ247

Data Operations and Support

A company uses an Amazon Kinesis Data Streams data stream to collect transactional data from multiple sources. The company uses an AWS Glue extract, transform, and load (ETL) pipeline to identify outliers in the stream data. When the workflow identifies an outlier, it sends a notification to an Amazon Simple Notification Service (Amazon SNS) topic. The SNS topic starts a second workflow that retrieves logs for the outliers and stores those logs in an Amazon S3 bucket.

The company encounters delays in notifications to the SNS topic when the data stream processes a high volume of data. On reviewing Amazon CloudWatch logs, the company observes a high value for the glue.driver.BlockManager.disk.diskSpaceUsed_MB metric when traffic is high. The company must resolve this issue.

Which solution meets this requirement with the LEAST operational effort?

  • A Increase the number of data processing units (DPUs) in AWS Glue ETL jobs.
  • B Use Amazon EMR to manage the ETL pipeline instead of AWS Glue.
  • C Use AWS Step Functions to orchestrate a parallel workflow state.
  • D Enable auto scaling for the AWS Glue ETL jobs.
Explanation

AWS Glue Auto Scaling dynamically adds workers when a job needs additional parallel processing capacity and removes idle workers as demand subsides. It accommodates high-volume processing periods without ongoing manual DPU adjustments, reducing the Glue job bottleneck that delays downstream notifications.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!