AWS Certified Data Engineer - Associate DEA-C01Demo

By Amazon · Question Mode

QuestionQ4

Data Operations and Support

A data engineer must run a data-transformation job whenever a user adds a file to an Amazon S3 bucket. The job will run for under 1 minute. It must send its output to the data engineer in an email message. The data engineer expects users to add one file every hour of the day.

Which solution meets these requirements in the MOST operationally efficient way?

  • A Create a small Amazon EC2 instance that polls the S3 bucket for new files. Run transformation code on a schedule to generate the output. Use operating system commands to send email messages.
  • B Run an Amazon Elastic Container Service (Amazon ECS) task to poll the S3 bucket for new files. Run transformation code on a schedule to generate the output. Use operating system commands to send email messages.
  • C Create an AWS Lambda function to transform the data. Use Amazon S3 Event Notifications to invoke the Lambda function when a new object is created. Publish the output to an Amazon Simple Notification Service (Amazon SNS) topic. Subscribe the data engineer’s email account to the topic.
  • D Deploy an Amazon EMR cluster. Use EMR File System (EMRFS) to access the files in the S3 bucket. Run transformation code on a schedule to generate the output to a second S3 bucket. Create an Amazon Simple Notification Service (Amazon SNS) topic. Configure Amazon S3 Event Notifications to notify the topic when a new object is created.
Explanation

Amazon S3 can invoke an AWS Lambda function when an object is created, enabling the transformation to run only when a file arrives. AWS Lambda is well suited to a sub-minute job and eliminates infrastructure polling and cluster management. The function can publish the transformed output to an Amazon SNS topic, and an email subscription to that topic delivers the output to the data engineer.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!