QuestionQ124
Data Ingestion and TransformationA data engineer requires a fully automated solution that checks multiple databases for new data and processes any data it discovers. The solution must run once every hour. It must be compatible with Amazon RDS, Amazon DynamoDB, and Amazon OpenSearch Service. It must be capable of processing up to 10 MB of data at a time, be optimized for cost and operational overhead, and provide robust error-handling capabilities.
Which solution meets these requirements?
- A Use Amazon EventBridge to invoke AWS Step Functions every hour to deploy an AWS Lambda function to check for data. Configure Step Functions steps to process data that the Lambda function finds. Implement error handling in each state.
- B Use Amazon EventBridge to invoke an AWS Lambda function every hour to check for data. Configure the function to send a message to an Amazon Simple Queue Service (Amazon SQS) queue when the function finds new data. Use a second Lambda function to read the queue and perform the processing.
- C Configure an Apache Spark application to run on Amazon EMR to check for data. Implement error handling in the application. Use Amazon EventBridge to invoke the application every hour.
- D Use Amazon Managed Workflows for Apache Airflow (Amazon MWAA) to create a workflow that runs a directed acyclic graph (DAG) every hour to check for data. Configure the DAG to process identified data. Implement error handling in a Python operator.
Community Discussion