QuestionQ272

Data Ingestion and Transformation

A data engineer is developing a data pipeline on AWS by using AWS Glue extract, transform, and load (ETL) jobs. The data engineer must process data from Amazon RDS and MongoDB, perform transformations, and load the transformed data into Amazon Redshift for analytics. Data updates must take place every hour.

Which combination of tasks satisfies these requirements with the LEAST operational overhead?

Choose two
  • A Configure AWS Glue triggers to run the ETL jobs every hour.
  • B Use AWS Glue DataBrew to clean and prepare the data for analytics.
  • C Use AWS Lambda functions to schedule and run the ETL jobs every hour.
  • D Use AWS Glue connections to establish connectivity between the data sources and Amazon Redshift.
  • E Use the Redshift Data API to load transformed data into Amazon Redshift.
Explanation

AWS Glue scheduled triggers can run ETL jobs on an hourly cron schedule, eliminating the need to build and operate a Lambda-based scheduler. AWS Glue connections hold reusable connection information, including credentials, URI strings, and VPC settings, for data sources and targets. AWS Glue supports MongoDB, Amazon RDS offerings, and Amazon Redshift, and Glue ETL jobs can write to Redshift directly through its Redshift integration.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!