QuestionQ146
Data Ingestion and TransformationA company extracts approximately 1 TB of data each day from data sources including SAP HANA, Microsoft SQL Server, MongoDB, Apache Kafka, and Amazon DynamoDB. Some data sources have undefined schemas or schemas that change.
A data engineer must implement a solution that detects the schemas for these data sources. The solution must extract, transform, and load the data into an Amazon S3 bucket. The company has a service level agreement (SLA) requiring that the data be loaded into the S3 bucket within 15 minutes of its creation.
Which solution meets these requirements with the LEAST operational overhead?
- A Use Amazon EMR to detect the schema and to extract, transform, and load the data into the S3 bucket. Create a pipeline in Apache Spark.
- B Use AWS Glue to detect the schema and to extract, transform, and load the data into the S3 bucket. Create a pipeline in Apache Spark.
- C Create a PySpark program in AWS Lambda to extract, transform, and load the data into the S3 bucket.
- D Create a stored procedure in Amazon Redshift to detect the schema and to extract, transform, and load the data into a Redshift Spectrum table. Access the table from Amazon S3.
Community Discussion