QuestionQ60

VerifiedData Ingestion and Transformation

A company keeps raw clickstream data in an Amazon S3 bucket. The company needs a solution that processes the data daily by using complex PySpark transformations that depend on custom internal libraries. After transformation, the company must store the data in Amazon Redshift for analytics. The solution must scale highly to support large data workloads.

Which solution meets these requirements with the LEAST operational overhead?

Explanation

AWS Glue provides serverless, managed Spark and PySpark ETL execution that scales for large batch workloads. Glue jobs can be scheduled, use additional custom Python libraries supplied to the job, read from Amazon S3, and write to Amazon Redshift through Glue connectivity. This removes the operational work of provisioning, scaling, patching, and maintaining EC2 instances or Amazon EMR clusters.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!