QuestionQ96

VerifiedData Ingestion and Transformation

An online retail company saves Application Load Balancer (ALB) access logs in an Amazon S3 bucket. The company wants to query the logs with Amazon Athena to analyze traffic patterns.

A data engineer creates an unpartitioned Athena table. As the data volume gradually grows, query response times also increase. The data engineer wants to improve Athena query performance.

Which solution meets these requirements with the LEAST operational effort?

Explanation

Partitioning allows Athena to limit scans to the relevant S3 log partitions, reducing data scanned and improving query performance. An AWS Glue crawler uses classifiers to infer data format and schema, groups data into tables or partitions, and writes table and partition metadata to the AWS Glue Data Catalog. It can also run incrementally to add new partitions, avoiding custom ETL or Lambda transformation code.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!