QuestionQ50

VerifiedContinuous Improvement for Existing Solutions

A company deploys workloads in multiple AWS accounts. Each account has a VPC with VPC flow logs published in text log format to a centralized Amazon S3 bucket. Each log file is compressed with gzip compression. The company must retain the log files indefinitely.

A security engineer occasionally analyzes the logs by using Amazon Athena to query the VPC flow logs. The query performance is degrading over time as the number of ingested logs is growing. A solutions architect must improve the performance of the log analysis and reduce the storage space that the VPC flow logs use.

Which solution will meet these requirements with the LARGEST performance improvement?

Explanation

Apache Parquet is a columnar storage format that dramatically improves Athena query performance compared to text/gzip files by enabling selective column projection and compression. Hourly partitions allow Athena to prune partitions during query execution, further reducing the data scanned. This combination provides the largest performance improvement and reduces storage overhead compared to recompression algorithms (bzip2) or storage class changes.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!