QuestionQ83

VerifiedDeployment and Orchestration of ML Workflows

An ML engineer is developing an ML pipeline. The pipeline must process a dataset in two ways by using Amazon Athena. It must use batch processing for large-scale data transformations and model training. It must also use near-real-time processing for low-latency inference and analytics queries.

Which file format provides the LEAST latency for both processing types?

Explanation

Apache Parquet is a columnar format optimized for fast analytical retrieval in Amazon Athena. Its column-level compression, predicate pushdown, metadata-based block skipping, and parallel reads reduce the amount of data scanned and I/O, improving performance for both large batch workloads and low-latency queries.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!