AWS Certified Data Engineer - Associate DEA-C01Demo

By Amazon · Question Mode

QuestionQ14

Data Operations and Support

A company receives test results from testing facilities located worldwide. The company stores the test results as millions of 1 KB JSON files in an Amazon S3 bucket. A data engineer must process the files, convert them to Apache Parquet format, and load them into Amazon Redshift tables. The data engineer uses AWS Glue to process the files, AWS Step Functions to orchestrate the processes, and Amazon EventBridge to schedule jobs.

The company has recently added more testing facilities. The time needed to process the files is increasing. The data engineer must reduce the data-processing time.

Which solution will MOST reduce the data-processing time?

  • A Use AWS Lambda to group the raw input files into larger files. Write the larger files back to Amazon S3. Use AWS Glue to process the files. Load the files into the Amazon Redshift tables.
  • B Use the AWS Glue dynamic frame file-grouping option to ingest the raw input files. Process the files. Load the files into the Amazon Redshift tables.
  • C Use the Amazon Redshift COPY command to move the raw input files from Amazon S3 directly into the Amazon Redshift tables. Process the files in Amazon Redshift.
  • D Use Amazon EMR instead of AWS Glue to group the raw input files. Process the files in Amazon EMR. Load the files into the Amazon Redshift tables.
Explanation

AWS Glue DynamicFrame file grouping combines many small S3 input files into larger in-memory partitions. For JSON input, this reduces per-file and task overhead and is specifically intended to improve performance when processing a large number of small files, while preserving the existing Glue transformation to Parquet and subsequent Amazon Redshift load.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!