AWS Certified Data Engineer - Associate DEA-C01Demo

By Amazon · Question Mode

QuestionQ45

Data Ingestion and Transformation

A company is migrating a legacy application to an Amazon S3-based data lake. A data engineer reviewed data associated with the legacy application and found that the legacy data includes duplicate information.

The data engineer must identify and remove duplicate information from the legacy application data.

Which solution meets these requirements with the LEAST operational overhead?

  • A Write a custom extract, transform, and load (ETL) job in Python. Use the DataFrame.drop_duplicates() function by importing the Pandas library to perform data deduplication.
  • B Write an AWS Glue extract, transform, and load (ETL) job. Use the FindMatches machine learning (ML) transform to transform the data to perform data deduplication.
  • C Write a custom extract, transform, and load (ETL) job in Python. Import the Python dedupe library. Use the dedupe library to perform data deduplication.
  • D Write an AWS Glue extract, transform, and load (ETL) job. Import the Python dedupe library. Use the dedupe library to perform data deduplication.
Explanation

AWS Glue FindMatches is a managed machine learning transform for identifying duplicate or matching records, including records that do not match exactly. It can produce an output copy of the input table with duplicate rows removed, avoiding the maintenance of custom ETL code and external Python deduplication libraries.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!