AWS Certified Data Engineer - Associate DEA-C01Demo

By Amazon · Question Mode

QuestionQ255

Data Ingestion and Transformation

A company is developing a new data pipeline to process data for business intelligence reports. Users have observed that data is absent from the reports.

A data engineer must add a data-quality check for columns containing null values and for referential integrity at a stage before the data is written to storage.

Which solution meets these requirements with the LEAST operational overhead?

  • A Use Amazon SageMaker Data Wrangler to create a Data Quality and Insights report.
  • B Use AWS Glue ETL jobs to perform a data quality evaluation transform on the data. Use an IsComplete rule on the requested columns. Use a ReferentialItegrity rule for each join.
  • C Use AWS Glue ETL jobs to perform a SQL transform on the data to determine whether requested column contain null values. Use a second SQL transform to check referential integrity.
  • D Use Amazon SageMaker Data Wrangler and a custom Python transform to create custom rules to check for null values and referential integrity.
Explanation

AWS Glue Data Quality can evaluate declarative rules inside an AWS Glue ETL job before data is written to storage. The IsComplete rule verifies that every value in a specified column is non-null, and ReferentialIntegrity validates that values in the primary dataset are represented in a referenced dataset. Using these built-in rules avoids maintaining custom SQL or Python validation logic.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!