QuestionQ85

VerifiedML Solution Monitoring, Maintenance, and Security

A company gathers customer data every day and stores it as compressed files in an Amazon S3 bucket partitioned by date. Each month, analysts download and process the data to validate its quality, then upload it to Amazon QuickSight dashboards.

An ML engineer must implement a solution that automatically validates data quality before the data is sent to QuickSight.

Which solution meets these requirements with the LEAST operational overhead?

Explanation

AWS Glue Data Quality is a serverless managed service that evaluates data-quality rules against AWS Glue Data Catalog tables. An AWS Glue crawler can update the catalog with newly discovered date partitions in the S3 data, allowing Glue Data Quality rules to validate the dataset without maintaining custom PySpark or Lambda code. AWS documents that Glue Data Quality rulesets are associated with Data Catalog tables and that the service supports S3 data sources and scheduled evaluations.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!