QuestionQ87

VerifiedDesign Solutions for Organizational Complexity

A financial services company receives a recurring data feed from its credit card servicing partner. Approximately 5,000 records are transmitted every 15 minutes in plaintext, delivered over HTTPS directly to an Amazon S3 bucket with server-side encryption. The feed includes sensitive credit card primary account number (PAN) data. The company must automatically mask the PAN before sending the data to another S3 bucket for further internal processing. It must also remove and merge specific fields, then convert the record to JSON format. In addition, more feeds are likely to be added in the future, so the design must be easily expandable.

Which solutions will satisfy these requirements?

Explanation

AWS Glue provides serverless ETL jobs that can read cataloged S3 data, apply custom transformations such as masking PAN values and reshaping fields, and write JSON output to another S3 bucket. A Glue crawler and custom classifier create the required metadata for varying feed formats, and an S3-delivery-triggered Lambda can start the Glue job. This pattern is extensible because additional feeds can be cataloged and processed through additional Glue job configurations.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!