QuestionQ109

VerifiedData Ingestion and Transformation

A banking company uses an application to collect large quantities of transactional data. The company uses Amazon Kinesis Data Streams for real-time analytics, and its application uses the PutRecord action to send data to Kinesis Data Streams.

A data engineer has noticed network outages at certain times of day. The data engineer wants to configure exactly-once delivery across the entire processing pipeline.

Which solution meets this requirement?

Explanation

A PutRecord call can time out after Kinesis has accepted the record but before the producer receives its acknowledgement. Retrying the call can then create duplicate records. Embedding a stable, unique source identifier in every record enables downstream processing to identify and remove duplicates, providing an exactly-once processing outcome across the pipeline.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!