QuestionQ33
VerifiedData Operations and SupportA data engineer must troubleshoot an AWS Glue job that reads from Amazon S3 and writes to Amazon Redshift. The data engineer enabled the bookmark feature for the AWS Glue job.
The data engineer set the AWS Glue job's maximum concurrency to 1.
The AWS Glue job is successfully writing output to Amazon Redshift. However, Amazon S3 files loaded in earlier AWS Glue job runs are reprocessed by later runs.
What is the likely reason that the AWS Glue job is reprocessing the files?
Community Discussion