QuestionQ281

Data Ingestion and Transformation

A company operates a data warehouse in an on-premises Oracle database. The company wants to create a data lake on AWS. It needs to load data warehouse tables into Amazon S3 and synchronize the tables with incremental data that arrives from the data warehouse each day.

Each table includes a column with monotonically increasing values. Every table is smaller than 50 GB. The data warehouse tables are refreshed nightly between 1 AM and 2 AM. A business intelligence team queries the tables daily between 10 AM and 8 PM.

Which solution meets these requirements in the MOST operationally efficient way?

  • A Use an AWS Database Migration Service (AWS DMS) full load plus CDC job to load tables that contain monotonically increasing data columns from the on-premises data warehouse to Amazon S3. Use custom logic in AWS Glue to append the daily incremental data to a full-load copy that is in Amazon S3.
  • B Use an AWS Glue Java Database Connectivity (JDBC) connection. Configure a job bookmark for a column that contains monotonically increasing values. Write custom logic to append the daily incremental data to a full-load copy that is in Amazon S3.
  • C Use an AWS Database Migration Service (AWS DMS) full load migration to load the data warehouse tables into Amazon S3 every day. Overwrite the previous day's full-load copy every day.
  • D Use AWS Glue to load a full copy of the data warehouse tables into Amazon S3 every day. Overwrite the previous day's full-load copy every day.
Explanation

AWS Glue job bookmarks for JDBC sources retain the value of specified bookmark keys and use it to identify rows not processed by earlier successful runs. A strictly monotonically increasing column is valid as a user-defined bookmark key, so a scheduled Glue job can perform the initial full load and subsequently read and append only daily new rows to Amazon S3.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!