QuestionQ97

Architecture Overview

A media company requires a data pipeline that ingests customer review data into a Snowflake table and applies transformations. The company must also use Amazon Comprehend for sentiment analysis and make the de-identified final dataset publicly available to advertising companies using different cloud providers in different Regions.

The data pipeline must run continuously and efficiently as new records arrive in object storage by leveraging event notifications. Operational complexity, infrastructure maintenance—including platform upgrades and security—and development effort must be minimal.

Which design meets these requirements?

Explanation

Snowpipe provides continuous, serverless loading from object storage in response to event notifications. Streams and tasks support incremental, automated transformations. A Snowflake external function can invoke a remote AWS service, such as an API Gateway/Lambda implementation that calls Amazon Comprehend, so sentiment results can be written directly to Snowflake without exporting and re-ingesting data. A Snowflake Marketplace listing can distribute the de-identified dataset to consumers across regions and cloud platforms through Snowflake sharing and fulfillment capabilities.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!