QuestionQ112

Implement Data Pipelines

A company deployed a data pipeline that streams customer transaction data from an on-premises PostgreSQL database to Snowflake for real-time analytics. A Data Engineer must configure a connector to continuously and reliably transfer the data to Snowflake.

Which connector configuration meets these requirements while delivering MINIMAL latency and OPTIMAL performance?

Explanation

The Snowflake Connector for Kafka runs on a Kafka Connect cluster as a sink connector that reads Kafka topics and loads them into Snowflake. This architecture supports continuous, fault-tolerant ingestion from a PostgreSQL-to-Kafka source pipeline. Snowflake’s current Kafka connector uses Snowpipe Streaming and provides near-real-time ingestion with high throughput and exactly-once, ordered delivery semantics.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!