SnowPro Advanced Data Engineer

By Snowflake · Question Mode

QuestionQ31

Implement Data Pipelines

A company uses Snowpipe to ingest millions of Change Data Capture (CDC) rows each day into a Snowflake staging table in real time. The CDC must be processed, combined with other Snowflake data, and loaded into a final table as part of the complete data pipeline.

How can a Data Engineer MOST efficiently process the incoming CDC continuously?

Explanation

A stream exposes the incremental CDC changes on the staging table, and a task can transform and load those changes only when SYSTEM$STREAM_HAS_DATA indicates that the stream contains data. This supports efficient, low-latency continuous processing without repeatedly scanning or rebuilding the full staging dataset.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!