SnowPro Advanced Data Engineer

By Snowflake · Question Mode

QuestionQ66

VerifiedDesign and Manage Snowflake Resources and Performance

A Data Engineer is developing a continuous data pipeline that receives data from Amazon Kinesis Firehose and loads it into a staging table for later use in the data-transformation process. The average file size is 300–500 MB.

The Engineer must ensure that Snowpipe performs well while minimizing costs.

How can this be accomplished?

Explanation

Snowpipe uses Snowflake-managed serverless compute rather than a user-selected virtual warehouse. For continuous loads, files of roughly 100–250 MB provide effective parallelism while reducing per-file queue-management overhead. When Amazon Data Firehose is producing larger files, reducing its buffer size to deliver files in that range is the recommended way to balance Snowpipe performance and cost.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!