QuestionQ23

Implement Snowflake data science best practices

A company’s platform team plans to integrate its existing data lake with Snowflake. The data lake is hundreds of TBs in size, and the team does not want to copy most of the data into Snowflake. A Data Scientist at the company needs to query and access the data lake’s metadata. An external stage in Snowflake already references the data lake’s location.

What is the MOST efficient way to integrate the existing data lake into the Snowflake environment?

  • A Move the data lake files to an internal stage in Snowflake to allow for access to the metadata and data from the data lake.
  • B Using the existing external stage, create SELECT statements that can be run on-demand, referencing the files in the data lake directly.
  • C Create a PIPE for each data lake table which will allow for on-demand querying of the data lake files by leveraging the Snowpipe service.
  • D Create an external table for each corresponding data lake table to enable querying data stored in files in the data lake as if the data lake table was inside a database.
Explanation

Snowflake external tables reference files in an external stage, allow those files to be queried as table data without loading most of the lake into Snowflake, and store file-level metadata such as filenames and version-related properties. This provides the required table-like query access and metadata access while avoiding data duplication.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!