SnowPro Advanced Data Engineer

By Snowflake · Question Mode

QuestionQ30

Implement Data Pipelines

A Data Engineer is configuring a Snowflake data-processing pipeline. The pipeline must process PDF files uploaded to a stage that has a directory table enabled. Data must be extracted from the files, after which the processed data will be loaded into a table.

What should be done to make sure the pipeline automatically processes every newly added PDF file?

Explanation

A stream on the directory table tracks metadata changes for files added to the stage. A task can use that stream, such as through SYSTEM$STREAM_HAS_DATA, to run the PDF-extraction logic and load the processed results into the target table when new files arrive.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!