QuestionQ13

VerifiedOfficial exam objectives/topics

A company has developed a sales reporting system in Python that connects to Snowflake through the Python Connector. Based on the user’s selections, the system creates the SQL queries required to retrieve report data. It first retrieves the customers matching the specified query parameters (an average of 1000 customer records per report run), then processes the customer records sequentially in a loop. Within that loop, it executes the generated SQL clause for the current customer to retrieve detailed data for that customer number from the sales data table.

When the Data Engineer tested the individual SQL clauses, they performed quickly enough (1 second to retrieve the customers and 0.5 second to retrieve sales data for one customer), but the report’s overall runtime is too long.

How can this situation be improved?

Explanation

Set-based SQL can retrieve the required sales details for all qualifying customers in one operation, such as by joining the customer selection to the sales table, instead of issuing roughly 1,000 serial per-customer queries. The serial loop turns otherwise fast 0.5-second queries into about 500 seconds of cumulative execution. Snowflake’s Python Connector also supports submitting multiple queries asynchronously, but eliminating the repeated-query pattern is the more direct improvement. Additional warehouse clusters address concurrent query workloads, not queries deliberately executed one at a time.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!