QuestionQ97

Prepare data

You have a Fabric workspace that uses the default Spark starter pool and runtime version 1.2.

You plan to read a CSV file named Sales_raw.csv in a lakehouse, select columns, and save the data as a Delta table to the managed area of the lakehouse. Sales_raw.csv contains 12 columns.

You have the following code.

Question Image

For each of the following statements, select Yes if the statement is true. Otherwise, select No.

Yes or No
StatementsYesNo
The Spark engine will read only the ‘SalesOrderNumber’, ‘OrderDate’, ‘CustomerName’, ‘UnitPrice’ columns from Sales_raw.csv.
Removing the partition will reduce the execution time of the query.
Adding inferSchema= ‘true’ to the options will increase the execution time of the query.
Explanation

Spark’s column pruning reads only the columns required by the projection and the Year expression: SalesOrderNumber, OrderDate, CustomerName, and UnitPrice. Year partitioning can enable partition pruning for filters on Year, so removing it does not inherently reduce execution time. CSV schema inference adds a scan/inspection step to determine column data types, increasing processing time.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!