QuestionQ71

VerifiedPrepare data

HOTSPOT

You have a Fabric workspace that uses the default Spark starter pool and runtime version 1.2.

You plan to read a CSV file named Sales_raw.csv in a lakehouse, select columns, and save the data as a Delta table to the managed area of the lakehouse. Sales_raw.csv contains 12 columns.

You have the following code.

Question Image

For each of the following statements, select Yes if the statement is true. Otherwise, select No.

Yes or No
StatementsYesNo
The Spark engine will read only the 'SalesOrderNumber', 'OrderDate','CustomerName', 'UnitPrice' columns from Sales_raw.csv.
The Year column replaces the OrderDate column in the table.
Adding inferSchema='true' to the options will increase the execution time of the query.
Explanation

Spark can apply column pruning to a CSV scan when only a subset of fields is projected. withColumn("Year", year("OrderDate")) adds Year and leaves OrderDate intact. CSV schema inference requires an additional inspection of the data to determine types, increasing the work performed.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!