QuestionQ23

Prepare and process data

You have an Azure Databricks workspace enabled for Unity Catalog.

You have a Lakeflow Spark Declarative Pipelines (SDP) pipeline that writes numerical data to a table named Table1 by using a data-quality validation rule named rule1.

You need to change rule1 to meet these requirements:

  • Ensure that amount is always greater than 0.
  • Prevent an update to Table1 from being committed when data that violates rule1 is detected.

Which statement should you execute?

  • A @dlt.expect_or_fail(“rule1”, ”amount > 0”)
  • B @dlt.expect(“rule1”, “amount > 0”)
  • C @dlt.expect_all_or_drop({“rule1”: “amount > 0”})
  • D @dlt.expect_or_drop(“rule1”, “amount > 0”)
Explanation

expect_or_fail enforces the amount > 0 expectation and fails the pipeline update when any record violates it. A failed table update is rolled back atomically, so no update to the target table is committed.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!