QuestionQ67

Advanced Feature Store Concepts

A Data Scientist has a Unity Catalog feature table named feature_table. For feature_table, the data scientist ran a Python notebook that calculates features from raw data in an upstream UC-managed Delta table and then writes them to feature_table.

As new data arrives, it must be added to feature_table as quickly as possible while retaining the table's existing features. They want to automate updates to feature_table while also optimizing cost. They created the following code in a notebook and want to automate that notebook in Databricks.

Question Image

They will use the notebook code to update feature_table and then run the notebook as a job.

Which job approach best meets their goals and requirements?

  • A A job on an interactive cluster that has a scheduled trigger on it that runs the job daily at a set time.
  • B A job on automated compute that has a scheduled trigger on it that runs the job daily at a set time.
  • C A job on automated compute that has a table update trigger configured on the job.
  • D A Job on an interactive cluster that has a table update trigger configured on the job.
Explanation

A table-update trigger starts the job when its monitored upstream table is updated, enabling prompt feature refreshes without a continuously running cluster. Jobs (automated) compute is intended for non-interactive job workloads and terminates after a run, unlike interactive all-purpose compute. The merge write mode upserts incoming feature rows while leaving existing rows that are absent from the incoming DataFrame unchanged.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!