DP-750: Implementing Data Engineering Solutions Using Azure DatabricksDemo

By Microsoft · Question Mode

QuestionQ10

Deploy and maintain data pipelines and workloads

You have an Azure Databricks workspace containing a Lakeflow Jobs job named Job1. Job1 runs hourly. At times, a job run takes more than one hour to finish. Overlapping runs must be avoided to prevent data corruption.

You need to configure the job’s scheduling behavior. What should you configure?

Select
Concurrency setting:
Execution behavior:
Explanation

Azure Databricks Lakeflow Jobs default the maximum concurrent runs setting to 1, and keeping that limit at one is what prevents two runs of the same hourly job from executing at the same time and corrupting data. By default, when a scheduled trigger fires while the job is already at its concurrent-run limit, Databricks skips the new run instead of running it; enabling the job's queueing option changes that behavior so the new run is held and queued until the in-progress run finishes, avoiding overlap without silently dropping a scheduled run.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!