DP-750: Implementing Data Engineering Solutions Using Azure DatabricksDemo

By Microsoft · Question Mode

QuestionQ35

Deploy and maintain data pipelines and workloads

You have an Azure Databricks workspace enabled for Unity Catalog.

You plan to ingest data from CSV files stored in Azure Data Lake Storage Gen2. New rows are appended frequently.

You need to implement a data-ingestion solution that meets the following requirements:

  • New data must be available near real-time (NRT).
  • The data must be stored in managed Delta tables.
  • The solution must minimize custom code and maintenance effort.

What should you include in the solution?

  • A scheduled Apache Spark batch jobs
  • B an Azure Data Factory pipeline
  • C Auto Loader
  • D an external table that references the CSV files
Explanation

Auto Loader incrementally detects and processes new files arriving in Azure Data Lake Storage, supports CSV input, and is designed for near-real-time ingestion. With Unity Catalog, it can write the streaming output to a managed Delta table, providing a managed, low-maintenance file-ingestion pattern.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!