QuestionQ190
Validation TestingA Machine Learning Engineer has created a machine learning model-training pipeline using multiple Databricks notebooks. Each notebook includes both feature-engineering functions and embedded unit tests. The pipeline has produced promising results and is now being readied for production deployment on Databricks. The engineer is considering refactoring the code by placing all functions and tests in separate Python files in the Databricks workspace, while using notebooks mainly for orchestration and execution. They understand and want to fully leverage Databricks architecture and MLOps capabilities.
Which advantage would this refactoring approach provide?
- A Test suites will automatically execute when Python modules are imported into any Databricks notebook or job.
- B Separate python files will automatically integrate with Databricks’ built-in automatic orchestration.
- C The total number of files in the Databricks workspace will be reduced, simplifying project management.
- D Functions in separate .py files can be version-controlled through Databricks Repos and reused across multiple notebooks and jobs.
Community Discussion