Certified Machine Learning ProfessionalDemo

Question

A Machine Learning Engineer is developing integration tests for an ML pipeline in Databricks. The existing integration test executes the entire workflow but requires four hours because it processes large datasets and performs extensive model training. They must choose the most effective approach to optimize integration-test execution while preserving test reliability. The approach must also follow MLOps best practices.

Which approach accomplishes this?

A Run integration tests only in the production environment using full datasets to ensure complete accuracy of the testing process.
B Use small subsets of production-like data and reduce training iterations while maintaining the same pipeline structure and validation checkpoints in a staging environment that closely matches production.
C Skip the model training phase entirely and only test data transformations and API endpoints using mock model predictions.
D Replace integration tests with unit tests for each pipeline component to reduce execution time and focus on individual component validation in a staging environment that closely matches production.

Explanation

End-to-end integration testing should retain the pipeline’s structure, training path, and validation checks in a staging environment that closely resembles production. Using representative smaller data subsets and fewer training iterations reduces execution time and cost without losing the integration coverage needed to verify that pipeline components work together. Databricks MLOps guidance explicitly recommends these tradeoffs for expensive or time-consuming model training.

Learn more

Certified Machine Learning ProfessionalDemo

AstroTutor

Notes

Readiness

QuestionQ129

Community Discussion