Certified Machine Learning ProfessionalDemo

Question

A Machine Learning Engineer requires a continuous deployment pipeline for models hosted on Databricks Model Serving. The deployment automation must run after a model is trained and registered with MLflow. Its objective is to deploy the latest model version from the MLflow Model Registry only when the model can satisfy the company’s strict latency requirement (P95 < 300ms) while handling production traffic.

How can the engineer confirm that new models satisfy the latency requirement when they are served in production?

A A/B test the latest model with Databricks Model Serving so that the latest model receives 5% of production traffic and the current model receives the rest. Use inference tables to calculate P95 latency and verify it is less than 300ms.
B Serve the latest model on Databricks Model Serving and use a load testing client to generate requests at the production request rate. Use the load testing client to calculate P95 latency and verify it is less than 300ms.
C Use the MLflow Get Run API to retrieve the model metrics from MLflow Tracking and verify that the model_latency metric is less than 300ms.
D Use the MLflow Get Run API to retrieve the model metrics from MLflow Tracking and verify that the inference_latency metric is less than 300ms.

Explanation

Databricks Model Serving can split endpoint traffic between multiple served models for A/B testing, so a candidate model can receive a controlled share of real production requests while the current model continues to serve the remainder. Inference tables retain per-request telemetry and model/version metadata, enabling calculation of the candidate version’s P95 latency and verification that it is below 300 ms under actual production conditions. Serve multiple models to a model serving endpoint Monitor model services using inference tables

Learn more

Certified Machine Learning ProfessionalDemo

AstroTutor

Notes

Readiness

QuestionQ12

Community Discussion