QuestionQ23

VerifiedDeployment and Orchestration of ML Workflows

A company has AWS Glue data-processing jobs orchestrated through an AWS Glue workflow. The AWS Glue jobs can run on a schedule or be started manually.

The company is developing Amazon SageMaker Pipelines for ML model development. During the data-processing phase of model development, the pipelines will use output from the AWS Glue jobs. An ML engineer must implement a solution that integrates the AWS Glue jobs with the pipelines.

Which solution meets these requirements with the LEAST operational overhead?

Explanation

Amazon SageMaker Pipelines Callback steps support integration with processes and AWS services that Pipelines does not directly provide. A callback step sends work to a customer-specified handler and waits for that handler to report step success or failure, allowing the handler to start the AWS Glue workflow and resume the pipeline only after the Glue jobs complete.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!