QuestionQ5

VerifiedDeployment and Orchestration of ML Workflows

A company plans to build several ML prediction models. Its training data is stored in Amazon S3. The complete dataset is more than 5 ТВ in size and contains CSV, JSON, Apache Parquet, and simple text files.

The data must be processed through several consecutive steps. These steps include complex manipulations that can require hours to complete. Some processing includes natural language processing (NLP) transformations. The entire process must be automated.

Which solution meets these requirements?

Explanation

Amazon SageMaker Pipelines orchestrates interconnected, dependency-based ML workflow steps and can include SageMaker processing jobs for large-scale, long-running custom data transformations such as NLP. Amazon EventBridge can automatically initiate a pipeline execution, including in response to events such as new objects in an S3 bucket. AWS Lambda is unsuitable for transformations that run for hours because a Lambda invocation has a maximum execution time of 15 minutes.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!