QuestionQ203
Using Spark MLA Data Scientist is developing an MLlib Pipeline to classify customer data. The Pipeline must execute the following in sequence:
- One-hot encoding of categorical variables
- Scaling of numerical features
- Training of a classification model
They need to design their code so that this sequence of operations executes correctly within the MLlib Pipeline.
Which approach accomplishes this?
- A The Data Scientist must define Spark User Defined Functions (UDFs) for each operation and provide them to the Pipeline as a sequential list, with the first element representing the last operation.
- B The Data Scientist must write custom Python functions for each operation and provide them to the Pipeline as a sequential list, with the first element representing the first operation.
- C The Data Scientist must define the transformer or estimator for each operation and provide them to the Pipeline as a sequential list, with the first element representing the first operation.
- D The Data Scientist must write custom Python functions for each operation and provide them to the Pipeline as a sequential list, with the first element representing the last operation.
Community Discussion