Professional Machine Learning Engineer

You are building an ML pipeline for data processing, model training, and model deployment that uses different Google Cloud services. You have developed code for every individual task and anticipate a high frequency of new files. You now need to add an orchestration layer above these tasks. The orchestration pipeline must run only when new files exist in your dataset in a Cloud Storage bucket. You also need to minimize compute-node costs. What should you do?