QuestionQ97

VerifiedMaintaining and automating data workloads

You work for a manufacturing plant that combines application log files into one log file each day at 2:00 AM. You have written a Google Cloud Dataflow job to process this log file. You need to ensure that the log file is processed once daily as inexpensively as possible. What should you do?

Explanation

App Engine Cron Service supports recurring scheduled tasks and can invoke an App Engine handler that launches a Dataflow pipeline. Scheduling the existing batch pipeline once per day processes the consolidated file automatically without operating a continuously running streaming job.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!