QuestionQ95

VerifiedIngesting and processing the data

You submit a new batch job to Dataflow. The job starts successfully, processes several elements, and then unexpectedly fails and shuts down. You open the Dataflow monitoring interface and see errors related to a specific DoFn in the pipeline. What is the most likely cause of these errors?

Explanation

Exceptions in Apache Beam user code, including a DoFn, are reported in the Dataflow monitoring interface. An unhandled exception while processing an element causes its work item to fail; repeated failures of a batch work item cause Dataflow to fail the pipeline.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!