QuestionQ114
VerifiedMaintaining and automating data workloadsYou migrated a Hadoop job from an on-premises cluster to Dataproc and GCS. Your Spark job is a complex analytical workload with many shuffle operations, and the initial data consists of Parquet files averaging 200–400 MB each. After migrating to Dataproc, you observe some performance degradation and want to optimize it. Your organization is highly cost-sensitive, so you want to keep using Dataproc on preemptible VMs, with only 2 non-preemptible workers for this workload.
What should you do?
Community Discussion