Professional Data Engineer

By Google · Question Mode

QuestionQ56

Designing data processing systems

You have thousands of Apache Spark jobs running in an on-premises Apache Hadoop cluster. You want to migrate these jobs to Google Cloud. You want managed services to run the jobs rather than maintaining a long-lived Hadoop cluster yourself. You are on a tight timeline and want to minimize code changes. What should you do?

Explanation

Dataproc is a fully managed Apache Spark and Hadoop service that can run existing Spark workloads, minimizing application rewrites and removing the need to administer a persistent Hadoop cluster. Cloud Storage provides durable, scalable storage for the migrated data and integrates with Dataproc.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!