Dataproc on GKE overview Stay organized with collections Save and categorize content based on your preferences.
Dataproc on GKE allows you to execute Big Data applications using the Dataproc jobs API on GKE clusters. Use the Google Cloud console, Google Cloud CLI or the Dataproc API (HTTP request or Cloud Client Libraries) to create a Dataproc on GKE virtual cluster, then submit a Spark, PySpark, SparkR, or Spark-SQL job to the Dataproc service.
Dataproc on GKE deploys Dataproc virtual clusters on a GKE cluster. Unlike Dataproc on Compute Engine clusters, Dataproc on GKE virtual clusters do not include separate master and worker VMs. Instead, when you create a Dataproc on GKE virtual cluster, Dataproc on GKE creates node pools within a GKE cluster. Dataproc on GKE jobs are run as pods on these node pools. The node pools and scheduling of pods on the node pools are managed by GKE.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-10-30 UTC."],[],[]]
