GKE lets you add a GPU-enabled node pool to an existing cluster rather than converting the whole cluster to GPU machine types, which keeps costs down since only the nodes that need GPUs incur that expense. You can't add GPUs to existing node pools, so a new node pool is needed (L4 GPUs use the G2 machine series). To schedule GPU workloads onto these nodes, pods must include a nodeSelector referencing the accelerator label GKE automatically applies to GPU nodes — cloud.google.com/gke-accelerator, with a value identifying the specific GPU type such as nvidia-l4 — and must request GPUs with an nvidia.com/gpu resource limit. Because the cluster already has a non-GPU node pool, GKE taints the GPU nodes and automatically adds the matching toleration to pods that request GPUs. This lets the ML team's pods land on the GPU-enabled nodes while other teams' non-GPU workloads continue running on the standard node pool, minimizing both operational effort (no separate cluster to build/manage) and cost (GPUs are only provisioned where needed).
Community Discussion