Skip to content

Node Provisioning with Custom Class

Imported from Confluence

Content may be outdated. Verify before following any procedures. View original | Last updated: September 2024

We can specify multiple instance types for any nodepool via ComputeClass object. This custom compute class works only on GKE version 1.30.3-gke.1451000 or later.

To test ComputeClass features with our manual nodepool, We did the following step. 

  1. We upgraded direct-infra-dev cluster version to 1.30.4-gke.1348000
  2. Create a ComputeClass object with extra machine type with a nodepool name in priority

| 

apiVersion: cloud.google.com/v1 kind: ComputeClass metadata:   name: my-test-class spec:   priorities:  - nodepools: [test-nodepool]  - machineType: n2-standard-4     spot: true    minCores: 2    storage:       bootDiskSize: 50  nodePoolAutoCreation:     enabled: true
| | --- |

The class has priority list for auto-scaling pods, it tries to schedule pods on test-nodepool first , if for any reason any node available resources or new node can’t be created, then auto-scaling tries to create a new nodepool with machine type n2-standard-4 from priority list and schedule the pods in the new nodepool. 

  1. Create a test nodepool which is pointed in ComputeClass object 
"test-nodepool" = {
      config = {
        max_pods_per_node = 32
        disk_size_gb      = 50
        machine_type      = "e2-medium"
        disk_type         = "pd-balanced"
        spot              = true
        od                = false
        auto_upgrade      = true
        auto_repair       = true
        min_count         = 0
        max_count         = 0
        total_min_count   = 0
        total_max_count   = 2
      }
      k8s_labels_override = { "cloud.google.com/compute-class" = "my-test-class" }
      taints_override = [
        {
          key    = "cloud.google.com/compute-class"
          value  = "my-test-class"
          effect = "NO_SCHEDULE"
        }
      ]
    }

apiVersion: apps/v1

kind: Deployment
metadata:

 name: custom-workload
spec:

 replicas: 3
 selector:

   matchLabels:
     app: custom-workload

 template:
   metadata:
     labels:

       app: custom-workload
   spec:

     nodeSelector:
       cloud.google.com/compute-class: my-test-class

     tolerations:
     - key: "cloud.google.com/compute-class"

       operator: "Equal"
       value: "my-test-class"
       effect: "NoSchedule"

     containers:
     - name: test

       image: nginx
       resources:

         requests:
           cpu: 100m
           memory: 1.5Gi

The kubectl output below

k get pods  -o wide

NAME                               READY   STATUS    RESTARTS   AGE     IP             NODE                                                  NOMINATED NODE   READINESS GATES

custom-workload-6c85f765c7-k9skp   1/1     Running   0          4m41s   172.31.1.130   gke-gke-infra-direct-nap-n2-standard--69658079-lkvf            

custom-workload-6c85f765c7-pnrw4   1/1     Running   0          4m41s   172.31.1.66    gke-gke-infra-direct-de-test-nodepool-e634ca8b-wps7            

custom-workload-6c85f765c7-smn6f   1/1     Running   0          4m41s   172.31.0.66    gke-gke-infra-direct-de-test-nodepool-e634ca8b-vsrh