Running workload on dedicated Node Pools¶
Imported from Confluence
Content may be outdated. Verify before following any procedures. View original | Last updated: January 2023
Running workload on dedicated node pools are done by usage of NodeSelectors, Toleration and Affinity rules.
Below is example how to run kubernetes workload(spike) on dedicated spot node pools with automatic switch on-demand in case of lack spot instances.
- Two node pools need to be deployed, first for spot instances, second for on-demand:
{
name = "spike-spot"
max_pods_per_node = 32
disk_size_gb = 100
machine_type = "e2-highmem-8"
disk_type = "pd-standard"
spot = true
auto_upgrade = true
auto_repair = true
min_count = 0
max_count = 50
},
{
name = "spike-on-demand"
max_pods_per_node = 32
disk_size_gb = 100
machine_type = "e2-highmem-8"
disk_type = "pd-standard"
auto_upgrade = true
auto_repair = true
min_count = 0
max_count = 50
},
- To be able to schedule workload on dedicated nodes two set of labels need to be created for both node pools described above, one with same name, second with different name:
node_pools_labels = {
spike-on-demand = {
service = "spike",
node_pool = "on-demand"
}
spike-spot = {
service = "spike",
node_pool = "spot"
}
}
- To be able to restrict other workloads to be scheduled on this nodes we need to add same taints for both node pools:
node_pools_taints = {
spike-on-demand = [
{
key = "noexecute"
value = "spike"
effect = "NO_EXECUTE"
},
{
key = "noschedule"
value = "spike"
effect = "NO_SCHEDULE"
},
]
spike-spot = [
{
key = "noexecute"
value = "spike"
effect = "NO_EXECUTE"
},
{
key = "noschedule"
value = "spike"
effect = "NO_SCHEDULE"
},
]
}
- We would like to define preffered way of scheduling pods first into spot instances. In case if there is lack of spot instances we would like to schedule them on on-demand. Appropriare node Affinity rules need to be specified.
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- preference:
matchExpressions:
- key: "node_pool"
operator: In
values:
- "spot"
weight: 1
#### Below is example of simple ubuntu pod deployment on dedicated node pool:
apiVersion: v1
kind: Pod
metadata:
name: ubuntu
labels:
app: ubuntu
spec:
containers:
- image: ubuntu
command:
- "sleep"
- "604800"
imagePullPolicy: IfNotPresent
name: ubuntu
nodeSelector:
service: spike
tolerations:
- effect: "NoSchedule"
key: "noschedule"
operator: "Equal"
value: "spike"
- effect: "NoExecute"
key: "noexecute"
operator: "Equal"
value: "spike"
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- preference:
matchExpressions:
- key: "node_pool"
operator: In
values:
- "spike-spot"
weight: 1
restartPolicy: Always
Result: