add helm pre-delete hook which deletes all the webhooks (#3148)

* add helm pre-delete hook for graceful uninstallation of webhooks

Signed-off-by: rahulii <r.sawra@gmail.com>

* remove white spaces

Signed-off-by: rahulii <r.sawra@gmail.com>

Co-authored-by: shuting <shuting@nirmata.com>

charts/kyverno/README.md

@@ -128,6 +128,8 @@ The following table lists the configurable parameters of the kyverno chart and t
 | `testImage.tag` | tag for chart test image | `nil` |
 | `tolerations` | list of node taints to tolerate | `[]` |
 | `topologySpreadConstraints` | node/pod topology spread constrains | `[]` |  |
+| `webhooksCleanup.enable` | create a helm pre-delete hook to cleanup webhooks | `false`|
+| `webhooksCleanup.image` | kubectl image to run commands for deleting webhooks| `bitnami/kubectl:latest`|
 
 Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example,
 

charts/kyverno/ci/hooks-values.yaml

@@ -0,0 +1,3 @@
+---
+webhooksCleanup:
+  enable: true

charts/kyverno/templates/helm-pre-delete-hook.yaml

@@ -0,0 +1,29 @@
+{{- if .Values.webhooksCleanup.enable }}
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: {{ template "kyverno.fullname" . }}-hook-pre-delete
+  labels: {{ include "kyverno.labels" . | nindent 4 }}
+    app: kyverno
+  annotations:
+    "helm.sh/hook": pre-delete
+    "helm.sh/hook-delete-policy": hook-succeeded,hook-failed
+spec:
+  template:
+    spec:
+      serviceAccount: {{ template "kyverno.serviceAccountName" . }}
+      containers:
+        - name: kubectl
+          image: {{ .Values.webhooksCleanup.image }}
+          command:
+            - sh
+            - '-c'
+            - >-
+              kubectl delete validatingwebhookconfiguration kyverno-policy-validating-webhook-cfg;
+              kubectl delete validatingwebhookconfiguration kyverno-resource-validating-webhook-cfg;
+              kubectl delete mutatingwebhookconfiguration kyverno-policy-mutating-webhook-cfg;
+              kubectl delete mutatingwebhookconfiguration kyverno-resource-mutating-webhook-cfg;
+              kubectl delete mutatingwebhookconfiguration kyverno-verify-mutating-webhook-cfg;
+      restartPolicy: Never
+  backoffLimit: 2
+{{- end }}

charts/kyverno/values.yaml

@@ -264,3 +264,7 @@ networkPolicy:
   enabled: false
 # A list of valid from selectors according to https://kubernetes.io/docs/concepts/services-networking/network-policies
   ingressFrom: []
+
+webhooksCleanup:
+  enable: false
+  image: "bitnami/kubectl:latest"