mirrors/kyverno
mirrors/kyverno
1
0
Fork 0
mirror of https://github.com/kyverno/kyverno.git synced 2025-04-16 01:07:14 +00:00
Code Activity
kyverno/charts/kyverno/README.md.gotmpl
Chip Zoller df641106a1
update migration guide with generate guidance (#7409) 
Signed-off-by: Chip Zoller <chipzoller@gmail.com>
2023-06-03 23:55:55 +02:00

404 lines
20 KiB
Go Template
Raw Blame History

{{ template "chart.header" . }}
{{ template "chart.deprecationWarning" . }}
{{ template "chart.description" . }}
{{ template "chart.badgesSection" . }}
## About
[Kyverno](https://kyverno.io) is a Kubernetes Native Policy Management engine.
It allows you to:
- Manage policies as Kubernetes resources (no new language required.)
- Validate, mutate, and generate resource configurations.
- Select resources based on labels and wildcards.
- View policy enforcement as events.
- Scan existing resources for violations.
This chart bootstraps a Kyverno deployment on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager.
Access the complete user documentation and guides at: https://kyverno.io.
## Installing the Chart
**IMPORTANT IMPORTANT IMPORTANT IMPORTANT**
This chart changed significantly between `v2` and `v3`. If you are upgrading from `v2`, please read `Migrating from v2 to v3` section.
**Add the Kyverno Helm repository:**
```console
$ helm repo add kyverno https://kyverno.github.io/kyverno/
```
**Create a namespace:**
You can install Kyverno in any namespace. The examples use `kyverno` as the namespace.
```console
$ kubectl create namespace kyverno
```
**Install the Kyverno chart:**
```console
$ helm install kyverno --namespace kyverno kyverno/kyverno
```
The command deploys Kyverno on the Kubernetes cluster with default configuration. The [installation](https://kyverno.io/docs/installation/) guide lists the parameters that can be configured during installation.
The Kyverno ClusterRole/ClusterRoleBinding that manages webhook configurations must have the suffix `:webhook`. Ex., `*:webhook` or `kyverno:webhook`.
Other ClusterRole/ClusterRoleBinding names are configurable.
**Notes on using ArgoCD:**
When deploying this chart with ArgoCD you will need to enable `Replace` in the `syncOptions`, and you probably want to ignore diff in aggregated cluster roles.
You can do so by following instructions in these pages of ArgoCD documentation:
- [Enable Replace in the syncOptions](https://argo-cd.readthedocs.io/en/stable/user-guide/sync-options/#replace-resource-instead-of-applying-changes)
- [Ignore diff in aggregated cluster roles](https://argo-cd.readthedocs.io/en/stable/user-guide/diffing/#ignoring-rbac-changes-made-by-aggregateroles)
ArgoCD uses helm only for templating but applies the results with `kubectl`.
Unfortunately `kubectl` adds metadata that will cross the limit allowed by Kubernetes. Using `Replace` overcomes this limitation.
Another option is to use server side apply, this will be supported in ArgoCD v2.5.
Finally, we introduced new CRDs in 1.8 to manage resource-level reports. Those reports are associated with parent resources using an `ownerReference` object.
As a consequence, ArgoCD will show those reports in the UI, but as they are managed dynamically by Kyverno it can pollute your dashboard.
You can tell ArgoCD to ignore reports globally by adding them under the `resource.exclusions` stanza in the ArgoCD ConfigMap.
```yaml
resource.exclusions: |
- apiGroups:
- kyverno.io
kinds:
- AdmissionReport
- BackgroundScanReport
- ClusterAdmissionReport
- ClusterBackgroundScanReport
clusters:
- '*'
```
Below is an example of ArgoCD Application manifest that should work with this chart.
```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: kyverno
namespace: argocd
spec:
destination:
namespace: kyverno
server: https://kubernetes.default.svc
project: default
source:
chart: kyverno
repoURL: https://kyverno.github.io/kyverno
targetRevision: 2.6.0
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
- Replace=true
```
## Migrating from v2 to v3
Direct upgrades from v2 of the Helm chart to v3 are not supported due to the number of breaking changes and manual intervention is required. Review and select an option after carefully reading below. Because either method requires down time, an upgrade should only be performed during a maintenance window. Regardless of the chosen option, please read all release notes very carefully to understand the full extent of changes brought by Kyverno 1.10. Release notes can be found at https://github.com/kyverno/kyverno/releases.
**IMPORTANT NOTE**: If you currently use [clone-type](https://kyverno.io/docs/writing-policies/generate/#clone-source) generate rules with synchronization enabled, please do not upgrade to 1.10.0 as there is a bug which may prevent synchronization from occurring on all downstream (generated) resources when the source is updated. Please wait for a future patch where this should be resolved. See [issue 7170](https://github.com/kyverno/kyverno/issues/7170) for further details.
### Option 1 - Uninstallation and Reinstallation
The first option for upgrading, which is the recommended option, involves backing up Kyverno policy resources, uninstalling Kyverno, and reinstalling with v3 of the chart. Policy Reports for policies which have background mode enabled will be regenerated upon the next scan interval.
**Pros**
* Reduced complexity with minimal effort
* Allows re-checking older policies against new validation webhooks in 1.10
**Cons**
* Policy Reports which contained results only from admission mode and from policies/rules where background scans were disabled will be lost.
* Requires additional steps if data-type generate rules are used
Follow the procedure below.
1. READ THE COMPLETE RELEASE NOTES FIRST
2. Backup and export all Kyverno policy resources to a YAML manifest. Use the command `kubectl get pol,cpol,cleanpol,ccleanpol,polex -A -o yaml > kyvernobackup.yaml`.
1. Before performing this step, if you use [data-type](https://kyverno.io/docs/writing-policies/generate/#data-source) generate rules with synchronization enabled (`generate.synchronize: true`) disable synchronization first (set `generate.synchronize: false`). If you do not perform this step first, uninstallation of Kyverno in the subsequent step, which removes all policies, will result in deletion of generated resources.
3. Uninstall your current version of Kyverno.
4. Review the [New Chart Values](#new-chart-values) section and translate your desired features and configurations to the new format.
5. Install the v3 chart with Kyverno 1.10.
6. Restore your Kyverno policies. Use the command `kubectl create -f kyvernobackup.yaml`.
1. Before performing this step, if step 2.1 applied to you, enable synchronization (set `generate.synchronize: true`) AND add the field `spec.generateExisting: true`. This will cause existing, generated resources to be refreshed with the new labeling system used by Kyverno 1.10. Note that this may increment the `resourceVersion` field on all downstream resources. Also, understand that when re-installing these policies with `spec.generateExisting: true`, it could result in additional resources being created at that moment based upon the current match defined in the policy. You may need to further refine the match/exclude blocks of your rules to account for this.
### Option 2 - Scale to Zero
In the second option, Kyverno policies do not have to be backed up however you perform more manual work in order to prepare for the upgrade to chart v3.
**Pros**
* Policy Reports which contained results from admission mode will be preserved
* Kyverno policies do not need to be backed up first
**Cons**
* Older policies will not be revalidated for correctness according to the breaking schema changes. Some policies may not work as they did before.
* Requires additional steps if data-type generate rules are used
Follow the procedure below.
1. READ THE COMPLETE RELEASE NOTES FIRST
2. Scale the `kyverno` Deployment to zero replicas.
3. If coming from 1.9 and you have installed the cleanup controller, scale the `kyverno-cleanup-controller` Deployment to zero replicas.
4. If step 3 applied to you, now delete the cleanup Deployment.
5. Review the [New Chart Values](#new-chart-values) section and translate your desired features and configurations to the new format.
6. Upgrade to the v3 chart by passing the mandatory flag `upgrade.fromV2=true`.
7. If you use [data-type](https://kyverno.io/docs/writing-policies/generate/#data-source) generate rules with synchronization enabled (`generate.synchronize: true`), after the upgrade modify those policies to add the field `spec.generateExisting: true`. This will cause existing, generated resources to be refreshed with the new labeling system used by Kyverno 1.10. Note that this may increment the `resourceVersion` field on all downstream resources. Also, understand that when making this modification, it could result in additional resources being created at that moment based upon the current match defined in the policy. You may need to further refine the match/exclude blocks of your rules to account for this.
### New Chart Values
In `v3` chart values changed significantly, please read the instructions below to migrate your values:
- `config.metricsConfig` is now `metricsConfig`
- `resourceFiltersExcludeNamespaces` has been replaced with `config.resourceFiltersExcludeNamespaces`
- `excludeKyvernoNamespace` has been replaced with `config.excludeKyvernoNamespace`
- `config.existingConfig` has been replaced with `config.create` and `config.name` to __support bring your own config__
- `config.existingMetricsConfig` has been replaced with `metricsConfig.create` and `metricsConfig.name` to __support bring your own config__
- `namespace` has been renamed `namespaceOverride`
- `installCRDs` has been replaced with `crds.install`
- `testImage` has been replaced with `test.image`
- `testResources` has been replaced with `test.resources`
- `testSecurityContext` has been replaced with `test.securityContext`
- `replicaCount` has been replaced with `admissionController.replicas`
- `updateStrategy` has been replaced with `admissionController.updateStrategy`
- `priorityClassName` has been replaced with `admissionController.priorityClassName`
- `hostNetwork` has been replaced with `admissionController.hostNetwork`
- `dnsPolicy` has been replaced with `admissionController.dnsPolicy`
- `nodeSelector` has been replaced with `admissionController.nodeSelector`
- `tolerations` has been replaced with `admissionController.tolerations`
- `topologySpreadConstraints` has been replaced with `admissionController.topologySpreadConstraints`
- `podDisruptionBudget` has been replaced with `admissionController.podDisruptionBudget`
- `antiAffinity` has been replaced with `admissionController.antiAffinity`
- `antiAffinity.enable` has been replaced with `admissionController.antiAffinity.enabled`
- `podAntiAffinity` has been replaced with `admissionController.podAntiAffinity`
- `podAffinity` has been replaced with `admissionController.podAffinity`
- `nodeAffinity` has been replaced with `admissionController.nodeAffinity`
- `startupProbe` has been replaced with `admissionController.startupProbe`
- `livenessProbe` has been replaced with `admissionController.livenessProbe`
- `readinessProbe` has been replaced with `admissionController.readinessProbe`
- `createSelfSignedCert` has been replaced with `admissionController.createSelfSignedCert`
- `serviceMonitor` has been replaced with `admissionController.serviceMonitor`
- `podSecurityContext` has been replaced with `admissionController.podSecurityContext`
- `tufRootMountPath` has been replaced with `admissionController.tufRootMountPath`
- `sigstoreVolume` has been replaced with `admissionController.sigstoreVolume`
- `initImage` has been replaced with `admissionController.initContainer.image`
- `initResources` has been replaced with `admissionController.initContainer.resources`
- `image` has been replaced with `admissionController.container.image`
- `image.pullSecrets` has been replaced with `admissionController.imagePullSecrets`
- `resources` has been replaced with `admissionController.container.resources`
- `service` has been replaced with `admissionController.service`
- `metricsService` has been replaced with `admissionController.metricsService`
- `initContainer.extraArgs` has been replaced with `admissionController.initContainer.extraArgs`
- `envVarsInit` has been replaced with `admissionController.initContainer.extraEnvVars`
- `envVars` has been replaced with `admissionController.container.extraEnvVars`
- `extraArgs` has been replaced with `admissionController.container.extraArgs`
- `extraInitContainers` has been replaced with `admissionController.extraInitContainers`
- `extraContainers` has been replaced with `admissionController.extraContainers`
- `podLabels` has been replaced with `admissionController.podLabels`
- `podAnnotations` has been replaced with `admissionController.podAnnotations`
- `securityContext` has been replaced with `admissionController.container.securityContext` and `admissionController.initContainer.securityContext`
- `rbac` has been replaced with `admissionController.rbac`
- `generatecontrollerExtraResources` has been replaced with `admissionController.rbac.clusterRole.extraResources`
- `networkPolicy` has been replaced with `admissionController.networkPolicy`
- all `extraArgs` now use objects instead of arrays
- logging, tracing and metering are now configured using `*Controller.logging`, `*Controller.tracing` and `*Controller.metering`
- Labels and selectors have been reworked and due to immutability, upgrading from `v2` to `v3` is going to be rejected. The easiest solution is to uninstall `v2` and reinstall `v3` once values have been adapted to the changes described above.
- Image tags are now validated and must be strings, if you use image tags in the `1.35` form please add quotes around the tag value.
- Image references are now using the `registry` setting, if you override the registry or repository fields please use `registry` (`--set image.registry=ghcr.io --set image.repository=kyverno/kyverno` instead of `--set image.repository=ghcr.io/kyverno/kyverno`).
- Admission controller `Deployment` name changed from `kyverno` to `kyverno-admission-controller`.
- `config.excludeUsername` was renamed to `config.excludeUsernames`
- `config.excludeGroupRole` was renamed to `config.excludeGroups`
Hardcoded defaults for `config.excludeGroups` and `config.excludeUsernames` have been removed, please review those fields if you provide your own exclusions.
## Uninstalling the Chart
To uninstall/delete the `kyverno` deployment:
```console
$ helm delete -n kyverno kyverno
```
The command removes all the Kubernetes components associated with the chart and deletes the release.
## Values
The chart values are organised per component.
{{- $other := list -}}
{{- $crds := list -}}
{{- $config := list -}}
{{- $metricsConfig := list -}}
{{- $features := list -}}
{{- $admissionController := list -}}
{{- $backgroundController := list -}}
{{- $cleanupController := list -}}
{{- $reportsController := list -}}
{{- $grafana := list -}}
{{- $test := list -}}
{{- $apiVersionOverride := list -}}
{{- $webhooksCleanup := list -}}
{{- $cleanupJobs := list -}}
{{- range .Values -}}
{{- if (hasPrefix "admissionController." .Key) -}}
{{- $admissionController = append $admissionController . -}}
{{- else if (hasPrefix "backgroundController." .Key) -}}
{{- $backgroundController = append $backgroundController . -}}
{{- else if (hasPrefix "cleanupController." .Key) -}}
{{- $cleanupController = append $cleanupController . -}}
{{- else if (hasPrefix "reportsController." .Key) -}}
{{- $reportsController = append $reportsController . -}}
{{- else if (hasPrefix "crds." .Key) -}}
{{- $crds = append $crds . -}}
{{- else if (hasPrefix "config." .Key) -}}
{{- $config = append $config . -}}
{{- else if (hasPrefix "metricsConfig." .Key) -}}
{{- $metricsConfig = append $metricsConfig . -}}
{{- else if (hasPrefix "webhooksCleanup." .Key) -}}
{{- $webhooksCleanup = append $webhooksCleanup . -}}
{{- else if (hasPrefix "grafana." .Key) -}}
{{- $grafana = append $grafana . -}}
{{- else if (hasPrefix "features." .Key) -}}
{{- $features = append $features . -}}
{{- else if (hasPrefix "test." .Key) -}}
{{- $test = append $test . -}}
{{- else if (hasPrefix "apiVersionOverride." .Key) -}}
{{- $apiVersionOverride = append $apiVersionOverride . -}}
{{- else if (hasPrefix "cleanupJobs." .Key) -}}
{{- $cleanupJobs = append $cleanupJobs . -}}
{{- else -}}
{{- $other = append $other . -}}
{{- end -}}
{{- end }}
### Custom resource definitions
{{ template "chart.valuesTable" (dict "Values" $crds) }}
### Config
{{ template "chart.valuesTable" (dict "Values" $config) }}
### Metrics config
{{ template "chart.valuesTable" (dict "Values" $metricsConfig) }}
### Features
{{ template "chart.valuesTable" (dict "Values" $features) }}
### Admission controller
{{ template "chart.valuesTable" (dict "Values" $admissionController) }}
### Background controller
{{ template "chart.valuesTable" (dict "Values" $backgroundController) }}
### Cleanup controller
{{ template "chart.valuesTable" (dict "Values" $cleanupController) }}
### Reports controller
{{ template "chart.valuesTable" (dict "Values" $reportsController) }}
### Grafana
{{ template "chart.valuesTable" (dict "Values" $grafana) }}
### Webhooks cleanup
{{ template "chart.valuesTable" (dict "Values" $webhooksCleanup) }}
### Test
{{ template "chart.valuesTable" (dict "Values" $test) }}
### Api version override
{{ template "chart.valuesTable" (dict "Values" $apiVersionOverride) }}
### Cleanup jobs
{{ template "chart.valuesTable" (dict "Values" $cleanupJobs) }}
### Other
{{ template "chart.valuesTable" (dict "Values" $other) }}
## TLS Configuration
If `admissionController.createSelfSignedCert` is `true`, Helm will take care of the steps of creating an external self-signed certificate described in option 2 of the [installation documentation](https://kyverno.io/docs/installation/#option-2-use-your-own-ca-signed-certificate)
If `admissionController.createSelfSignedCert` is `false`, Kyverno will generate a self-signed CA and a certificate, or you can provide your own TLS CA and signed-key pair and create the secret yourself as described in the [documentation](https://kyverno.io/docs/installation/#customize-the-installation-of-kyverno).
## Default resource filters
[Kyverno resource filters](https://kyverno.io/docs/installation/#resource-filters) are a used to exclude resources from the Kyverno engine rules processing.
This chart comes with default resource filters that apply exclusions on a couple of namespaces and resource kinds:
- all resources in `kube-system`, `kube-public` and `kube-node-lease` namespaces
- all resources in all namespaces for the following resource kinds:
- `Event`
- `Node`
- `APIService`
- `TokenReview`
- `SubjectAccessReview`
- `SelfSubjectAccessReview`
- `Binding`
- `ReplicaSet`
- `AdmissionReport`
- `ClusterAdmissionReport`
- `BackgroundScanReport`
- `ClusterBackgroundScanReport`
- all resources created by this chart itself
Those default exclusions are there to prevent disruptions as much as possible.
Under the hood, Kyverno installs an admission controller for critical cluster resources.
A cluster can become unresponsive if Kyverno is not up and running, ultimately preventing pods to be scheduled in the cluster.
You can however override the default resource filters by setting the `config.resourceFilters` stanza.
It contains an array of string templates that are passed through the `tpl` Helm function and joined together to produce the final `resourceFilters` written in the Kyverno config map.
Please consult the [values.yaml](./values.yaml) file before overriding `config.resourceFilters` and use the apropriate templates to build your desired exclusions list.
## High availability
Running a highly-available Kyverno installation is crucial in a production environment.
In order to run Kyverno in high availability mode, you should set `replicaCount` to `3` or more.
You should also pay attention to anti affinity rules, spreading pods across nodes and availability zones.
Please see https://kyverno.io/docs/installation/#security-vs-operability for more informations.
{{ template "chart.sourcesSection" . }}
{{ template "chart.requirementsSection" . }}
{{ template "chart.maintainersSection" . }}
{{ template "helm-docs.versionFooter" . }}
View git blame Copy permalink