Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com> Signed-off-by: Charles-Edouard Brétéché <charled.breteche@gmail.com>
14 KiB
Developer Instructions
This document covers basic needs to work with Kyverno codebase.
It contains instructions to build, run, and test Kyverno.
- Tools
- Building local binaries
- Building local images
- Pushing images
- Deploying a local build
- Code generation
Tools
Building and/or testing Kyverno requires additional tooling.
We use make
to simplify installing the tools we use.
Tools will be installed in the .tools
folder when possible, this allows keeping installed tools local to the Kyverno repository.
The .tools
folder is ignored by git
and binaries should not be committed.
Note
: If you don't install tools, they will be downloaded/installed as necessary when running
make
targets.
You can manually install tools by running:
make install-tools
To remove installed tools, run:
make clean-tools
Building local binaries
The Kyverno repository contains code for three different binaries:
kyvernopre
: Binary to update/cleanup existing resources in clusters. This is typically run as an init container before Kyverno controller starts.kyverno
: The Kyverno controller binary.cli
: The Kyverno command line interface.
Note
: You can build all binaries at once by running
make build-all
.
Building kyvernopre locally
To build kyvernopre
binary on your local system, run:
make build-kyvernopre
The binary should be created at ./cmd/initContainer/kyvernopre
.
Building kyverno locally
To build kyverno
binary on your local system, run:
make build-kyverno
The binary should be created at ./cmd/kyverno/kyverno
.
Building cli locally
To build cli
binary on your local system, run:
make build-cli
The binary should be created at ./cmd/cli/kubectl-kyverno/kubectl-kyverno
.
Building local images
In the same spirit as building local binaries, you can build local docker images instead of local binaries.
Currently, we are supporting two build systems:
Note
: We started with
docker
and are progressively moving toko
.
As the ko
based build system matures, we will deprecate and remove docker
based builds.
Choosing between docker
and ko
boils down to a prefix when invoking make
targets.
For example:
make docker-build-kyverno
creates a docker image using thedocker
build systemmake ko-build-kyverno
creates a docker image using theko
build system
Building local images with docker
When building local images with docker you can specify the registry used to create the image names by setting the REGISTRY
environment variable (default value is ghcr.io
).
Note
: You can build all local images at once by running
make docker-build-all
.
Building kyvernopre image locally
To build kyvernopre
image on your local system, run:
make docker-build-kyvernopre
The resulting image should be available locally, named ghcr.io/kyverno/kyvernopre
(by default, if REGISTRY
environment variable was not set).
Building kyverno image locally
To build kyverno
image on your local system, run:
make docker-build-kyverno
The resulting image should be available locally, named ghcr.io/kyverno/kyverno
(by default, if REGISTRY
environment variable was not set).
Building cli image locally
To build cli
image on your local system, run:
make docker-build-cli
The resulting image should be available locally, named ghcr.io/kyverno/kyverno-cli
(by default, if REGISTRY
environment variable was not set).
Building local images with ko
When building local images with ko you can't specify the registry used to create the image names. It will always be ko.local
.
Note
: You can build all local images at once by running
make ko-build-all
.
Building kyvernopre image locally
To build kyvernopre
image on your local system, run:
make ko-build-kyvernopre
The resulting image should be available locally, named ko.local/github.com/kyverno/kyverno/cmd/initcontainer
.
Building kyverno image locally
To build kyverno
image on your local system, run:
make ko-build-kyverno
The resulting image should be available locally, named ko.local/github.com/kyverno/kyverno/cmd/kyverno
.
Building cli image locally
To build cli
image on your local system, run:
make ko-build-cli
The resulting image should be available locally, named ko.local/github.com/kyverno/kyverno/cmd/cli/kubectl-kyverno
.
Pushing images
Pushing images is very similar to building local images, except that built images will be published on a remote image registry.
Currently, we are supporting two build systems:
Note
: We started with
docker
and are progressively moving toko
.
As the ko
based build system matures, we will deprecate and remove docker
based builds.
When pushing images you can specify the registry you want to publish images to by setting the REGISTRY
environment variable (default value is ghcr.io
).
Pushing images with docker
Authenticating to the remote registry is not done automatically in the Makefile
.
You need to be authenticated before invoking targets responsible for pushing images.
Note
: You can push all images at once by running
make docker-publish-all
ormake docker-publish-all-dev
.
Pushing kyvernopre image
To push kyvernopre
image on a remote registry, run:
# push stable image
make docker-publish-kyvernopre
or
# push dev image
make docker-publish-kyvernopre-dev
The resulting image should be available remotely, named ghcr.io/kyverno/kyvernopre
(by default, if REGISTRY
environment variable was not set).
Pushing kyverno image
To push kyverno
image on a remote registry, run:
# push stable image
make docker-publish-kyverno
or
# push dev image
make docker-publish-kyverno-dev
The resulting image should be available remotely, named ghcr.io/kyverno/kyverno
(by default, if REGISTRY
environment variable was not set).
Pushing cli image
To push cli
image on a remote registry, run:
# push stable image
make docker-publish-cli
or
# push dev image
make docker-publish-cli-dev
The resulting image should be available remotely, named ghcr.io/kyverno/kyverno-cli
(by default, if REGISTRY
environment variable was not set).
Pushing images with ko
Authenticating to the remote registry is done automatically in the Makefile
with ko login
.
To allow authentication you will need to set REGISTRY_USERNAME
and REGISTRY_PASSWORD
environment variables before invoking targets responsible for pushing images.
Note
: You can push all images at once by running
make ko-publish-all
ormake ko-publish-all-dev
.
Pushing kyvernopre image
To push kyvernopre
image on a remote registry, run:
# push stable image
make ko-publish-kyvernopre
or
# push dev image
make ko-publish-kyvernopre-dev
The resulting image should be available remotely, named ghcr.io/kyverno/kyvernopre
(by default, if REGISTRY
environment variable was not set).
Pushing kyverno image
To push kyverno
image on a remote registry, run:
# push stable image
make ko-publish-kyverno
or
# push dev image
make ko-publish-kyverno-dev
The resulting image should be available remotely, named ghcr.io/kyverno/kyverno
(by default, if REGISTRY
environment variable was not set).
Pushing cli image
To push cli
image on a remote registry, run:
# push stable image
make ko-publish-cli
or
# push dev image
make ko-publish-cli-dev
The resulting image should be available remotely, named ghcr.io/kyverno/kyverno-cli
(by default, if REGISTRY
environment variable was not set).
Deploying a local build
After building local images, it is often useful to deploy those images in a local cluster.
We use KinD to create local clusters easily.
Create a local cluster
If you already have a local KinD cluster running, you can skip this step.
To create a local KinD cluster, run:
make kind-create-cluster
You can override the k8s version by setting the KIND_IMAGE
environment variable (default value is kindest/node:v1.24.0
).
You can also override the KinD cluster name by setting the KIND_NAME
environment variable (default value is kind
).
Build and load local images
To build local images and load them on a local KinD cluster, run:
# build kyvernopre image and load it in KinD cluster
make kind-load-kyvernopre
or
# build kyverno image and load it in KinD cluster
make kind-load-kyverno
or
# build kyvernopre and kyverno images and load them in KinD cluster
make kind-load-all
You can override the KinD cluster name by setting the KIND_NAME
environment variable (default value is kind
).
Deploy with helm
To build local images, load them on a local KinD cluster, and deploy helm charts, run:
# build images, load them in KinD cluster and deploy kyverno helm chart
make kind-deploy-kyverno
or
# deploy kyverno-policies helm chart
make kind-deploy-kyverno-policies
or
# build images, load them in KinD cluster and deploy helm charts
make kind-deploy-all
This will build local images, load built images in every node of the KinD cluster, and deploy kyverno
and/or kyverno-policies
helm charts in the cluster (overriding image repositories and tags).
Note
: This actually uses
ko
to build local images.
You can override the KinD cluster name by setting the KIND_NAME
environment variable (default value is kind
).
Code generation
We are using code generation tools to create the following portions of code:
Note
: You can run
make codegen-all
to build all generated code at once.
Generating kubernetes API client
TODO
Generating API deep copy functions
Based on the APIs golang code definitions, you can generate the corresponding deep copy functions by running:
# generate all deep copy functions
make codegen-deepcopy-all
or
# generate kyverno deep copy functions
make codegen-deepcopy-kyverno
or
# generate policy reports deep copy functions
make codegen-deepcopy-report
This will output files named zz_generated.deepcopy.go
in every API package.
Generating CRD definitions
Based on the APIs golang code definitions, you can generate the corresponding CRDs manifests by running:
# generate all CRDs
make codegen-crds-all
or
# generate Kyverno CRDs
make codegen-crds-kyverno
or
# generate policy reports CRDs
make codegen-crds-report
This will output CRDs manifests /config/crds.
Building and publishing an image locally
First, make sure you install ko
Publishing to your local Docker daemon
Set the KO_DOCKER_REPO
environment variable to ko.local
:
KO_DOCKER_REPO=ko.local
Then build and publish an image:
ko build ./cmd/kyverno --preserve-import-paths
The image will be available locally as ko.local/github.com/kyverno/kyverno/cmd/kyverno
.
Publishing to a local KinD cluster
First, create your KinD cluster:
kind create cluster
Set the KO_DOCKER_REPO
environment variable to kind.local
:
KO_DOCKER_REPO=kind.local
Then build and publish an image:
ko build ./cmd/kyverno --preserve-import-paths
This will build and load the image into your KinD cluster as:
kind.local/github.com/kyverno/kyverno/cmd/kyverno
If you have multiple KinD clusters, or created them with a non-default name, set KIND_CLUSTER_NAME=<your-cluster-name>
.
Publishing to a remote registry
Set the KO_DOCKER_REPO
environment variable to the registry you'd like to push to:
For example:
KO_DOCKER_REPO=gcr.io/my-project/kyverno
KO_DOCKER_REPO=my-dockerhub-user/my-dockerhub-repo
KO_DOCKER_REPO=<ACCOUNTID>.dkr.ecr.<REGION>.amazonaws.com
Then build and publish an image:
ko build ./cmd/kyverno
The output will tell you the image name and digest of the image you just built.