Add auto-generated code for interfacing our CRD API. On top of this, a
CR controller can be implemented. This patch uses k8s/code-generator
for code generation. Run "make generate" in order to (re-)generate
everything. Path to the code-generator repository may need to be
specified:
K8S_CODE_GENERATOR=path/to/code-generator make apigen
Code-generator version 0.20.7 was used to create this patch. Install
k8s code-generator tools and clone the repo with:
git clone https://github.com/kubernetes/code-generator -b v0.20.7 <path/to/code-generator>
go install k8s.io/code-generator/cmd/...(at)v0.20.7
Add a cluster-scoped Custom Resource Definition for specifying labeling
rules. Nodes (node features, node objects) are cluster-level objects and
thus the natural and encouraged setup is to only have one NFD deployment
per cluster - the set of underlying features of the node stays the same
independent of how many parallel NFD deployments you have. Our extension
points (hooks, feature files and now CRs) can be be used by multiple
actors (depending on us) simultaneously. Having the CRD cluster-scoped
hopefully drives deployments in this direction. It also should make
deployment of vendor-specific labeling rules easy as there is no need to
worry about the namespace.
This patch virtually replicates the source.custom.FeatureSpec in a CRD
API (located in the pkg/apis/nfd/v1alpha1 package) with the notable
exception that "MatchOn" legacy rules are not supported. Legacy rules
are left out in order to keep the CRD simple and clean.
The duplicate functionality in source/custom will be dropped by upcoming
patches.
This patch utilizes controller-gen (from sigs.k8s.io/controller-tools)
for generating the CRD and deepcopy methods. Code can be (re-)generated
with "make generate". Install controller-gen with:
go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.7.0
Update kustomize and helm deployments to deploy the CRD.
Use 'go generate' for auto-generating code. Drop the old 'mock' and
'apigen' makefile targets. Those are replaced with a single
make generate
which (re-)generates everything.
There have been recent changes made to the noderesourcetopology API
storing the proto file generated using go-to-protobuf tool and
this code inports the proto generated in the API in the topology-updater.proto
The PRs corresponding to the changes are as follows:
https://github.com/k8stopologyawareschedwg/noderesourcetopology-api/pull/9https://github.com/k8stopologyawareschedwg/noderesourcetopology-api/pull/13
Commands used to generate topology-updater.pb.go file:
go install github.com/golang/protobuf/protoc-gen-go@v1.4.3
go mod vendor
protoc --go_opt=paths=source_relative --go_out=plugins=grpc:. pkg/topologyupdater/topology-updater.proto -I. -Ivendor
As part of implmentation of this patch, reserved (non-allocatable) CPUs
are evaluated by performing a difference between all the CPUs on a system
(determined by using ghw) and allocatable CPUs (determined by querying
GetAllocatableResources podResource API endpoint).
When aggregator creates the NUMA zones, it will skip the zone creation if
there are no allocatable resources. In this update we creates those missing
zone with zero allocatable/available resources so we won't have holes in the
array of reported zones.
Co-Authored-by: Talor Itzhak <titzhak@redhat.com>
Signed-off-by: Swati Sehgal <swsehgal@redhat.com>
Setup the topologyupdater API for gRPC communication of
nfd-topology-updater with master
We generate pb.go file to reflect latest dependency changes
using github.com/golang/protobuf/protoc-gen-go and generate
grpc files via:
`protoc pkg/topologyupdater/topology-updater.proto --go_out=plugins=grpc:.`
Please refer to: https://github.com/k8stopologyawareschedwg/noderesourcetopology-api/blob/master/pkg/apis/topology/v1alpha1/types.go
Co-Authored-by: Artyom Lukianov <alukiano@redhat.com>
Co-Authored-by: Francesco Romani <fromani@redhat.com>
Signed-off-by: Swati Sehgal <swsehgal@redhat.com>
Implement new registration infrastructure under the "source" package.
This change loosens the coupling between label sources and the
nfd-worker, making it easier to refactor and move the code around.
Also, create a separate interface (ConfigurableSource) for configurable
feature sources in order to eliminate boilerplate code.
Add safety checks to the sources that they actually implement the
interfaces they should.
In sake of consistency and predictability (of behavior) change all
methods of the sources to use pointer receivers.
Add simple unit tests for the new functionality and include source/...
into make test target.
Implement functionality virtually replicating deployment templates for
nfd-master and nfd-worker daemonset (nfd-master.yaml.template and
nfd-worker-daemonset.yaml.template) by adding a kustomize overlay named
"default".
We split the resources into multiple bases (rbac, master and
worker-daemonset) so that relevant parts are re-usable in
other deployment scenarios added later (e.g. "one-shot job", and
"combined daemonset").
This patch adds one component (components/common) doing the required
kustomization for the example deployment.
For auto-generating api(s).
Also, re-generate/refresh the gRPC with `make apigen` (with protoc
v3.17.3 and protoc-gen-go from github.com/golang/protobuf v1.5.2) to
sync up things.
* Add support for configurable runtime full and minimal images.
* Fixups and renamings.
* Change variables *_IMG_* to *_IMAGE_*
* Fix args in Dockerfile also.
cert-manager can be used to automate TLS certificate management for
nfd-master and the nfd-worker pod(s).
Add a template to deploy cert-manager CA Issuer and Certificates and
document steps how to use them.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Build a "minimal" variant of the nfd image based on
gcr.io/distroless/base. The motivations behind the minimal image are
image hardening (security) and reducing the image footprint (from ca.
108MB down to about 40MB).
The practical effect of deploying the minimal image is that no runtimes
for running worker hooks are present, not even a shell. This means that
only statically linked linked hook binaries are supported. Also, because
of the image hardening live debugging of the minimal image by attaching
to the container is not possible, and, the "full" image needs to be used
for that purpose.
This commit adds Helm chart for node-feature-discovery
Signed-off-by: Adrian Chiris <adrianc@nvidia.com>
Signed-off-by: Ivan Kolodiazhnyi <ikolodiazhny@nvidia.com>
Rely on markers instead which makes the update work also in the middle
of a template file. Previously it relied on the configuration data to be
at the very end of the file to be updated.
This Patch adds a check on make target e2e-test, to check if the
KUBECONFIG env var is set before running the test suite.
if not pressent return a valid error message.
Signed-off-by: Carlos Eduardo Arango Gutierrez <carangog@redhat.com>
Add a virtually empty ConfigMap that is mounted inside the workers.
Makes it easier to start customizing the worker deployment e.g. with just:
$ kubectl -n ${NFD_NS} edit configmap nfd-worker-conf
Create a new 'templates' make target for inserting the content of
nfd-worker.conf.example into the configmap spec of the templates. Thus,
'make templates' should be run whenever the example config is update.
Update the verify.sh prow script to check that the templates are up to
date.
This patch also streamlines the documentation about configuration
management, reflecting the changes.
Change build configuration related to site baseurl and Jekyll output
directory a bit. Making it more coherent e.g. by removing the need to
override baseurl for 'site-serve' makefile target.
Only show the dynamic version menu in production environment where it's
available. Local (development) builds only show a stub version menu with
one item.
Also, fix the theme customization. Append version list items to the
correct element in html so that they are displayed correctly. Drop
unnecessary <div/>.
Add rules for building and serving the html documentation inside a
container.
- 'make site-build' will build the html content unders docs/_site.
- 'make site-serve' builds the site content and serves it at
http://localhost:4000/
Change the default namespace used in deployment yamls from 'kube-system'
to 'node-feature-discovery'. Sync with what the templates have. Also,
this should be a safer and more desirable default for people just trying
out node-feature-discovery.
An attempt to organize the Makefile variables a bit further. The idea is
to only document variables that are really intended for build
customization and initialize all these variables in the Makefile with
'=?' so that they can be specified in the environment (and not just with
variable overrides). Thus, this patch drops some variables from the
README and change the initialization of some variables in the Makefile
so that the two files would be in sync.
make image and make push fails when trying to use an ENV VAR value, as
it is now they can only be overrided.
the Var modified are
GO_CMD
GO_FMT
IMAGE_BUILD_EXTRA_OPTS
IMAGE_BUILD_CMD
IMAGE_PUSH_CMD
IMAGE_TAG_NAME
IMAGE_REGISTRY
IMAGE_NAME
IMAGE_EXTRA_TAG_NAMES
K8S_NAMESPACE
HOSTMOUNT_PREFIX
This new behaviour has also been updated on the README.
Signed-off-by: Carlos Eduardo Arango Gutierrez <carangog@redhat.com>
Add a new script for running end-to-end tests. Implement a Makefile rule
for polling the container image. We need this for prow as there is no
other way for syncing the jobs (e2e postsubmit job must wait until the
image-pushing job has built and published the image). This is
implemented in the Makefile as all the image-related variables are also
defined/available there.
Also, adapt the TravisCI config to use the new script.
Introduce a new makefile variable and dockerfile build arg named
HOSTMOUNT_PREFIX for changing the prefix used for mounting host
directories inside the container. Adds a sed rule in Makefile to
generate deployment yamls accordingly.
Implement an end-to-end test with all feature sources enabled. The new
test runs nfd-worker as a daemonset on all (schedulable) nodes of the
test cluster which makes it possible to cover a wide range features,
assuming the test cluster is heterogenous containing nodes with varying
system configurations.
The features available depends on the node(s) the e2e testa are run on.
Thus, some runtime parameterization of the tests is needed. The patch
adds a new command line test flag 'nfd.e2e-config' that is used to
specify the per-node feature labels and annotations that is expected to
be present in the cluster. An example configuration file is provided
with the patch. The pod spec of nfd-worker deployment is changed to
better correspond the default deployment and thus enable wider feature
discovery. This means using hostnetwork and adding mounts for /sys /boot
and /etc/os-release.
The patch changes node object management so that all nfd-related labels
are removed after each test (not just the ones the test is expected to
add). Also, all nfd-related annotations are now removed.
For running end-to-end tests it is required to explicitly specify in the
command line the kubeconfig to be used, e.g.:
$ make e2e-test KUBECONFIG=$HOME/.kube/config
