Refactor the code, moving the hostpath helper functionality to new
"pkg/utils/hostpath" package. This breaks odd-ish dependency
"pkg/utils" -> "source".
Use the same base image (golang) for auto-generation as for building
container images. This makes sure that we stay in sync in terms of the
golang version used.
In some cases (CI) it is useful to run NFD e2e tests using
ephemeral clusters. To save time and bandwidth, it is also useful
to prime the ephemeral cluster with the images under test.
In these circumstances there is no risk of running a stale image,
and having a `Always` PullPolicy hardcoded actually makes
the whole exercise null.
So we add a new option, disabled by default, to make the e2e
manifest use the `IfNotPresent` pull policy, to effectively
cover this use case.
Signed-off-by: Francesco Romani <fromani@redhat.com>
Run code auto-generation inside a container instead of the host system.
Our auto-generation depends on specific versions of a multitude of tools
(like k8s code-generator, controller-gen, protoc, mockery etc). This
made it really awkward (and error-prone) to run in the host environment,
especially if/when you needed different versions of those tools for
other projects. Making it even more unwieldy, the required versions of
tools were not neatly documented anywhere (except for git commits,
perhaps).
With this patch we have a "fixed environment", as we build a special
auto-generate-builder container which has correct versions of all the
dependencies. Using the container makes auto-generation easy to run
anywhere, independent of the host system, giving reproducibility and
reliability. Also, the patch moves the auto-generation steps out from
the makefile into a separate script, making the makefile cleaner and the
script easier to maintain.
Switch over to the "non-point-release" version of the image. Now we
always use the latest patch version of golang with latest security
fixes, for example, without the need to manually bump the version after
every point release.
This patch also makes the builder image configurable through a Makefile
variable.
For reproducible builds we should used fixed point-release versions in
release-brances.
Add auto-generated code for interfacing our CRD API. On top of this, a
CR controller can be implemented. This patch uses k8s/code-generator
for code generation. Run "make generate" in order to (re-)generate
everything. Path to the code-generator repository may need to be
specified:
K8S_CODE_GENERATOR=path/to/code-generator make apigen
Code-generator version 0.20.7 was used to create this patch. Install
k8s code-generator tools and clone the repo with:
git clone https://github.com/kubernetes/code-generator -b v0.20.7 <path/to/code-generator>
go install k8s.io/code-generator/cmd/...(at)v0.20.7
Add a cluster-scoped Custom Resource Definition for specifying labeling
rules. Nodes (node features, node objects) are cluster-level objects and
thus the natural and encouraged setup is to only have one NFD deployment
per cluster - the set of underlying features of the node stays the same
independent of how many parallel NFD deployments you have. Our extension
points (hooks, feature files and now CRs) can be be used by multiple
actors (depending on us) simultaneously. Having the CRD cluster-scoped
hopefully drives deployments in this direction. It also should make
deployment of vendor-specific labeling rules easy as there is no need to
worry about the namespace.
This patch virtually replicates the source.custom.FeatureSpec in a CRD
API (located in the pkg/apis/nfd/v1alpha1 package) with the notable
exception that "MatchOn" legacy rules are not supported. Legacy rules
are left out in order to keep the CRD simple and clean.
The duplicate functionality in source/custom will be dropped by upcoming
patches.
This patch utilizes controller-gen (from sigs.k8s.io/controller-tools)
for generating the CRD and deepcopy methods. Code can be (re-)generated
with "make generate". Install controller-gen with:
go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.7.0
Update kustomize and helm deployments to deploy the CRD.
Use 'go generate' for auto-generating code. Drop the old 'mock' and
'apigen' makefile targets. Those are replaced with a single
make generate
which (re-)generates everything.
There have been recent changes made to the noderesourcetopology API
storing the proto file generated using go-to-protobuf tool and
this code inports the proto generated in the API in the topology-updater.proto
The PRs corresponding to the changes are as follows:
https://github.com/k8stopologyawareschedwg/noderesourcetopology-api/pull/9https://github.com/k8stopologyawareschedwg/noderesourcetopology-api/pull/13
Commands used to generate topology-updater.pb.go file:
go install github.com/golang/protobuf/protoc-gen-go@v1.4.3
go mod vendor
protoc --go_opt=paths=source_relative --go_out=plugins=grpc:. pkg/topologyupdater/topology-updater.proto -I. -Ivendor
As part of implmentation of this patch, reserved (non-allocatable) CPUs
are evaluated by performing a difference between all the CPUs on a system
(determined by using ghw) and allocatable CPUs (determined by querying
GetAllocatableResources podResource API endpoint).
When aggregator creates the NUMA zones, it will skip the zone creation if
there are no allocatable resources. In this update we creates those missing
zone with zero allocatable/available resources so we won't have holes in the
array of reported zones.
Co-Authored-by: Talor Itzhak <titzhak@redhat.com>
Signed-off-by: Swati Sehgal <swsehgal@redhat.com>
Setup the topologyupdater API for gRPC communication of
nfd-topology-updater with master
We generate pb.go file to reflect latest dependency changes
using github.com/golang/protobuf/protoc-gen-go and generate
grpc files via:
`protoc pkg/topologyupdater/topology-updater.proto --go_out=plugins=grpc:.`
Please refer to: https://github.com/k8stopologyawareschedwg/noderesourcetopology-api/blob/master/pkg/apis/topology/v1alpha1/types.go
Co-Authored-by: Artyom Lukianov <alukiano@redhat.com>
Co-Authored-by: Francesco Romani <fromani@redhat.com>
Signed-off-by: Swati Sehgal <swsehgal@redhat.com>
Implement new registration infrastructure under the "source" package.
This change loosens the coupling between label sources and the
nfd-worker, making it easier to refactor and move the code around.
Also, create a separate interface (ConfigurableSource) for configurable
feature sources in order to eliminate boilerplate code.
Add safety checks to the sources that they actually implement the
interfaces they should.
In sake of consistency and predictability (of behavior) change all
methods of the sources to use pointer receivers.
Add simple unit tests for the new functionality and include source/...
into make test target.
Implement functionality virtually replicating deployment templates for
nfd-master and nfd-worker daemonset (nfd-master.yaml.template and
nfd-worker-daemonset.yaml.template) by adding a kustomize overlay named
"default".
We split the resources into multiple bases (rbac, master and
worker-daemonset) so that relevant parts are re-usable in
other deployment scenarios added later (e.g. "one-shot job", and
"combined daemonset").
This patch adds one component (components/common) doing the required
kustomization for the example deployment.
For auto-generating api(s).
Also, re-generate/refresh the gRPC with `make apigen` (with protoc
v3.17.3 and protoc-gen-go from github.com/golang/protobuf v1.5.2) to
sync up things.
* Add support for configurable runtime full and minimal images.
* Fixups and renamings.
* Change variables *_IMG_* to *_IMAGE_*
* Fix args in Dockerfile also.
cert-manager can be used to automate TLS certificate management for
nfd-master and the nfd-worker pod(s).
Add a template to deploy cert-manager CA Issuer and Certificates and
document steps how to use them.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Build a "minimal" variant of the nfd image based on
gcr.io/distroless/base. The motivations behind the minimal image are
image hardening (security) and reducing the image footprint (from ca.
108MB down to about 40MB).
The practical effect of deploying the minimal image is that no runtimes
for running worker hooks are present, not even a shell. This means that
only statically linked linked hook binaries are supported. Also, because
of the image hardening live debugging of the minimal image by attaching
to the container is not possible, and, the "full" image needs to be used
for that purpose.
This commit adds Helm chart for node-feature-discovery
Signed-off-by: Adrian Chiris <adrianc@nvidia.com>
Signed-off-by: Ivan Kolodiazhnyi <ikolodiazhny@nvidia.com>
Rely on markers instead which makes the update work also in the middle
of a template file. Previously it relied on the configuration data to be
at the very end of the file to be updated.
This Patch adds a check on make target e2e-test, to check if the
KUBECONFIG env var is set before running the test suite.
if not pressent return a valid error message.
Signed-off-by: Carlos Eduardo Arango Gutierrez <carangog@redhat.com>
Add a virtually empty ConfigMap that is mounted inside the workers.
Makes it easier to start customizing the worker deployment e.g. with just:
$ kubectl -n ${NFD_NS} edit configmap nfd-worker-conf
Create a new 'templates' make target for inserting the content of
nfd-worker.conf.example into the configmap spec of the templates. Thus,
'make templates' should be run whenever the example config is update.
Update the verify.sh prow script to check that the templates are up to
date.
This patch also streamlines the documentation about configuration
management, reflecting the changes.
Change build configuration related to site baseurl and Jekyll output
directory a bit. Making it more coherent e.g. by removing the need to
override baseurl for 'site-serve' makefile target.
Only show the dynamic version menu in production environment where it's
available. Local (development) builds only show a stub version menu with
one item.
Also, fix the theme customization. Append version list items to the
correct element in html so that they are displayed correctly. Drop
unnecessary <div/>.
Add rules for building and serving the html documentation inside a
container.
- 'make site-build' will build the html content unders docs/_site.
- 'make site-serve' builds the site content and serves it at
http://localhost:4000/
Change the default namespace used in deployment yamls from 'kube-system'
to 'node-feature-discovery'. Sync with what the templates have. Also,
this should be a safer and more desirable default for people just trying
out node-feature-discovery.
An attempt to organize the Makefile variables a bit further. The idea is
to only document variables that are really intended for build
customization and initialize all these variables in the Makefile with
'=?' so that they can be specified in the environment (and not just with
variable overrides). Thus, this patch drops some variables from the
README and change the initialization of some variables in the Makefile
so that the two files would be in sync.
make image and make push fails when trying to use an ENV VAR value, as
it is now they can only be overrided.
the Var modified are
GO_CMD
GO_FMT
IMAGE_BUILD_EXTRA_OPTS
IMAGE_BUILD_CMD
IMAGE_PUSH_CMD
IMAGE_TAG_NAME
IMAGE_REGISTRY
IMAGE_NAME
IMAGE_EXTRA_TAG_NAMES
K8S_NAMESPACE
HOSTMOUNT_PREFIX
This new behaviour has also been updated on the README.
Signed-off-by: Carlos Eduardo Arango Gutierrez <carangog@redhat.com>
