Add support for management of Extended Resources via the
NodeFeatureRule CRD API.
There are usage scenarios where users want to advertise features
as extended resources instead of labels (or annotations).
This patch enables the discovery of extended resources, via annotation
and patch of node.status.capacity and node.status.allocatable. By using
the NodeFeatureRule API.
Co-authored-by: Carlos Eduardo Arango Gutierrez <eduardoa@nvidia.com>
Co-authored-by: Markus Lehtonen <markus.lehtonen@intel.com>
Co-authored-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Carlos Eduardo Arango Gutierrez <eduardoa@nvidia.com>
Disallow taints having a key with "kubernetes.io/" or "*.kubernetes.io/"
prefix. This is a precaution to protect the user from messing up with
the "official" well-known taints from Kubernetes itself. The only
exception is that the "nfd.node.kubernetes.io/" prefix is allowed.
However, there is one allowed NFD-specific namespace (and its
sub-namespaces) i.e. "feature.node.kubernetes.io" under the
kubernetes.io domain that can be used for NFD-managed taints.
Also disallow unprefixed taint keys. We don't add a default prefix to
unprefixed taints (like we do for labels) from NodeFeatureRules. This is
to prevent unpleasant surprises to users that need to manage matching
tolerations for their workloads.
Make the default master pod run with no special options. Move the
customizations of the master pod to the setup functions of the tests
that actually need it.
Also, cleanup the configuration of nfd-worker of some tests.
Wait for the deletion of NFD CRDs to complete before trying to re-create
them. Prevents errors in case CRDs already exist on the cluster when
e2e-tests are launched.
The node cleanup function was not removing all NFD-labels. It omitted
NFD-originated labels that used a non-default label namespace. This
patch fixes the issue by getting all NFD-managed labels from the special
annotation (nfd.node.kubernetes.io/feature-labels).
The patch also adds the ability to cleanup extended resources in a
similar way. This will be needed by future work.
Also changes the order of cleaning up CRs and the node. It is the right
order as cleaning up the CRs may still update the node.
Similar to the nfd-worker, in this PR we want to support the
dynamic run-time configurability through a config file for the nfd-master.
We'll use a json or yaml configuration file along with the fsnotify in
order to watch for changes in the config file. As a result, we're
allowing dynamic control of logging params, allowed namespaces,
extended resources, label whitelisting, and denied namespaces.
Signed-off-by: AhmedGrati <ahmedgrati1999@gmail.com>
Reduce the wait time of nfd-worker pods to be in ready-state (before
proceeding with tests) from five to two seconds. Make tests faster to
run. Two seconds should be enough for nfd-workers to do their job and
get nodes labeled.
The docker image that used during e2e test
composed of repo and tag flags that are
passed to the test itself.
The problem is that the docker image initialized
before the flags are parsed. Hence, it will always contains
the default flags value.
Moving the variable into a separate function, fixing the issue.
Also, moving the global variables to `e2e_test.go` since
it commonly used by all tests.
Signed-off-by: Talor Itzhak <titzhak@redhat.com>
Use the "single-dash" version of nfd command line flags in deployment
files and e2e-tests. No impact in functionality, just aligns with
documentation and other parts of the codebase.
Add an initial test set for the NodeFeature API. This is done simply by
running a second pass of the tests but with -enable-nodefeature-api
(i.e. NodeFeature API enabled and gRPC disabled). This should give basic
confidence that the API actually works and form a basis for further
imporovements on testing the new CRD API.
Drop the pod-security.kubernetes.io/enforce label from the test
namespace, i.e. remove pod security admission enforcement. NFD-worker
uses restricted host mounts (/sys) etc so pod creation fails even in
privileged mode if pod security admission enforcement is enabled.
Only generate CRDs once in the beginning of the test run. Use the "Ordered"
option for the test container so that we can utilize ginkgo.BeforeAll to
only do stuff once before the first test. Changing from unordered to
ordered shouldn't make a big difference here.
Add a cleanup function to remove stale NodeFeatureRule objects that are
cluster-scoped and not deleted with the test namespace.
Use RuntimeDefault seccomp profile in nfd worker and topology
updater pod spec similar to nfd master.
Signed-off-by: Feruzjon Muyassarov <feruzjon.muyassarov@intel.com>
After introducing NodeFeatureRule we packed two CRD definitions in one
yaml file. Our e2e-tests were not prepared to that and the file itself
was also renamed so it couldn't even be read by the test suite.
With this change the e2e-tests start to create NodeFeatre CRD in the
test cluster, preparing for the addition of e2e-tests for NodeFeature
API.
Drop the gRPC communication to nfd-master and connect to the Kubernetes
API server directly when updating NodeResourceTopology objects.
Topology-updater already has connection to the API server for listing
Pods so this is not that dramatic change. It also simplifies the code
a lot as there is no need for the NFD gRPC client and no need for
managing TLS certs/keys.
This change aligns nfd-topology-updater with the future direction of
nfd-worker where the gRPC API is being dropped and replaced by a
CRD-based API.
This patch also update deployment files and documentation to reflect
this change.
Fixes stricter API check on daemonset pod spec that started to cause e2e
test failures. RestartPolicyNever that we previously set (by defaylt)
isn't compatible with DaemonSets.
The new package should provide pod-related utilities,
hence let's move all the daemonset-related utilities
to their own package as well.
Signed-off-by: Talor Itzhak <titzhak@redhat.com>
By moving those utils in to a seperate package,
we can make the functions names shorter and clearer.
For example, instead of:
```
testutils.NFDWorkerPod(opts...)
testutils.NFDMasterPod(opts...)
testutils.SpecWithContainerImage(...)
```
we'll have:
```
testpod.NFDWorker(opts...)
testpod.NFDMaster(opts...)
testpod.SpecWithContainerImage(...)
```
It will also make the package more isolated and portable.
Signed-off-by: Talor Itzhak <titzhak@redhat.com>
The master pod need these `SecurityContext` configurations
In order to run inside a namespace with restricted policy
Signed-off-by: Talor Itzhak <titzhak@redhat.com>
