mirror of
https://github.com/kubernetes-sigs/node-feature-discovery.git
synced 2025-03-18 06:18:20 +00:00
Merge pull request #1778 from marquiz/devel/nf-api-ga
feature-gates: mark NodeFeatureAPI as GA
This commit is contained in:
commit
e30cf85f61
10 changed files with 728 additions and 822 deletions
|
@ -17,6 +17,7 @@ The feature gates are set using the `-feature-gates` command line flag or
|
|||
| Name | Default | Stage | Since | Until |
|
||||
| --------------------- | ------- | ------ | ------- | ------ |
|
||||
| `NodeFeatureAPI` | true | Beta | V0.14 | |
|
||||
| `NodeFeatureAPI` | true | GA | V0.17 | |
|
||||
| `DisableAutoPrefix` | false | Alpha | V0.16 | |
|
||||
| `NodeFeatureGroupAPI` | false | Alpha | V0.16 | |
|
||||
|
||||
|
|
|
@ -39,7 +39,7 @@ documentation](../feature-gates.md).
|
|||
Example:
|
||||
|
||||
```bash
|
||||
nfd-master -feature-gates NodeFeatureAPI=false
|
||||
nfd-master -feature-gates NodeFeatureGroupAPI=true
|
||||
```
|
||||
|
||||
### -prune
|
||||
|
@ -181,9 +181,6 @@ The `-enable-leader-election` flag enables leader election for NFD-Master.
|
|||
It is advised to turn on this flag when running more than one instance of
|
||||
NFD-Master.
|
||||
|
||||
Does not have effect if the [NodeFeatureAPI](feature-gates.md#nodefeatureapi)
|
||||
feature gate is disabled.
|
||||
|
||||
Default: false
|
||||
|
||||
```bash
|
||||
|
@ -347,9 +344,6 @@ nfd-master -options='{"noPublish": true}'
|
|||
The `-nfd-api-parallelism` flag can be used to specify the maximum
|
||||
number of concurrent node updates.
|
||||
|
||||
Does not have effect if the [NodeFeatureAPI](feature-gates.md#nodefeatureapi)
|
||||
feature gate is disabled.
|
||||
|
||||
Default: 10
|
||||
|
||||
Example:
|
||||
|
@ -443,9 +437,6 @@ The resync means nfd-master replaying all NodeFeature and NodeFeatureRule object
|
|||
thus effectively re-syncing all nodes in the cluster (i.e. ensuring labels, annotations,
|
||||
extended resources and taints are in place).
|
||||
|
||||
Does not have effect if the [NodeFeatureAPI](feature-gates.md#nodefeatureapi)
|
||||
feature gate is disabled.
|
||||
|
||||
Default: 1 hour.
|
||||
|
||||
Example:
|
||||
|
|
|
@ -156,9 +156,6 @@ The resync means nfd-master replaying all NodeFeature and NodeFeatureRule object
|
|||
thus effectively re-syncing all nodes in the cluster (i.e. ensuring labels, annotations,
|
||||
extended resources and taints are in place).
|
||||
|
||||
Does not have effect if the [NodeFeatureAPI](feature-gates.md#nodefeatureapi)
|
||||
feature gate is disabled.
|
||||
|
||||
Default: 1 hour.
|
||||
|
||||
Example:
|
||||
|
@ -231,9 +228,6 @@ leaderElection:
|
|||
The `nfdApiParallelism` option can be used to specify the maximum
|
||||
number of concurrent node updates.
|
||||
|
||||
Does not have effect if the [NodeFeatureAPI](feature-gates.md#nodefeatureapi)
|
||||
feature gate is disabled.
|
||||
|
||||
Default: 10
|
||||
|
||||
Example:
|
||||
|
|
|
@ -39,7 +39,7 @@ documentation](../feature-gates.md).
|
|||
Example:
|
||||
|
||||
```bash
|
||||
nfd-master -feature-gates NodeFeatureAPI=false
|
||||
nfd-master -feature-gates NodeFeatureGroupAPI=true
|
||||
```
|
||||
|
||||
### -config
|
||||
|
@ -153,9 +153,6 @@ Kubernetes API server. It is needed for manipulating
|
|||
[NodeFeature](../usage/custom-resources.md#nodefeature) objects. An empty value
|
||||
(which is also the default) implies in-cluster kubeconfig.
|
||||
|
||||
Does not have effect if the [NodeFeatureAPI](feature-gates.md#nodefeatureapi)
|
||||
feature gate is disabled.
|
||||
|
||||
Default: *empty*
|
||||
|
||||
Example:
|
||||
|
|
|
@ -40,14 +40,6 @@ labels directly.
|
|||
Note that RBAC rules must be created for each extension for them to be able to
|
||||
create and manipulate NodeFeature objects in their namespace.
|
||||
|
||||
The NodeFeature CRD API can be disabled with the
|
||||
[NodeFeatureAPI](../reference/feature-gates.md#nodefeatureapi) feature gate
|
||||
(`-feature-gates NodeFeatureAPI=false` on command line). The `-feature-gates`
|
||||
command line flag must be specified for both nfd-master and nfd-worker as it
|
||||
will enable the gRPC communication between them. Note that the gRPC API is
|
||||
**DEPRECATED** and will be removed in a future release, at which point the
|
||||
NodeFeature API cannot be disabled.
|
||||
|
||||
### A NodeFeature example
|
||||
|
||||
Consider the following referential example:
|
||||
|
|
|
@ -24,7 +24,6 @@ default garbage collector interval is set to 1h which is the value when no
|
|||
|
||||
## Configuration
|
||||
|
||||
In Helm deployments (see
|
||||
[garbage collector parameters](../deployment/helm.md#garbage-collector-parameters))
|
||||
NFD-GC will only be deployed when `featureGates.NodeFeatureAPI` or
|
||||
`topologyUpdater.enable` is set to true.
|
||||
In Helm deployments see
|
||||
[garbage collector parameters](../deployment/helm.md#garbage-collector-parameters)
|
||||
for altering the nfd-gc configuration.
|
||||
|
|
|
@ -30,20 +30,6 @@ and creates node labels accordingly. The feature data used as the input is
|
|||
received from nfd-worker instances through
|
||||
[NodeFeature](custom-resources.md#nodefeature-custom-resource) objects.
|
||||
|
||||
> **NOTE**: when gRPC (**DEPRECATED**) is used for communicating
|
||||
> the features (by setting the flag `-feature-gates NodeFeatureAPI=false` on both
|
||||
> nfd-master and nfd-worker, or via Helm values.featureGates.NodeFeatureAPI=false),
|
||||
> (re-)labelling only happens when a request is received from nfd-worker.
|
||||
> That is, in practice rules are evaluated and labels for each node are created
|
||||
> on intervals specified by the
|
||||
> [`core.sleepInterval`](../reference/worker-configuration-reference.md#coresleepinterval)
|
||||
> configuration option of nfd-worker instances. This means that modification or
|
||||
> creation of NodeFeatureRule objects does not instantly cause the node
|
||||
> labels to be updated. Instead, the changes only come visible in node labels
|
||||
> as nfd-worker instances send their labelling requests. This limitation is not
|
||||
> present when gRPC interface is disabled
|
||||
> and [NodeFeature](custom-resources.md#nodefeature-custom-resource) API is used.
|
||||
|
||||
## Master configuration
|
||||
|
||||
NFD-Master supports dynamic configuration through a configuration file. The
|
||||
|
|
|
@ -35,7 +35,7 @@ var (
|
|||
)
|
||||
|
||||
var DefaultNFDFeatureGates = map[featuregate.Feature]featuregate.FeatureSpec{
|
||||
NodeFeatureAPI: {Default: true, PreRelease: featuregate.Beta},
|
||||
NodeFeatureAPI: {Default: true, PreRelease: featuregate.GA, LockToDefault: true},
|
||||
DisableAutoPrefix: {Default: false, PreRelease: featuregate.Alpha},
|
||||
NodeFeatureGroupAPI: {Default: false, PreRelease: featuregate.Alpha},
|
||||
}
|
||||
|
|
|
@ -208,7 +208,10 @@ func TestRun(t *testing.T) {
|
|||
})
|
||||
}
|
||||
|
||||
// TODO: remove this test with the rest of the TLS code and docs.
|
||||
// Also drop the certs from test/data/.
|
||||
func TestRunTls(t *testing.T) {
|
||||
t.Skip("gRPC cannot be enabled, NodeFeatureAPI GA")
|
||||
masterArgs := &master.Args{
|
||||
CaFile: data.FilePath("ca.crt"),
|
||||
CertFile: data.FilePath("nfd-test-master.crt"),
|
||||
|
|
|
@ -201,14 +201,6 @@ func cleanupCRs(ctx context.Context, cli *nfdclient.Clientset, namespace string)
|
|||
var _ = NFDDescribe(Label("nfd-master"), func() {
|
||||
f := framework.NewDefaultFramework("node-feature-discovery")
|
||||
|
||||
nfdTestSuite := func(useNodeFeatureApi bool) {
|
||||
createPodSpecOpts := func(opts ...testpod.SpecOption) []testpod.SpecOption {
|
||||
if !useNodeFeatureApi {
|
||||
return append(opts, testpod.SpecWithContainerExtraArgs("-feature-gates", "NodeFeatureAPI=false"))
|
||||
}
|
||||
return opts
|
||||
}
|
||||
|
||||
Context("when deploying a single nfd-master pod", Ordered, func() {
|
||||
var (
|
||||
crds []*apiextensionsv1.CustomResourceDefinition
|
||||
|
@ -219,13 +211,8 @@ var _ = NFDDescribe(Label("nfd-master"), func() {
|
|||
|
||||
checkNodeFeatureObject := func(ctx context.Context, name string) {
|
||||
_, err := nfdClient.NfdV1alpha1().NodeFeatures(f.Namespace.Name).Get(ctx, name, metav1.GetOptions{})
|
||||
if useNodeFeatureApi {
|
||||
By(fmt.Sprintf("Check that NodeFeature object for the node %q was created", name))
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
} else {
|
||||
By(fmt.Sprintf("Check that NodeFeature object for the node %q hasn't been created", name))
|
||||
Expect(err).To(HaveOccurred())
|
||||
}
|
||||
}
|
||||
|
||||
BeforeAll(func(ctx context.Context) {
|
||||
|
@ -264,10 +251,7 @@ var _ = NFDDescribe(Label("nfd-master"), func() {
|
|||
|
||||
// Launch nfd-master
|
||||
By("Creating nfd master pod and nfd-master service")
|
||||
podSpecOpts := createPodSpecOpts(
|
||||
append(extraMasterPodSpecOpts,
|
||||
testpod.SpecWithContainerImage(dockerImage()),
|
||||
)...)
|
||||
podSpecOpts := append(extraMasterPodSpecOpts, testpod.SpecWithContainerImage(dockerImage()))
|
||||
|
||||
masterPod := e2epod.NewPodClient(f).CreateSync(ctx, testpod.NFDMaster(podSpecOpts...))
|
||||
|
||||
|
@ -299,11 +283,11 @@ var _ = NFDDescribe(Label("nfd-master"), func() {
|
|||
|
||||
// Launch nfd-worker
|
||||
By("Creating a nfd worker pod")
|
||||
podSpecOpts := createPodSpecOpts(
|
||||
podSpecOpts := []testpod.SpecOption{
|
||||
testpod.SpecWithRestartPolicy(corev1.RestartPolicyNever),
|
||||
testpod.SpecWithContainerImage(dockerImage()),
|
||||
testpod.SpecWithContainerExtraArgs("-oneshot", "-label-sources=fake"),
|
||||
)
|
||||
}
|
||||
workerPod := testpod.NFDWorker(podSpecOpts...)
|
||||
workerPod, err = f.ClientSet.CoreV1().Pods(f.Namespace.Name).Create(ctx, workerPod, metav1.CreateOptions{})
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
@ -330,11 +314,9 @@ var _ = NFDDescribe(Label("nfd-master"), func() {
|
|||
err = f.ClientSet.CoreV1().Pods(f.Namespace.Name).Delete(ctx, workerPod.Name, metav1.DeleteOptions{})
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
if useNodeFeatureApi {
|
||||
By("Verify that labels from nfd-worker are garbage-collected")
|
||||
delete(expectedLabels, workerPod.Spec.NodeName)
|
||||
eventuallyNonControlPlaneNodes(ctx, f.ClientSet).WithTimeout(1 * time.Minute).Should(MatchLabels(expectedLabels, nodes))
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
|
@ -355,9 +337,7 @@ var _ = NFDDescribe(Label("nfd-master"), func() {
|
|||
fConf := cfg.DefaultFeatures
|
||||
|
||||
By("Creating nfd-worker daemonset")
|
||||
podSpecOpts := createPodSpecOpts(
|
||||
testpod.SpecWithContainerImage(dockerImage()),
|
||||
)
|
||||
podSpecOpts := []testpod.SpecOption{testpod.SpecWithContainerImage(dockerImage())}
|
||||
workerDS := testds.NFDWorker(podSpecOpts...)
|
||||
workerDS, err = f.ClientSet.AppsV1().DaemonSets(f.Namespace.Name).Create(ctx, workerDS, metav1.CreateOptions{})
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
@ -490,12 +470,12 @@ var _ = NFDDescribe(Label("nfd-master"), func() {
|
|||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
By("Creating nfd-worker daemonset with configmap mounted")
|
||||
podSpecOpts := createPodSpecOpts(
|
||||
podSpecOpts := []testpod.SpecOption{
|
||||
testpod.SpecWithContainerImage(dockerImage()),
|
||||
testpod.SpecWithContainerExtraArgs("-label-sources=custom"),
|
||||
testpod.SpecWithConfigMap(cm1.Name, filepath.Join(custom.Directory, "cm1")),
|
||||
testpod.SpecWithConfigMap(cm2.Name, filepath.Join(custom.Directory, "cm2")),
|
||||
)
|
||||
}
|
||||
workerDS := testds.NFDWorker(podSpecOpts...)
|
||||
|
||||
workerDS, err = f.ClientSet.AppsV1().DaemonSets(f.Namespace.Name).Create(ctx, workerDS, metav1.CreateOptions{})
|
||||
|
@ -518,11 +498,9 @@ var _ = NFDDescribe(Label("nfd-master"), func() {
|
|||
err = f.ClientSet.AppsV1().DaemonSets(f.Namespace.Name).Delete(ctx, workerDS.Name, metav1.DeleteOptions{})
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
if useNodeFeatureApi {
|
||||
By("Verify that labels from nfd-worker are garbage-collected")
|
||||
delete(expectedLabels, targetNodeName)
|
||||
eventuallyNonControlPlaneNodes(ctx, f.ClientSet).WithTimeout(1 * time.Minute).Should(MatchLabels(expectedLabels, nodes))
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
|
@ -539,10 +517,6 @@ var _ = NFDDescribe(Label("nfd-master"), func() {
|
|||
}
|
||||
})
|
||||
It("labels from the NodeFeature objects should be created", Label("nfd-worker"), func(ctx context.Context) {
|
||||
if !useNodeFeatureApi {
|
||||
Skip("NodeFeature API not enabled")
|
||||
}
|
||||
|
||||
// We pick one node targeted for our NodeFeature objects
|
||||
nodes, err := getNonControlPlaneNodes(ctx, f.ClientSet)
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
@ -573,10 +547,10 @@ var _ = NFDDescribe(Label("nfd-master"), func() {
|
|||
eventuallyNonControlPlaneNodes(ctx, f.ClientSet).Should(MatchLabels(expectedLabels, nodes))
|
||||
|
||||
By("Creating nfd-worker daemonset")
|
||||
podSpecOpts := createPodSpecOpts(
|
||||
podSpecOpts := []testpod.SpecOption{
|
||||
testpod.SpecWithContainerImage(dockerImage()),
|
||||
testpod.SpecWithContainerExtraArgs("-label-sources=fake"),
|
||||
)
|
||||
}
|
||||
workerDS := testds.NFDWorker(podSpecOpts...)
|
||||
workerDS, err = f.ClientSet.AppsV1().DaemonSets(f.Namespace.Name).Create(ctx, workerDS, metav1.CreateOptions{})
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
@ -632,10 +606,6 @@ var _ = NFDDescribe(Label("nfd-master"), func() {
|
|||
})
|
||||
|
||||
It("denied labels should not be created by the NodeFeature object", func(ctx context.Context) {
|
||||
if !useNodeFeatureApi {
|
||||
Skip("NodeFeature API not enabled")
|
||||
}
|
||||
|
||||
nodes, err := getNonControlPlaneNodes(ctx, f.ClientSet)
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
|
@ -714,11 +684,11 @@ core:
|
|||
cm, err = f.ClientSet.CoreV1().ConfigMaps(f.Namespace.Name).Create(ctx, cm, metav1.CreateOptions{})
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
By("Creating nfd-worker daemonset")
|
||||
podSpecOpts := createPodSpecOpts(
|
||||
podSpecOpts := []testpod.SpecOption{
|
||||
testpod.SpecWithContainerImage(dockerImage()),
|
||||
testpod.SpecWithConfigMap(cm.Name, "/etc/kubernetes/node-feature-discovery"),
|
||||
testpod.SpecWithTolerations(testTolerations),
|
||||
)
|
||||
}
|
||||
workerDS := testds.NFDWorker(podSpecOpts...)
|
||||
workerDS, err = f.ClientSet.AppsV1().DaemonSets(f.Namespace.Name).Create(ctx, workerDS, metav1.CreateOptions{})
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
@ -877,35 +847,27 @@ core:
|
|||
err = f.ClientSet.AppsV1().DaemonSets(f.Namespace.Name).Delete(ctx, workerDS.Name, metav1.DeleteOptions{})
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
if useNodeFeatureApi {
|
||||
By("Verify that labels from nfd-worker are garbage-collected")
|
||||
expectedLabels = map[string]k8sLabels{
|
||||
"*": {},
|
||||
}
|
||||
eventuallyNonControlPlaneNodes(ctx, f.ClientSet).WithTimeout(1 * time.Minute).Should(MatchLabels(expectedLabels, nodes))
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
// Test NodeFeatureGroups
|
||||
Context("and NodeFeatureGroups objects deployed", Label("nodefeaturegroup"), func() {
|
||||
BeforeEach(func(ctx context.Context) {
|
||||
// We need a NodeFeature from the node, can't be a fake one
|
||||
if !useNodeFeatureApi {
|
||||
Skip("NodeFeature API not enabled")
|
||||
}
|
||||
// enable the node feature group api
|
||||
extraMasterPodSpecOpts = []testpod.SpecOption{
|
||||
testpod.SpecWithContainerExtraArgs(
|
||||
"--feature-gates=NodeFeatureGroupAPI=true",
|
||||
),
|
||||
testpod.SpecWithContainerExtraArgs("--feature-gates=NodeFeatureGroupAPI=true"),
|
||||
}
|
||||
})
|
||||
It("custom NodeFeatureGroup should be updated", func(ctx context.Context) {
|
||||
By("Creating nfd-worker daemonset")
|
||||
podSpecOpts := createPodSpecOpts(
|
||||
podSpecOpts := []testpod.SpecOption{
|
||||
testpod.SpecWithContainerImage(dockerImage()),
|
||||
)
|
||||
}
|
||||
workerDS := testds.NFDWorker(podSpecOpts...)
|
||||
workerDS, err := f.ClientSet.AppsV1().DaemonSets(f.Namespace.Name).Create(ctx, workerDS, metav1.CreateOptions{})
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
@ -955,10 +917,6 @@ denyLabelNs: ["*.denied.ns","random.unwanted.ns"]
|
|||
})
|
||||
It("master configuration should take place", func(ctx context.Context) {
|
||||
// deploy node feature object
|
||||
if !useNodeFeatureApi {
|
||||
Skip("NodeFeature API not enabled")
|
||||
}
|
||||
|
||||
nodes, err := getNonControlPlaneNodes(ctx, f.ClientSet)
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
|
@ -1016,10 +974,6 @@ resyncPeriod: "1s"
|
|||
})
|
||||
It("labels should be restored to the original ones", func(ctx context.Context) {
|
||||
// deploy node feature object
|
||||
if !useNodeFeatureApi {
|
||||
Skip("NodeFeature API not enabled")
|
||||
}
|
||||
|
||||
nodes, err := getNonControlPlaneNodes(ctx, f.ClientSet)
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
|
@ -1065,17 +1019,6 @@ resyncPeriod: "1s"
|
|||
})
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
// Run the actual tests
|
||||
Context("when running NFD with gRPC API enabled", func() {
|
||||
nfdTestSuite(false)
|
||||
})
|
||||
|
||||
Context("when running NFD with NodeFeature CRD API enabled", func() {
|
||||
nfdTestSuite(true)
|
||||
})
|
||||
|
||||
})
|
||||
|
||||
// getNonControlPlaneNodes gets the nodes that are not tainted for exclusive control-plane usage
|
||||
|
|
Loading…
Add table
Reference in a new issue