1
0
Fork 0
mirror of https://github.com/kubernetes-sigs/node-feature-discovery.git synced 2024-12-14 11:57:51 +00:00
Commit graph

843 commits

Author SHA1 Message Date
Markus Lehtonen
9487fbeb18 Utilize go generate
Use 'go generate' for auto-generating code. Drop the old 'mock' and
'apigen' makefile targets. Those are replaced with a single
  make generate

which (re-)generates everything.
2021-09-21 13:36:37 +03:00
Kubernetes Prow Robot
16139ae1f5
Merge pull request #525 from k8stopologyawareschedwg/topology-updater-implementation
Introducing NFD Topology Updater exposing Resource hardware Topology info through CRs
2021-09-21 03:04:23 -07:00
Swati Sehgal
a2c066dc0d topologyupdater: manifests: topologyupdater deployment files
- create an overlay for deployment of all components
- create an overlay for just topologyupdater deployment (to be deployed in
  conjunction with the default overlay)
- create a separate overlay for deployment of master and topologyupdater-job

Signed-off-by: Swati Sehgal <swsehgal@redhat.com>
2021-09-21 10:48:10 +01:00
Swati Sehgal
a311719d1e topologyupdater: Updates based on latest changes made to CRD API
There have been recent changes made to the noderesourcetopology API
storing the proto file generated using go-to-protobuf tool and
this code inports the proto generated in the API in the topology-updater.proto
The PRs corresponding to the changes are as follows:
https://github.com/k8stopologyawareschedwg/noderesourcetopology-api/pull/9
https://github.com/k8stopologyawareschedwg/noderesourcetopology-api/pull/13

Commands used to generate topology-updater.pb.go file:

go install github.com/golang/protobuf/protoc-gen-go@v1.4.3
go mod vendor
protoc --go_opt=paths=source_relative  --go_out=plugins=grpc:. pkg/topologyupdater/topology-updater.proto -I. -Ivendor

As part of implmentation of this patch, reserved (non-allocatable) CPUs
are evaluated by performing a difference between all the CPUs on a system
(determined by using ghw) and allocatable CPUs (determined by querying
GetAllocatableResources podResource API endpoint).

When aggregator creates the NUMA zones, it will skip the zone creation if
there are no allocatable resources. In this update we creates those missing
zone with zero allocatable/available resources so we won't have holes in the
array of reported zones.

Co-Authored-by: Talor Itzhak <titzhak@redhat.com>
Signed-off-by: Swati Sehgal <swsehgal@redhat.com>
2021-09-21 10:48:10 +01:00
Swati Sehgal
832f82baaa topologyupdater: Handle pods with devices and integral CPU requests
For accounting we should consider all guaranteed pods with
integral CPU requests and all the pods with device requests
This patch ensures that pods are only considered
for accounting disregarding non-guranteed pods without any
device request.

Signed-off-by: Swati Sehgal <swsehgal@redhat.com>
2021-09-21 10:48:10 +01:00
Swati Sehgal
aa7ae9265c topologyupdater: watch/consider only guaranteed pods for accounting
- Files obtained after running make mock
- Run `go get github.com/vektra/mockery` and make sure that
  mockery is in your $PATH
- run `make mock`

Signed-off-by: Swati Sehgal <swsehgal@redhat.com>
2021-09-21 10:48:10 +01:00
Francesco Romani
b4c92e4eed topologyupdater: Bootstrap nfd-topology-updater in NFD
- This patch allows to expose Resource Hardware Topology information
  through CRDs in Node Feature Discovery.
- In order to do this we introduce another software component called
  nfd-topology-updater in addition to the already existing software
  components nfd-master and nfd-worker.
- nfd-master was enhanced to communicate with nfd-topology-updater
  over gRPC followed by creation of CRs corresponding to the nodes
  in the cluster exposing resource hardware topology information
  of that node.
- Pin kubernetes dependency to one that include pod resource implementation
- This code is responsible for obtaining hardware information from the system
  as well as pod resource information from the Pod Resource API in order to
  determine the allocatable resource information for each NUMA zone. This
  information along with Costs for NUMA zones (obtained by reading NUMA distances)
  is gathered by nfd-topology-updater running on all the nodes
  of the cluster and propagate NUMA zone costs to master in order to populate
  that information in the CRs corresponding to the nodes.
- We use GHW facilities for obtaining system information like CPUs, topology,
  NUMA distances etc.
- This also includes updates made to Makefile and Dockerfile and Manifests for
  deploying nfd-topology-updater.
- This patch includes unit tests
- As part of the Topology Aware Scheduling work, this patch captures
  the configured Topology manager scope in addition to the Topology manager policy.
  Based on the value of both attribues a single string will be populated to the CRD.
  The string value will be on of the following {SingleNUMANodeContainerLevel,
  SingleNUMANodePodLevel, BestEffort, Restricted, None}

Co-Authored-by: Artyom Lukianov <alukiano@redhat.com>
Co-Authored-by: Francesco Romani <fromani@redhat.com>
Co-Authored-by: Talor Itzhak <titzhak@redhat.com>
Signed-off-by: Swati Sehgal <swsehgal@redhat.com>
2021-09-21 10:47:39 +01:00
Francesco Romani
00cc07da76 topologyupdater: gRPC API definition
Setup the topologyupdater API for gRPC communication of
nfd-topology-updater with master

We generate pb.go file to reflect latest dependency changes
using github.com/golang/protobuf/protoc-gen-go and generate
grpc files via:
`protoc pkg/topologyupdater/topology-updater.proto --go_out=plugins=grpc:.`

Please refer to: https://github.com/k8stopologyawareschedwg/noderesourcetopology-api/blob/master/pkg/apis/topology/v1alpha1/types.go

Co-Authored-by: Artyom Lukianov <alukiano@redhat.com>
Co-Authored-by: Francesco Romani <fromani@redhat.com>
Signed-off-by: Swati Sehgal <swsehgal@redhat.com>
2021-09-21 10:47:39 +01:00
Kubernetes Prow Robot
102003f8b3
Merge pull request #597 from marquiz/devel/source-register
source: make sources register themselves
2021-09-19 09:10:09 -07:00
Kubernetes Prow Robot
4ee535f1e5
Merge pull request #599 from marquiz/fixes/example-config
deployment: fix formatting of the worker conf sample
2021-09-19 09:02:09 -07:00
Markus Lehtonen
3706de9308 deployment: fix formatting of the worker conf sample 2021-09-17 14:25:48 +03:00
Markus Lehtonen
81378a3235 source: make sources register themselves
Implement new registration infrastructure under the "source" package.
This change loosens the coupling between label sources and the
nfd-worker, making it easier to refactor and move the code around.

Also, create a separate interface (ConfigurableSource) for configurable
feature sources in order to eliminate boilerplate code.

Add safety checks to the sources that they actually implement the
interfaces they should.

In sake of consistency and predictability (of behavior) change all
methods of the sources to use pointer receivers.

Add simple unit tests for the new functionality and include source/...
into make test target.
2021-09-15 18:41:37 +03:00
Kubernetes Prow Robot
5c7706c5bd
Merge pull request #596 from marquiz/devel/label-source
source: rename FeatureSource to LabelSource
2021-09-15 07:44:31 -07:00
Markus Lehtonen
befa7e9796 source: rename FeatureSource to LabelSource
Prepare for separating feature detection from label creation.
2021-09-13 22:48:33 +03:00
Kubernetes Prow Robot
795a7e543e
Merge pull request #592 from marquiz/devel/logging-verbosity
source/network: silence annoying/useless log message
2021-09-13 01:16:09 -07:00
Markus Lehtonen
bd5ee9c616 source/network: silence annoying/useless log message
However, log an error if something unexpected happens, i.e. the file to
read maximum number of vfs exists (sriov_totalvfs) but read fails.
2021-09-13 09:40:06 +03:00
Kubernetes Prow Robot
dc7bda0023
Merge pull request #590 from ArangoGutierrez/fix/docs_borkenlink
Fix broken link for worker-conf example
2021-09-02 10:36:57 -07:00
Carlos Eduardo Arango Gutierrez
f9e09e5fd4
Fix broken link for worker-conf example
Signed-off-by: Carlos Eduardo Arango Gutierrez <carangog@redhat.com>
2021-09-02 11:41:03 -05:00
Kubernetes Prow Robot
92920d48d3
Merge pull request #589 from vaibhav2107/link-deploy
Fix a link in deployment-and-usage.md
2021-09-02 05:46:10 -07:00
Vaibhav
ac720ef57a Fix a link in deployment-and-usage.md 2021-09-01 22:41:04 +05:30
Kubernetes Prow Robot
189f86bec8
Merge pull request #548 from marquiz/devel/profile-ns
nfd-master: allow profile.node.kubernetes.io label ns
2021-08-27 07:24:04 -07:00
Kubernetes Prow Robot
487588bc62
Merge pull request #585 from jonkerj/optional-hostpaths
deployment: make /usr/src hostpath mount optional
2021-08-27 01:18:04 -07:00
Kubernetes Prow Robot
f307870347
Merge pull request #586 from marquiz/devel/verify-kustomize
scripts/test-infra: verify buildability of kustomize overlays
2021-08-26 11:30:21 -07:00
Markus Lehtonen
8132d17f89 scripts/test-infra: verify buildability of kustomize overlays
Not perfect and fool proof evidence but better than nothing.
2021-08-26 19:47:25 +03:00
Kubernetes Prow Robot
a47ef9f48e
Merge pull request #587 from swatisehgal/fix-gitignore
Only add kustomization.yaml in the root to .gitignore
2021-08-26 08:33:23 -07:00
Swati Sehgal
4f65f30b7c Only add kustomization.yaml in the root to .gitignore
The current addition of kustomization.yaml to gitignore file
means that any file in the project tree with the name
kustomization.yaml will be ignored. However, that is not
the desired behaviour. One of the primary examples of this
is when we add deployments corresponding to nfd-topology-updater,
we want to be able to ensure that the kustomization.yamls are not
ignored.

Signed-off-by: Swati Sehgal <swsehgal@redhat.com>
2021-08-26 15:27:03 +01:00
Jorik Jonker
501ff37592 deployment: optional mount of /usr/src
This commit makes the mount of /usr/src optional in the Helm chart, and
removes it from the kustomization. Reason is that some systems do not
have a /usr/src (such as Talos) *and* have a R/O filesystem. Since
/usr/src is optional per FHS 3.0, NFD should not assume its presence.

Signed-off-by: Jorik Jonker <jorik@kippendief.biz>
2021-08-26 10:52:26 +02:00
Vaibhav Goel
87feb9610b
Fix the typo in deployment-and-usage.md (#575)
* Fix the typo in deployment-and-usage.md

* Update in doc/get-started/deployment-and-usage.md
2021-08-25 00:32:39 -07:00
Kubernetes Prow Robot
e16d4c9b20
Merge pull request #543 from marquiz/devel/custom-kconfig-refactor
source/custom: refactor kconfig rule internal representation
2021-08-24 06:45:14 -07:00
Kubernetes Prow Robot
f24606be36
Merge pull request #583 from marquiz/fixes/gitignore
gitignore: add kustomization.yaml
2021-08-23 12:50:22 -07:00
Markus Lehtonen
0ab8edf7a1 gitignore: add kustomization.yaml
Want to ignore kustomization.yaml which is auto-generated by the make
yamls target.
2021-08-23 16:24:22 +03:00
Kubernetes Prow Robot
a20c8872fb
Merge pull request #552 from marquiz/devel/worker-refactor
nfd-worker: split out gRPC connection handling
2021-08-23 05:58:00 -07:00
Markus Lehtonen
112744bc50 nfd-worker: split out gRPC connection handling
Refactor the worker code and split out gRPC client connection handling
into a separate base type. The intent is to promote re-usability of code
for other NFD clients, too.
2021-08-20 15:29:27 +03:00
Kubernetes Prow Robot
150b8271fa
Merge pull request #580 from marquiz/devel/readme
README: update deployment instructions to use v0.9.0
2021-08-20 04:51:24 -07:00
Markus Lehtonen
45a5b83a5e README: update deployment instructions to use v0.9.0 2021-08-20 09:05:40 +03:00
Kubernetes Prow Robot
6e53fbc1ef
Merge pull request #578 from marquiz/fixes/kustomize
kustomize: fix broken master-worker-combined base
2021-08-19 13:29:25 -07:00
Markus Lehtonen
a3b2d97513 kustomize: fix broken master-worker-combined base
Got broken unnoticed with the addition of liveness and readiness probes.
2021-08-19 23:22:28 +03:00
Kubernetes Prow Robot
37b3af0810
Merge pull request #577 from marquiz/fixes/scripts
scripts: fix prepare-release.sh to update all kustomize files
2021-08-19 12:21:25 -07:00
Markus Lehtonen
9db5807bf4 scripts: fix prepare-release.sh to update all kustomize files 2021-08-19 22:15:58 +03:00
Kubernetes Prow Robot
c07c6354fc
Merge pull request #572 from marquiz/docs/cstate
docs: clarify the cpu.cstate feature
2021-08-19 07:55:24 -07:00
Kubernetes Prow Robot
a0a8304958
Merge pull request #571 from jschintag/cstate-fix
cstate/pstate: Skip check on non intel arches
2021-08-19 07:43:24 -07:00
Jan Schintag
ac0e5b1b52 cstate/pstate: Skip check on non intel arches
Intel driver is not available on other arches, skip check.

Signed-off-by: Jan Schintag <jan.schintag@de.ibm.com>
2021-08-19 16:37:54 +02:00
Kubernetes Prow Robot
440159a0e9
Merge pull request #563 from ArangoGutierrez/devel/issue522
Add Readiness and liveliness probes to nfd-master
2021-08-18 09:40:08 -07:00
Kubernetes Prow Robot
234f385cdd
Merge pull request #570 from marquiz/devel/errors
Better error reporting of kernel and cpu feature sources
2021-08-18 08:44:08 -07:00
Carlos Eduardo Arango Gutierrez
dece85b394
Add livenessProbe via grpc to nfd-master
Signed-off-by: Carlos Eduardo Arango Gutierrez <carangog@redhat.com>
2021-08-18 10:23:10 -05:00
Kubernetes Prow Robot
137a10f239
Merge pull request #573 from marquiz/devel/kustomize
Move to kustomize
2021-08-18 06:06:08 -07:00
Markus Lehtonen
7dfd7d289f docs: document all the provided kustomize overlays 2021-08-18 15:10:25 +03:00
Markus Lehtonen
63c1256d08 Drop deployment templates
Resort to kustomize, instead. Update docs and scripts, accordingly.

Bump cert-manager version in the deployment instructions to v1.5.1.
2021-08-18 15:10:25 +03:00
Markus Lehtonen
1f8a6d7819 kustomize: add standard-combined overlay
Replicates nfd-daemonset-combined.yaml.template.

In addition to the overlay we need to add a separate set of patches
under components/common in order to handle the double-container pod.
2021-08-18 15:10:25 +03:00
Markus Lehtonen
b38cf997d5 kustomize: add prune overlay
Add an overlay for deploying "nfd-master --prune". Replaces
nfd-prune.yaml.template.
2021-08-18 15:10:25 +03:00