2016-07-22 22:13:48 -07:00
|
|
|
# Node feature discovery for [Kubernetes](https://kubernetes.io)
|
|
|
|
|
2016-07-25 22:44:45 -07:00
|
|
|
- [Overview](#overview)
|
2016-08-30 17:04:43 -07:00
|
|
|
- [Command line interface](#command-line-interface)
|
|
|
|
- [Feature discovery](#feature-discovery)
|
|
|
|
- [Feature sources](#feature-sources)
|
|
|
|
- [Feature labels](#feature-labels)
|
|
|
|
- [Getting started](#getting-started)
|
|
|
|
- [System requirements](#system-requirements)
|
|
|
|
- [Usage](#usage)
|
2016-07-12 16:44:03 -07:00
|
|
|
- [Building from source](#building-from-source)
|
2016-08-30 17:04:43 -07:00
|
|
|
- [Targeting nodes with specific features](#targeting-nodes-with-specific-features)
|
|
|
|
- [References](#references)
|
2016-07-25 22:44:45 -07:00
|
|
|
- [License](#license)
|
|
|
|
|
|
|
|
## Overview
|
|
|
|
|
|
|
|
This software enables node feature discovery for Kubernetes. It detects
|
|
|
|
hardware features available on each node in a Kubernetes cluster, and advertises
|
|
|
|
those features using node labels.
|
|
|
|
|
2016-08-30 17:04:43 -07:00
|
|
|
## Command line interface
|
|
|
|
|
|
|
|
```
|
|
|
|
node-feature-discovery.
|
|
|
|
|
|
|
|
Usage:
|
|
|
|
node-feature-discovery [--no-publish --sources=<sources>]
|
|
|
|
node-feature-discovery -h | --help
|
|
|
|
node-feature-discovery --version
|
|
|
|
|
|
|
|
Options:
|
|
|
|
-h --help Show this screen.
|
|
|
|
--version Output version and exit.
|
|
|
|
--sources=<sources> Comma separated list of feature sources. [Default: cpuid,rdt,pstate]
|
|
|
|
--no-publish Do not publish discovered features to the cluster-local Kubernetes API server.
|
|
|
|
```
|
|
|
|
|
|
|
|
## Feature discovery
|
|
|
|
|
2016-07-25 22:44:45 -07:00
|
|
|
### Feature sources
|
|
|
|
|
2016-07-19 15:35:42 -07:00
|
|
|
The current set of feature sources are the following:
|
|
|
|
|
2016-07-25 22:44:45 -07:00
|
|
|
- [CPUID][cpuid] for x86 CPU details
|
|
|
|
- [Intel Resource Director Technology][intel-rdt]
|
|
|
|
- [Intel P-State driver][intel-pstate]
|
|
|
|
|
|
|
|
### Feature labels
|
|
|
|
|
|
|
|
The published node labels encode a few pieces of information:
|
|
|
|
|
|
|
|
- A "namespace" to denote the vendor/provider (e.g. `node.alpha.intel.com`).
|
|
|
|
- The version of this discovery code that wrote the label, according to
|
|
|
|
`git describe --tags --dirty --always`.
|
|
|
|
- The source for each label (e.g. `cpuid`).
|
|
|
|
- The name of the discovered feature as it appears in the underlying
|
|
|
|
source, (e.g. `AESNI` from cpuid).
|
|
|
|
|
|
|
|
_Note: only features that are available on a given node are labeled, so
|
|
|
|
the only label value published for features is the string `"true"`._
|
|
|
|
|
|
|
|
```json
|
|
|
|
{
|
2016-08-30 17:04:43 -07:00
|
|
|
"node.alpha.intel.com/node-feature-discovery.version": "v0.1.0",
|
2016-07-25 22:44:45 -07:00
|
|
|
"node.alpha.intel.com/v0.1.0-cpuid-<feature-name>": "true",
|
|
|
|
"node.alpha.intel.com/v0.1.0-rdt-<feature-name>": "true",
|
|
|
|
"node.alpha.intel.com/v0.1.0-pstate-<feature-name>": "true"
|
|
|
|
}
|
|
|
|
```
|
|
|
|
|
2016-08-30 17:04:43 -07:00
|
|
|
The `--sources` flag controls which sources to use for discovery.
|
|
|
|
|
2016-07-12 16:44:03 -07:00
|
|
|
### Intel Resource Director Technology (RDT) Features
|
|
|
|
|
|
|
|
| Feature name | Description |
|
|
|
|
| :------------: | :---------------------------------------------------------------------------------: |
|
|
|
|
| RDTMON | Intel Cache Monitoring Technology (CMT) and Intel Memory Bandwidth Monitoring (MBM)
|
|
|
|
| RDTL3CA | Intel L3 Cache Allocation Technology
|
|
|
|
| RDTL2CA | Intel L2 Cache Allocation Technology
|
|
|
|
|
2016-08-30 17:04:43 -07:00
|
|
|
### CPUID Features (Partial List)
|
2016-07-12 16:44:03 -07:00
|
|
|
|
|
|
|
| Feature name | Description |
|
|
|
|
| :------------: | :----------------------------------------------------------: |
|
|
|
|
| ADX | Multi-Precision Add-Carry Instruction Extensions (ADX)
|
|
|
|
| AESNI | Advanced Encryption Standard (AES) New Instructions (AES-NI)
|
|
|
|
| AVX | Advanced Vector Extensions (AVX)
|
|
|
|
| AVX2 | Advanced Vector Extensions 2 (AVX2)
|
|
|
|
| BMI1 | Bit Manipulation Instruction Set 1 (BMI)
|
|
|
|
| BMI2 | Bit Manipulation Instruction Set 2 (BMI2)
|
|
|
|
| SSE4.1 | Streaming SIMD Extensions 4.1 (SSE4.1)
|
|
|
|
| SSE4.2 | Streaming SIMD Extensions 4.2 (SSE4.2)
|
|
|
|
| SGX | Software Guard Extensions (SGX)
|
|
|
|
|
2016-07-12 02:30:49 +02:00
|
|
|
### System Requirements
|
2016-07-12 16:44:03 -07:00
|
|
|
|
|
|
|
1. Linux (x86_64)
|
2016-07-19 15:35:42 -07:00
|
|
|
1. [kubectl] [kubectl-setup] (properly set up and configured to work with your
|
|
|
|
Kubernetes cluster)
|
2016-07-12 16:44:03 -07:00
|
|
|
1. [Docker] [docker-down] (only required to build and push docker images)
|
|
|
|
|
|
|
|
### Usage
|
|
|
|
|
|
|
|
Feature discovery is done as a one-shot job. There is an example script in this
|
|
|
|
repo that demonstrates how to deploy the job to unlabeled nodes.
|
2016-07-12 02:30:49 +02:00
|
|
|
|
|
|
|
```
|
2016-07-12 16:44:03 -07:00
|
|
|
./label-nodes.sh
|
2016-07-12 02:30:49 +02:00
|
|
|
```
|
|
|
|
|
2016-07-12 16:44:03 -07:00
|
|
|
The discovery script will launch a job on each each unlabeled node in the
|
2016-07-19 15:35:42 -07:00
|
|
|
cluster. When the job runs, it contacts the Kubernetes API server to add labels
|
|
|
|
to the node to advertise hardware features (initially, from `cpuid` and RDT).
|
2016-07-12 16:44:03 -07:00
|
|
|
|
|
|
|
## Building from source
|
|
|
|
|
|
|
|
Download the source code.
|
|
|
|
|
2016-07-12 02:30:49 +02:00
|
|
|
```
|
2016-08-30 17:04:43 -07:00
|
|
|
git clone https://github.com/kubernetes-incubator/node-feature-discovery
|
2016-07-12 02:30:49 +02:00
|
|
|
```
|
|
|
|
|
2016-07-18 12:35:38 -07:00
|
|
|
**Build the Docker image:**
|
2016-07-12 02:30:49 +02:00
|
|
|
|
|
|
|
```
|
2016-07-12 16:44:03 -07:00
|
|
|
cd <project-root>
|
2016-07-19 15:35:42 -07:00
|
|
|
make
|
2016-07-12 02:30:49 +02:00
|
|
|
```
|
2016-07-12 16:44:03 -07:00
|
|
|
|
2016-07-19 15:35:42 -07:00
|
|
|
**NOTE: To override the `DOCKER_REGISTRY_USER` use the `-e` option as follows:
|
|
|
|
`DOCKER_REGISTRY_USER=<my-username> make docker -e`**
|
|
|
|
|
2016-07-12 16:44:03 -07:00
|
|
|
Push the Docker Image (optional)
|
|
|
|
|
2016-07-12 02:30:49 +02:00
|
|
|
```
|
2016-07-19 15:35:42 -07:00
|
|
|
docker push <registry-user>/<image-name>:<version>
|
2016-07-12 02:30:49 +02:00
|
|
|
```
|
|
|
|
|
2016-07-18 12:35:38 -07:00
|
|
|
**Change the job spec to use your custom image (optional):**
|
2016-07-12 16:44:03 -07:00
|
|
|
|
|
|
|
To use your published image from the step above instead of the
|
2016-08-30 17:04:43 -07:00
|
|
|
`kubernetesincubator/node-feature-discovery` image, edit line 40 in the file
|
|
|
|
[node-feature-discovery-job.json.template](node-feature-discovery-job.json.template)
|
|
|
|
to the new location (`<registry-user>/<image-name>[:<version>]`).
|
2016-07-12 16:44:03 -07:00
|
|
|
|
2016-07-14 16:56:43 +02:00
|
|
|
## Targeting Nodes with Specific Features
|
|
|
|
|
2016-07-19 15:35:42 -07:00
|
|
|
Nodes with specific features can be targeted using the `nodeSelector` field. The
|
|
|
|
following example shows how to target nodes with Intel TurboBoost enabled.
|
2016-07-14 16:56:43 +02:00
|
|
|
|
|
|
|
```json
|
|
|
|
{
|
|
|
|
"apiVersion": "v1",
|
|
|
|
"kind": "Pod",
|
|
|
|
"metadata": {
|
|
|
|
"labels": {
|
|
|
|
"env": "test"
|
|
|
|
},
|
|
|
|
"name": "golang-test"
|
|
|
|
},
|
|
|
|
"spec": {
|
|
|
|
"containers": [
|
|
|
|
{
|
|
|
|
"image": "golang",
|
|
|
|
"name": "go1",
|
|
|
|
}
|
|
|
|
],
|
|
|
|
"nodeSelector": {
|
2016-07-19 15:35:42 -07:00
|
|
|
"node.alpha.intel.com/v0.1.0-pstate-turbo": "true"
|
2016-07-14 16:56:43 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
```
|
|
|
|
|
|
|
|
For more details on targeting nodes, see [node selection][node-sel].
|
|
|
|
|
2016-08-30 17:04:43 -07:00
|
|
|
## References
|
2016-07-31 21:35:52 -07:00
|
|
|
|
|
|
|
Github issues
|
|
|
|
|
|
|
|
- [#28310](https://github.com/kubernetes/kubernetes/issues/28310)
|
|
|
|
- [#28311](https://github.com/kubernetes/kubernetes/issues/28311)
|
|
|
|
- [#28312](https://github.com/kubernetes/kubernetes/issues/28312)
|
|
|
|
|
|
|
|
[Design proposal](https://docs.google.com/document/d/1uulT2AjqXjc_pLtDu0Kw9WyvvXm-WAZZaSiUziKsr68/edit)
|
|
|
|
|
2016-07-14 16:56:43 +02:00
|
|
|
|
2016-07-22 22:13:48 -07:00
|
|
|
## License
|
|
|
|
|
|
|
|
This is open source software released under the [Apache 2.0 License](LICENSE).
|
2016-07-25 22:44:45 -07:00
|
|
|
|
|
|
|
<!-- Links -->
|
|
|
|
[cpuid]: http://man7.org/linux/man-pages/man4/cpuid.4.html
|
|
|
|
[intel-rdt]: http://www.intel.com/content/www/us/en/architecture-and-technology/resource-director-technology.html
|
|
|
|
[intel-pstate]: https://www.kernel.org/doc/Documentation/cpu-freq/intel-pstate.txt
|
2016-07-19 15:35:42 -07:00
|
|
|
[docker-down]: https://docs.docker.com/engine/installation
|
|
|
|
[golang-down]: https://golang.org/dl
|
|
|
|
[gcc-down]: https://gcc.gnu.org
|
2016-07-12 02:30:49 +02:00
|
|
|
[kubectl-setup]: https://coreos.com/kubernetes/docs/latest/configure-kubectl.html
|
2016-07-19 15:35:42 -07:00
|
|
|
[node-sel]: http://kubernetes.io/docs/user-guide/node-selection
|