1
0
Fork 0
mirror of https://github.com/kubernetes-sigs/node-feature-discovery.git synced 2024-12-14 11:57:51 +00:00
node-feature-discovery/docs/usage/features.md
Fabiano Fidêncio 10672e1bba cpu: Expose the total number of keys for TDX
The total amount of keys that can be used on a specific TDX system is
exposed via the cgroups misc.capacity. See:

```
$ cat /sys/fs/cgroup/misc.capacity
tdx 31
```

The first step to properly manage the amount of keys present in a node
is exposing it via the NFD, and that's exactly what this commit does.

An example of how it ends up being exposed via the NFD:

```
$ kubectl get node 984fee00befb.jf.intel.com -o jsonpath='{.metadata.labels}'  | jq | grep tdx.total_keys
  "feature.node.kubernetes.io/cpu-security.tdx.total_keys": "31",
```

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-31 09:12:26 +02:00

15 KiB

title layout sort
Feature labels default 1

Feature labels

{: .no_toc}

Table of contents

{: .no_toc .text-delta}

  1. TOC {:toc}

Features are advertised as labels in the Kubernetes Node object.

Built-in labels

Label creation in nfd-worker is performed by a set of separate modules called label sources. The core.labelSources configuration option (or -label-sources flag) of nfd-worker controls which sources to enable for label generation.

All built-in labels use the feature.node.kubernetes.io label namespace and have the following format.

feature.node.kubernetes.io/<feature> = <value>

NOTE: Consecutive runs of nfd-worker will update the labels on a given node. If features are not discovered on a consecutive run, the corresponding label will be removed. This includes any restrictions placed on the consecutive run, such as restricting discovered features with the -label-whitelist flag of nfd-master or core.labelWhiteList option of nfd-worker.

CPU

Feature name Value Description
cpu-cpuid.<cpuid-flag> true CPU capability is supported. NOTE: the capability might be supported but not enabled.
cpu-hardware_multithreading true Hardware multithreading, such as Intel HTT, enabled (number of logical CPUs is greater than physical CPUs)
cpu-coprocessor.nx_gzip true Nest Accelerator for GZIP is supported(Power).
cpu-power.sst_bf.enabled true Intel SST-BF (Intel Speed Select Technology - Base frequency) enabled
cpu-pstate.status string The status of the Intel pstate driver when in use and enabled, either 'active' or 'passive'.
cpu-pstate.turbo bool Set to 'true' if turbo frequencies are enabled in Intel pstate driver, set to 'false' if they have been disabled.
cpu-pstate.scaling_governor string The value of the Intel pstate scaling_governor when in use, either 'powersave' or 'performance'.
cpu-cstate.enabled bool Set to 'true' if cstates are set in the intel_idle driver, otherwise set to 'false'. Unset if intel_idle cpuidle driver is not active.
cpu-rdt.<rdt-flag> true Intel RDT capability is supported. See RDT flags for details.
cpu-security.sgx.enabled true Set to 'true' if Intel SGX is enabled in BIOS (based a non-zero sum value of SGX EPC section sizes).
cpu-security.se.enabled true Set to 'true' if IBM Secure Execution for Linux (IBM Z & LinuxONE) is available and enabled (requires /sys/firmware/uv/prot_virt_host facility)
cpu-security.tdx.enabled true Set to 'true' if Intel TDX is available on the host and has been enabled (requires /sys/module/kvm_intel/parameters/tdx).
cpu-security.tdx.total_keys int The total amount of keys an Intel TDX enabled host can provide, based on the /sys/fs/cgroup/misc.capacity information.
cpu-security.sev.enabled true Set to 'true' if ADM SEV is available on the host and has been enabled (requires /sys/module/kvm_intel/parameters/sev).
cpu-security.sev.es.enabled true Set to 'true' if ADM SEV-ES is available on the host and has been enabled (requires /sys/module/kvm_intel/parameters/sev_es).
cpu-security.sev.snp.enabled true Set to 'true' if ADM SEV-SNP is available on the host and has been enabled (requires /sys/module/kvm_intel/parameters/sev_snp).
cpu-sgx.enabled true DEPRECATED: use cpu-security.sgx.enabled instead.
cpu-se.enabled true DEPRECATED: use cpu-security.se.enabled instead.
cpu-model.vendor_id string Comparable CPU vendor ID.
cpu-model.family int CPU family.
cpu-model.id int CPU model number.

The CPU label source is configurable, see worker configuration and sources.cpu configuration options for details.

X86 CPUID flags (partial list)

Flag Description
ADX Multi-Precision Add-Carry Instruction Extensions (ADX)
AESNI Advanced Encryption Standard (AES) New Instructions (AES-NI)
AVX Advanced Vector Extensions (AVX)
AVX2 Advanced Vector Extensions 2 (AVX2)
AVXVNNI AVX (VEX encoded) VNNI neural network instructions
AMXBF16 Advanced Matrix Extension, tile multiplication operations on BFLOAT16 numbers
AMXINT8 Advanced Matrix Extension, tile multiplication operations on 8-bit integers
AMXFP16 Advanced Matrix Extension, tile multiplication operations on FP16 numbers
AMXTILE Advanced Matrix Extension, base tile architecture support
AVX512BF16 AVX-512 BFLOAT16 instructions
AVX512BITALG AVX-512 bit Algorithms
AVX512BW AVX-512 byte and word Instructions
AVX512CD AVX-512 conflict detection instructions
AVX512DQ AVX-512 doubleword and quadword instructions
AVX512ER AVX-512 exponential and reciprocal instructions
AVX512F AVX-512 foundation
AVX512FP16 AVX-512 FP16 instructions
AVX512IFMA AVX-512 integer fused multiply-add instructions
AVX512PF AVX-512 prefetch instructions
AVX512VBMI AVX-512 vector bit manipulation instructions
AVX512VBMI2 AVX-512 vector bit manipulation instructions, version 2
AVX512VL AVX-512 vector length extensions
AVX512VNNI AVX-512 vector neural network instructions
AVX512VP2INTERSECT AVX-512 intersect for D/Q
AVX512VPOPCNTDQ AVX-512 vector population count doubleword and quadword
AVXIFMA AVX-IFMA instructions
AVXNECONVERT AVX-NE-CONVERT instructions
AVXVNNIINT8 AVX-VNNI-INT8 instructions
CMPCCXADD CMPCCXADD instructions
ENQCMD Enqueue Command
GFNI Galois Field New Instructions
HYPERVISOR Running under hypervisor
MSRLIST Read/Write List of Model Specific Registers
PREFETCHI PREFETCHIT0/1 instructions
VAES AVX-512 vector AES instructions
VPCLMULQDQ Carry-less multiplication quadword
WRMSRNS Non-Serializing Write to Model Specific Register

By default, the following CPUID flags have been blacklisted: BMI1, BMI2, CLMUL, CMOV, CX16, ERMS, F16C, HTT, LZCNT, MMX, MMXEXT, NX, POPCNT, RDRAND, RDSEED, RDTSCP, SGX, SSE, SSE2, SSE3, SSE4, SSE42 and SSSE3. See sources.cpu configuration options to change the behavior.

See the full list in github.com/klauspost/cpuid.

Arm CPUID flags (partial list)

Flag Description
IDIVA Integer divide instructions available in ARM mode
IDIVT Integer divide instructions available in Thumb mode
THUMB Thumb instructions
FASTMUL Fast multiplication
VFP Vector floating point instruction extension (VFP)
VFPv3 Vector floating point extension v3
VFPv4 Vector floating point extension v4
VFPD32 VFP with 32 D-registers
HALF Half-word loads and stores
EDSP DSP extensions
NEON NEON SIMD instructions
LPAE Large Physical Address Extensions

Arm64 CPUID flags (partial list)

Flag Description
AES Announcing the Advanced Encryption Standard
EVSTRM Event Stream Frequency Features
FPHP Half Precision(16bit) Floating Point Data Processing Instructions
ASIMDHP Half Precision(16bit) Asimd Data Processing Instructions
ATOMICS Atomic Instructions to the A64
ASIMRDM Support for Rounding Double Multiply Add/Subtract
PMULL Optional Cryptographic and CRC32 Instructions
JSCVT Perform Conversion to Match Javascript
DCPOP Persistent Memory Support

Intel RDT flags

Flag Description
RDTMON Intel RDT Monitoring Technology
RDTCMT Intel Cache Monitoring (CMT)
RDTMBM Intel Memory Bandwidth Monitoring (MBM)
RDTL3CA Intel L3 Cache Allocation Technology
RDTl2CA Intel L2 Cache Allocation Technology
RDTMBA Intel Memory Bandwidth Allocation (MBA) Technology

Kernel

Feature Value Description
kernel-config.<option> true Kernel config option is enabled (set 'y' or 'm'). Default options are NO_HZ, NO_HZ_IDLE, NO_HZ_FULL and PREEMPT
kernel-selinux.enabled true Selinux is enabled on the node
kernel-version.full string Full kernel version as reported by /proc/sys/kernel/osrelease (e.g. '4.5.6-7-g123abcde')
kernel-version.major string First component of the kernel version (e.g. '4')
kernel-version.minor string Second component of the kernel version (e.g. '5')
kernel-version.revision string Third component of the kernel version (e.g. '6')

The kernel label source is configurable, see worker configuration and sources.kernel configuration options for details.

Memory

Feature Value Description
memory-numa true Multiple memory nodes i.e. NUMA architecture detected
memory-nv.present true NVDIMM device(s) are present
memory-nv.dax true NVDIMM region(s) configured in DAX mode are present

Network

Feature Value Description
network-sriov.capable true Single Root Input/Output Virtualization (SR-IOV) enabled Network Interface Card(s) present
network-sriov.configured true SR-IOV virtual functions have been configured

PCI

Feature Value Description
pci-<device label>.present true PCI device is detected
pci-<device label>.sriov.capable true Single Root Input/Output Virtualization (SR-IOV) enabled PCI device present

<device label> is format is configurable and set to <class>_<vendor> by default. For more more details about configuration of the pci labels, see sources.pci options and worker configuration instructions.

USB

Feature Value Description
usb-<device label>.present true USB device is detected

<device label> is format is configurable and set to <class>_<vendor>_<device> by default. For more more details about configuration of the usb labels, see sources.usb options and worker configuration instructions.

Storage

Feature Value Description
storage-nonrotationaldisk true Non-rotational disk, like SSD, is present in the node

System

Feature Value Description
system-os_release.ID string Operating system identifier
system-os_release.VERSION_ID string Operating system version identifier (e.g. '6.7')
system-os_release.VERSION_ID.major string First component of the OS version id (e.g. '6')
system-os_release.VERSION_ID.minor string Second component of the OS version id (e.g. '7')

Custom

The custom label source is designed for creating user defined labels. However, it has a few statically defined built-in labels:

Feature Value Description
custom-rdma.capable true The node has an RDMA capable Network adapter
custom-rdma.enabled true The node has the needed RDMA modules loaded to run RDMA traffic

User defined labels

NFD has many extension points for creating vendor and application specific labels. See the customization guide for detailed documentation.

Extended resources

This feature is experimental and by no means a replacement for the usage of device plugins.

Labels which have integer values, can be promoted to Kubernetes extended resources by listing them to the master -resource-labels command line flag. These labels won't then show in the node label section, they will appear only as extended resources.

An example use-case for the extended resources could be based on a hook which creates a label for the node SGX EPC memory section size. By giving the name of that label in the -resource-labels flag, that value will then turn into an extended resource of the node, allowing PODs to request that resource and the Kubernetes scheduler to schedule such PODs to only those nodes which have a sufficient capacity of said resource left.

Similar to labels, the default namespace feature.node.kubernetes.io is automatically prefixed to the extended resource, if the promoted label doesn't have a namespace.

Example usage of the command line arguments, using a new namespace: nfd-master -resource-labels=my_source-my.feature,sgx.some.ns/epc -extra-label-ns=sgx.some.ns

The above would result in following extended resources provided that related labels exist:

  sgx.some.ns/epc: <label value>
  feature.node.kubernetes.io/my_source-my.feature: <label value>