</code></pre></div></div><p>The object targets node named <codeclass="language-plaintext highlighter-rouge notranslate">node-1</code>. It lists two "flag type" features under the <codeclass="language-plaintext highlighter-rouge notranslate">vendor.flags</code> domain, two "attribute type" features and under the <codeclass="language-plaintext highlighter-rouge notranslate">vendor.config</code> domain and two "instance type" features under the <codeclass="language-plaintext highlighter-rouge notranslate">vendor.devices</code> domain. These features will not be directly affecting the node labels but they will be used as input when the <ahref="#nodefeaturerule-custom-resource"><codeclass="language-plaintext highlighter-rouge notranslate">NodeFeatureRule</code></a> objects are evaluated.</p><p>In addition, the example requests directly the <codeclass="language-plaintext highlighter-rouge notranslate">feature.node.kubenernetes.io/vendor-feature.enabled=true</code> node label to be created.</p><p>The <codeclass="language-plaintext highlighter-rouge notranslate">nfd.node.kubernetes.io/node-name=<node-name></code> must be in place for each NodeFeature object as NFD uses it to determine the node which it is targeting.</p><h3id="feature-types">Feature types</h3><p>Features are divided into three different types:</p><ul><li><strong>flag</strong> features: a set of names without any associated values, e.g. CPUID flags or loaded kernel modules</li><li><strong>attribute</strong> features: a set of names each of which has a single value associated with it (essentially a map of key-value pairs), e.g. kernel config flags or os release information</li><li><strong>instance</strong> features: a list of instances, each of which has multiple attributes (key-value pairs of their own) associated with it, e.g. PCI or USB devices</li></ul><h2id="nodefeaturerule-custom-resource">NodeFeatureRule custom resource</h2><p><codeclass="language-plaintext highlighter-rouge notranslate">NodeFeatureRule</code> objects provide an easy way to create vendor or application specific labels and taints. It uses a flexible rule-based mechanism for creating labels and optionally taints based on node features.</p><h3id="a-nodefeaturerule-example">A NodeFeatureRule example</h3><p>Consider the following referential example:</p><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="na">apiVersion</span><spanclass="pi">:</span><spanclass="s">nfd.k8s-sigs.io/v1alpha1</span>
</code></pre></div></div><p>It specifies one rule which creates node label <codeclass="language-plaintext highlighter-rouge notranslate">feature.node.kubenernetes.io/my-sample-feature=true</code> if both of the following conditions are true (<codeclass="language-plaintext highlighter-rouge notranslate">matchFeatures</code> implements a logical AND over the matchers):</p><ul><li>The <codeclass="language-plaintext highlighter-rouge notranslate">dummy</code> network driver module has been loaded</li><li>X86 option in kernel config is set to <codeclass="language-plaintext highlighter-rouge notranslate">=y</code></li></ul><p>Create a <codeclass="language-plaintext highlighter-rouge notranslate">NodeFeatureRule</code> with a yaml file:</p><divclass="language-bash highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code>kubectl apply <spanclass="nt">-f</span> https://raw.githubusercontent.com/kubernetes-sigs/node-feature-discovery/master/examples/nodefeaturerule.yaml
</code></pre></div></div><p>Now, on X86 platforms the feature label appears after doing <codeclass="language-plaintext highlighter-rouge notranslate">modprobe dummy</code> on a system and correspondingly the label is removed after <codeclass="language-plaintext highlighter-rouge notranslate">rmmod dummy</code>. Note a re-labeling delay up to the sleep-interval of nfd-worker (1 minute by default).</p><p>See <ahref="#label-rule-format">Label rule format</a> for detailed description of available fields and how to write labeling rules.</p><h3id="nodefeaturerule-tainting-feature">NodeFeatureRule tainting feature</h3><p>This feature is experimental.</p><p>In some circumstances, it is desirable to keep nodes with specialized hardware away from running general workload and instead leave them for workloads that need the specialized hardware. One way to achieve it is to taint the nodes with the specialized hardware and add corresponding toleration to pods that require the special hardware. NFD offers node tainting functionality which is disabled by default. User can define one or more custom taints via the <codeclass="language-plaintext highlighter-rouge notranslate">taints</code> field of the NodeFeatureRule CR. The same rule-based mechanism is applied here and the NFD taints only rule matching nodes.</p><p>To enable the tainting feature, <codeclass="language-plaintext highlighter-rouge notranslate">--enable-taints</code> flag needs to be set to <codeclass="language-plaintext highlighter-rouge notranslate">true</code>. If the flag <codeclass="language-plaintext highlighter-rouge notranslate">--enable-taints</code> is set to <codeclass="language-plaintext highlighter-rouge notranslate">false</code> (i.e. disabled), taints defined in the NodeFeatureRule CR have no effect and will be ignored by the NFD master.</p><p><strong>NOTE</strong>: Before enabling any taints, make sure to edit nfd-worker daemonset to tolerate the taints to be created. Otherwise, already running pods that do not tolerate the taint are evicted immediately from the node including the nfd-worker pod.</p><p>Example NodeFeatureRule with custom taints:</p><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="na">apiVersion</span><spanclass="pi">:</span><spanclass="s">nfd.k8s-sigs.io/v1alpha1</span>
</code></pre></div></div><p>In this example, if the <codeclass="language-plaintext highlighter-rouge notranslate">my sample taint rule</code> rule is matched, <codeclass="language-plaintext highlighter-rouge notranslate">feature.node.kubernetes.io/pci-0300_1d0f.present=true:NoExecute</code> and <codeclass="language-plaintext highlighter-rouge notranslate">feature.node.kubernetes.io/cpu-cpuid.ADX:NoExecute</code> taints are set on the node.</p><p>There are some limitations to the namespace part (i.e. prefix/) of the taint key:</p><ul><li><codeclass="language-plaintext highlighter-rouge notranslate">kubernetes.io/</code> and its sub-namespaces (like <codeclass="language-plaintext highlighter-rouge notranslate">sub.ns.kubernetes.io/</code>) cannot generally be used</li><li>the only exception is <codeclass="language-plaintext highlighter-rouge notranslate">feature.node.kubernetes.io/</code> and its sub-namespaces (like <codeclass="language-plaintext highlighter-rouge notranslate">sub.ns.feature.node.kubernetes.io</code>)</li><li>unprefixed keys (like <codeclass="language-plaintext highlighter-rouge notranslate">foo</code>) keys are disallowed</li></ul><h2id="local-feature-source">Local feature source</h2><p>NFD-Worker has a special feature source named <codeclass="language-plaintext highlighter-rouge notranslate">local</code> which is an integration point for external feature detectors. It provides a mechanism for pluggable extensions, allowing the creation of new user-specific features and even overriding built-in labels.</p><p>The <codeclass="language-plaintext highlighter-rouge notranslate">local</code> feature source has two methods for detecting features, hooks and feature files. The features discovered by the <codeclass="language-plaintext highlighter-rouge notranslate">local</code> source can further be used in label rules specified in <ahref="#nodefeaturerule-custom-resource"><codeclass="language-plaintext highlighter-rouge notranslate">NodeFeatureRule</code></a> objects and the <ahref="#custom-feature-source"><codeclass="language-plaintext highlighter-rouge notranslate">custom</code></a> feature source.</p><p><strong>NOTE:</strong> Be careful when creating and/or updating hook or feature files while NFD is running. In order to avoid race conditions you should write into a temporary file (outside the <codeclass="language-plaintext highlighter-rouge notranslate">source.d</code> and <codeclass="language-plaintext highlighter-rouge notranslate">features.d</code> directories), and, atomically create/update the original file by doing a filesystem move operation.</p><h3id="a-hook-example">A hook example</h3><p>Consider a shell script <codeclass="language-plaintext highlighter-rouge notranslate">/etc/kubernetes/node-feature-discovery/source.d/my-hook.sh</code> having the following stdout output, or alternatively, a plaintext file <codeclass="language-plaintext highlighter-rouge notranslate">/etc/kubernetes/node-feature-discovery/features.d/my-features</code> having the following contents:</p><divclass="language-plaintext highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code>my-feature.1
</code></pre></div></div><p>This will translate into the following node labels:</p><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="s">feature.node.kubernetes.io/my-feature.1</span><spanclass="pi">:</span><spanclass="s2">"</span><spanclass="s">true"</span>
</code></pre></div></div><h3id="hooks">Hooks</h3><p><strong>DEPRECATED</strong> The <codeclass="language-plaintext highlighter-rouge notranslate">local</code> source executes hooks found in <codeclass="language-plaintext highlighter-rouge notranslate">/etc/kubernetes/node-feature-discovery/source.d/</code>. The hook files must be executable and they are supposed to print all discovered features in <codeclass="language-plaintext highlighter-rouge notranslate">stdout</code>. Since NFD v0.13 the default container image only supports statically linked ELF binaries.</p><p><codeclass="language-plaintext highlighter-rouge notranslate">stderr</code> output of hooks is propagated to NFD log so it can be used for debugging and logging.</p><p>NFD tries to execute any regular files found from the hooks directory. Any additional data files the hook might need (e.g. a configuration file) should be placed in a separate directory in order to avoid NFD unnecessarily trying to execute them. A subdirectory under the hooks directory can be used, for example <codeclass="language-plaintext highlighter-rouge notranslate">/etc/kubernetes/node-feature-discovery/source.d/conf/</code>.</p><p><strong>NOTE:</strong> Hooks are being DEPRECATED and will be removed in a future release. For backward compatibility, currently hooks are enabled by default and can be disabled via <codeclass="language-plaintext highlighter-rouge notranslate">sources.local.hooksEnabled</code> field in the worker configuration.</p><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="na">sources</span><spanclass="pi">:</span>
</code></pre></div></div><p><strong>NOTE:</strong> NFD will blindly run any executables placed/mounted in the hooks directory. It is the user's responsibility to review the hooks for e.g. possible security implications.</p><p><strong>NOTE:</strong> The <ahref="/node-feature-discovery/master/deployment/image-variants.html#full">full</a> image variant provides backwards-compatibility with older NFD versions by including a more expanded environment, supporting bash and perl runtimes.</p><h3id="feature-files">Feature files</h3><p>The <codeclass="language-plaintext highlighter-rouge notranslate">local</code> source reads files found in <codeclass="language-plaintext highlighter-rouge notranslate">/etc/kubernetes/node-feature-discovery/features.d/</code>.</p><h3id="input-format">Input format</h3><p>The hook stdout and feature files are expected to contain features in simple key-value pairs, separated by newlines:</p><divclass="language-plaintext highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><name>[=<value>]
</code></pre></div></div><p>The label value defaults to <codeclass="language-plaintext highlighter-rouge notranslate">true</code>, if not specified.</p><p>Label namespace may be specified with <codeclass="language-plaintext highlighter-rouge notranslate"><namespace>/<name>[=<value>]</code>.</p><h3id="mounts">Mounts</h3><p>The standard NFD deployments contain <codeclass="language-plaintext highlighter-rouge notranslate">hostPath</code> mounts for <codeclass="language-plaintext highlighter-rouge notranslate">/etc/kubernetes/node-feature-discovery/source.d/</code> and <codeclass="language-plaintext highlighter-rouge notranslate">/etc/kubernetes/node-feature-discovery/features.d/</code>, making these directories from the host available inside the nfd-worker container.</p><h4id="injecting-labels-from-other-pods">Injecting labels from other pods</h4><p>One use case for the hooks and/or feature files is detecting features in other Pods outside NFD, e.g. in Kubernetes device plugins. By using the same <codeclass="language-plaintext highlighter-rouge notranslate">hostPath</code> mounts for <codeclass="language-plaintext highlighter-rouge notranslate">/etc/kubernetes/node-feature-discovery/source.d/</code> and <codeclass="language-plaintext highlighter-rouge notranslate">/etc/kubernetes/node-feature-discovery/features.d/</code> in the side-car (e.g. device plugin) creates a shared area for deploying hooks and feature files to NFD. NFD will periodically scan the directories and run any hooks and read any feature files it finds.</p><h2id="custom-feature-source">Custom feature source</h2><p>The <codeclass="language-plaintext highlighter-rouge notranslate">custom</code> feature source in nfd-worker provides a rule-based mechanism for label creation, similar to the <ahref="#nodefeaturerule-custom-resource"><codeclass="language-plaintext highlighter-rouge notranslate">NodeFeatureRule</code></a> objects. The difference is that the rules are specified in the worker configuration instead of a Kubernetes API object.</p><p>See <ahref="/node-feature-discovery/master/usage/nfd-worker.html#worker-configuration">worker configuration</a> for instructions how to set-up and manage the worker configuration.</p><h3id="an-example-custom-feature-source-configuration">An example custom feature source configuration</h3><p>Consider the following referential configuration for nfd-worker:</p><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="na">core</span><spanclass="pi">:</span>
</code></pre></div></div><p>It specifies one rule which creates node label <codeclass="language-plaintext highlighter-rouge notranslate">feature.node.kubenernetes.io/my-sample-feature=true</code> if both of the following conditions are true (<codeclass="language-plaintext highlighter-rouge notranslate">matchFeatures</code> implements a logical AND over the matchers):</p><ul><li>The <codeclass="language-plaintext highlighter-rouge notranslate">dummy</code> network driver module has been loaded</li><li>X86 option in kernel config is set to <codeclass="language-plaintext highlighter-rouge notranslate">=y</code></li></ul><p>In addition, the configuration only enables the <codeclass="language-plaintext highlighter-rouge notranslate">custom</code> source, disabling all built-in labels.</p><p>Now, on X86 platforms the feature label appears after doing <codeclass="language-plaintext highlighter-rouge notranslate">modprobe dummy</code> on a system and correspondingly the label is removed after <codeclass="language-plaintext highlighter-rouge notranslate">rmmod dummy</code>. Note a re-labeling delay up to the sleep-interval of nfd-worker (1 minute by default).</p><h3id="additional-configuration-directory">Additional configuration directory</h3><p>In addition to the rules defined in the nfd-worker configuration file, the <codeclass="language-plaintext highlighter-rouge notranslate">custom</code> feature source can read more configuration files located in the <codeclass="language-plaintext highlighter-rouge notranslate">/etc/kubernetes/node-feature-discovery/custom.d/</code> directory. This makes more dynamic and flexible configuration easier.</p><p>As an example, consider having file <codeclass="language-plaintext highlighter-rouge notranslate">/etc/kubernetes/node-feature-discovery/custom.d/my-rule.yaml</code> with the following content:</p><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="pi">-</span><spanclass="na">name</span><spanclass="pi">:</span><spanclass="s2">"</span><spanclass="s">my</span><spanclass="nv"></span><spanclass="s">e1000</span><spanclass="nv"></span><spanclass="s">rule"</span>
</code></pre></div></div><p>This simple rule will create <codeclass="language-plaintext highlighter-rouge notranslate">feature.node.kubenernetes.io/e1000.present=true</code> label if the <codeclass="language-plaintext highlighter-rouge notranslate">e1000</code> kernel module has been loaded.</p><p>The <ahref="https://github.com/kubernetes-sigs/node-feature-discovery/blob/master/deployment/overlays/samples/custom-rules"><codeclass="language-plaintext highlighter-rouge notranslate">samples/custom-rules</code></a> kustomize overlay sample contains an example for deploying a custom rule from a ConfigMap.</p><h2id="node-labels">Node labels</h2><p>Feature labels have the following format:</p><divclass="language-plaintext highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><namespace>/<name> = <value>
</code></pre></div></div><p>The namespace part (i.e. prefix) of the labels is controlled by nfd:</p><ul><li>All built-in labels use <codeclass="language-plaintext highlighter-rouge notranslate">feature.node.kubernetes.io</code>. This is also the default for user defined features that don't specify any namespace.</li><li>Namespaces may be excluded with the <ahref="/node-feature-discovery/master/reference/master-commandline-reference.html#-deny-label-ns"><codeclass="language-plaintext highlighter-rouge notranslate">-deny-label-ns</code></a> command line flag of nfd-master <ul><li>To allow specific namespaces that were denied, you can use <ahref="/node-feature-discovery/master/reference/master-commandline-reference.html#-extra-label-ns"><codeclass="language-plaintext highlighter-rouge notranslate">-extra-label-ns</code></a> command line flag of nfd-master. e.g: <codeclass="language-plaintext highlighter-rouge notranslate">nfd-master -deny-label-ns="*" -extra-label-ns=example.com</code></li></ul></li></ul><h2id="label-rule-format">Label rule format</h2><p>This section describes the rule format used in <ahref="#nodefeaturerule-custom-resource"><codeclass="language-plaintext highlighter-rouge notranslate">NodeFeatureRule</code></a> objects and in the configuration of the <ahref="#custom-feature-source"><codeclass="language-plaintext highlighter-rouge notranslate">custom</code></a> feature source.</p><p>It is based on a generic feature matcher that covers all features discovered by nfd-worker. The rules rely on a unified data model of the available features and a generic expression-based format. Features that can be used in the rules are described in detail in <ahref="#available-features">available features</a> below.</p><p>Take this rule as a referential example:</p><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="pi">-</span><spanclass="na">name</span><spanclass="pi">:</span><spanclass="s2">"</span><spanclass="s">my</span><spanclass="nv"></span><spanclass="s">feature</span><spanclass="nv"></span><spanclass="s">rule"</span>
</code></pre></div></div><p>This will yield <codeclass="language-plaintext highlighter-rouge notranslate">feature.node.kubenernetes.io/my-special-feature=my-value</code> node label if all of these are true (<codeclass="language-plaintext highlighter-rouge notranslate">matchFeatures</code> implements a logical AND over the matchers):</p><ul><li>the CPU has AVX512F capability</li><li>kernel version is 5.2 or later (must be v5.x)</li><li>an Intel network controller is present</li></ul><h3id="fields">Fields</h3><h4id="name">Name</h4><p>The <codeclass="language-plaintext highlighter-rouge notranslate">.name</code> field is required and used as an identifier of the rule.</p><h4id="labels">Labels</h4><p>The <codeclass="language-plaintext highlighter-rouge notranslate">.labels</code> is a map of the node labels to create if the rule matches.</p><h4id="labels-template">Labels template</h4><p>The <codeclass="language-plaintext highlighter-rouge notranslate">.labelsTemplate</code> field specifies a text template for dynamically creating labels based on the matched features. See <ahref="#templating">templating</a> for details.</p><p><strong>NOTE</strong> The <codeclass="language-plaintext highlighter-rouge notranslate">labels</code> field has priority over <codeclass="language-plaintext highlighter-rouge notranslate">labelsTemplate</code>, i.e. labels specified in the <codeclass="language-plaintext highlighter-rouge notranslate">labels</code> field will override anything originating from <codeclass="language-plaintext highlighter-rouge notranslate">labelsTemplate</code>.</p><h3id="taints">Taints</h3><p><em>taints</em> is a list of taint entries and each entry can have <codeclass="language-plaintext highlighter-rouge notranslate">key</code>, <codeclass="language-plaintext highlighter-rouge notranslate">value</code> and <codeclass="language-plaintext highlighter-rouge notranslate">effect</code>, where the <codeclass="language-plaintext highlighter-rouge notranslate">value</code> is optional. Effect could be <codeclass="language-plaintext highlighter-rouge notranslate">NoSchedule</code>, <codeclass="language-plaintext highlighter-rouge notranslate">PreferNoSchedule</code> or <codeclass="language-plaintext highlighter-rouge notranslate">NoExecute</code>. To learn more about the meaning of these effects, check out k8s <ahref="https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/">documentation</a>.</p><p><strong>NOTE</strong> taints field is not available for the custom rules of nfd-worker and only for NodeFeatureRule objects.</p><h4id="vars">Vars</h4><p>The <codeclass="language-plaintext highlighter-rouge notranslate">.vars</code> field is a map of values (key-value pairs) to store for subsequent rules to use. In other words, these are variables that are not advertised as node labels. See <ahref="#backreferences">backreferences</a> for more details on the usage of vars.</p><h4id="vars-template">Vars template</h4><p>The <codeclass="language-plaintext highlighter-rouge notranslate">.varsTemplate</code> field specifies a text template for dynamically creating vars based on the matched features. See <ahref="#templating">templating</a> for details on using templates and <ahref="#backreferences">backreferences</a> for more details on the usage of vars.</p><p><strong>NOTE</strong> The <codeclass="language-plaintext highlighter-rouge notranslate">vars</code> field has priority over <codeclass="language-plaintext highlighter-rouge notranslate">varsTemplate</code>, i.e. vars specified in the <codeclass="language-plaintext highlighter-rouge notranslate">vars</code> field will override anything originating from <codeclass="language-plaintext highlighter-rouge notranslate">varsTemplate</code>.</p><h4id="matchfeatures">MatchFeatures</h4><p>The <codeclass="language-plaintext highlighter-rouge notranslate">.matchFeatures</code> field specifies a feature matcher, consisting of a list of feature matcher terms. It implements a logical AND over the terms i.e. all of them
</code></pre></div></div><p>The <codeclass="language-plaintext highlighter-rouge notranslate">.matchFeatures[].feature</code> field specifies the feature against which to match.</p><p>The <codeclass="language-plaintext highlighter-rouge notranslate">.matchFeatures[].matchExpressions</code> field specifies a map of expressions which to evaluate against the elements of the feature.</p><p>In each MatchExpression <codeclass="language-plaintext highlighter-rouge notranslate">op</code> specifies the operator to apply. Valid values are described below.</p><table><thead><tr><th>Operator</th><th>Number of values</th><th>Matches when</th></tr></thead><tbody><tr><td><codeclass="language-plaintext highlighter-rouge notranslate">In</code></td><td>1 or greater</td><td>Input is equal to one of the values</td></tr><tr><td><codeclass="language-plaintext highlighter-rouge notranslate">NotIn</code></td><td>1 or greater</td><td>Input is not equal to any of the values</td></tr><tr><td><codeclass="language-plaintext highlighter-rouge notranslate">InRegexp</code></td><td>1 or greater</td><td>Values of the MatchExpression are treated as regexps and input matches one or more of them</td></tr><tr><td><codeclass="language-plaintext highlighter-rouge notranslate">Exists</code></td><td>0</td><td>The key exists</td></tr><tr><td><codeclass="language-plaintext highlighter-rouge notranslate">DoesNotExist</code></td><td>0</td><td>The key does not exists</td></tr><tr><td><codeclass="language-plaintext highlighter-rouge notranslate">Gt</code></td><td>1</td><td>Input is greater than the value. Both the input and value must be integer numbers.</td></tr><tr><td><codeclass="language-plaintext highlighter-rouge notranslate">Lt</code></td><td>1</td><td>Input is less than the value. Both the input and value must be integer numbers.</td></tr><tr><td><codeclass="language-plaintext highlighter-rouge notranslate">GtLt</code></td><td>2</td><td>Input is between two values. Both the input and value must be integer numbers.</td></tr><tr><td><codeclass="language-plaintext highlighter-rouge notranslate">IsTrue</code></td><td>0</td><td>Input is equal to "true"</td></tr><tr><td><codeclass="language-plaintext highlighter-rouge notranslate">IsFalse</code></td><td>0</td><td>Input is equal "false"</td></tr></tbody></table><p>The <codeclass="language-plaintext highlighter-rouge notranslate">value</code> field of MatchExpression is a list of string arguments to the operator.</p><p>The behavior of MatchExpression depends on the <ahref="#feature-types">feature type</a>: for <em>flag</em> and <em>attribute</em> features the MatchExpression operates on the feature element whose name matches the <codeclass="language-plaintext highlighter-rouge notranslate"><key></code>. However, for <em>instance</em> features all MatchExpressions are evaluated against the attributes of each instance separately.</p><h4id="matchany">MatchAny</h4><p>The <codeclass="language-plaintext highlighter-rouge notranslate">.matchAny</code> field is a list of of <ahref="#matchfeatures"><codeclass="language-plaintext highlighter-rouge notranslate">matchFeatures</code></a> matchers. A logical OR is applied over the matchers, i.e. at least one of them must match in order for the rule to trigger.</p><p>Consider the following example:</p><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="na">matchAny</span><spanclass="pi">:</span>
</code></pre></div></div><p>This matches if kernel module kmod-1 is loaded and a network controller from vendor 0eee is present, OR, if kernel module kmod-2 has been loaded and a network controller from vendor 0fff is present (OR both of these conditions are true).</p><h3id="available-features">Available features</h3><p>The following features are available for matching:</p><table><thead><tr><th>Feature</th><th><ahref="#feature-types">Feature type</a></th><th>Elements</th><th>Value type</th><th>Description</th></tr></thead><tbody><tr><td><strong><codeclass="language-plaintext highlighter-rouge notranslate">cpu.cpuid</code></strong></td><td>flag</td><td></td><td></td><td>Supported CPU capabilities</td></tr><tr><td></td><td></td><td><strong><codeclass="language-plaintext highlighter-rouge notranslate"><cpuid-flag></code></strong></td><td></td><td>CPUID flag is present</td></tr><tr><td><strong><codeclass="language-plaintext highlighter-rouge notranslate">cpu.cstate</code></strong></td><td>attribute</td><td></td><td></td><td>Status of cstates in the intel_idle cpuidle driver</td></tr><tr><td></td><td></td><td><strong><codeclass="language-plaintext highlighter-rouge notranslate">enabled</code></strong></td><td>bool</td><td>‘true' if cstates are set, otherwise ‘false'. Does not exist of intel_idle driver is not active.</td></tr><tr><td><strong><codeclass="language-plaintext highlighter-rouge notranslate">cpu.model</code></strong></td><td>attribute</td><td></td><td></td><td>CPU model related attributes</td></tr><tr><td></td><td></td><td><strong><codeclass="language-plaintext highlighter-rouge notranslate">family</code></strong></td><td>int</td><td>CPU family</td></tr><tr><td></td><td></td><td><strong><codeclass="language-plaintext highlighter-rouge notranslate">vendor_id</code></strong></td><td>string</td><td>CPU vendor ID</td></tr><tr><td></td><td></td><td><strong><codeclass="language-plaintext highlighter-rouge notranslate">id</code></strong></td><td>int</td><td>CPU model ID</td></tr><tr><td><strong><codeclass="language-plaintext highlighter-rouge notranslate">cpu.pstate</code></strong></td><td>attribute</td><td></td><td></td><td>State of the Intel pstate driver. Does not exist if the driver is not enabled.</td></tr><tr><td></td><td></td><td><strong><codeclass="language-plaintext highlighter-rouge notranslate">status</code></strong></td><td>string</td><td>Status of the driver, possible values are ‘active' and ‘passive'</td></tr><tr><td></td><td></td><td><strong><codeclass="language-plaintext highlighter-rouge notranslate">turbo</code></strong></td><td>bool</td><td>‘true' if turbo frequencies are enabled, otherwise ‘false'</td></tr><tr><td></td><td></td><td><strong><codeclass="language-plaintext highlighter-rouge notranslate">scaling</code></strong></td><td>string</td><td>Active scaling_governor, possible values are ‘powersave' or ‘performance'.</td></tr><tr><td><strong><codeclass="language-plaintext highlighter-rouge notranslate">cpu.rdt</code></strong></td><td>attribute</td><td></td><td></td><td>Intel RDT capabilities supported by the system</td></tr><tr><td></td><td></td><td><strong><codeclass="language-plaintext highlighter-rouge notranslate"><rdt-flag></code></strong></td><td></td><td>RDT capability is supported, see <ahref="#intel-rdt-flags">RDT flags</a> for details</td></tr><tr><td></td><td></td><td><strong><codeclass="language-plaintext highlighter-rouge notranslate">RDTL3CA_NUM_CLOSID</code></strong></td><td>int</td><td>The number or available CLOSID (Class of service ID) for Intel L3 Cache Allocation Technology</td></tr><tr><td><strong><codeclass="language-plaintext highlighter-rouge notranslate">cpu.security</code></strong></td><td>attribute</td><td></td><td></td><td>Features related to security and trusted execution environments</td></tr><tr><td></td><td></td><td><str
</code></pre></div></div><!----><p>The rule above will create individual labels <codeclass="language-plaintext highlighter-rouge notranslate">feature.node.kubernetes.io/vendor-<class-id>-<device-id>.present=true</code> for each network controller device (device class starting with 02) from vendor 0ffff.</p><p>All the matched features of each feature matcher term under <codeclass="language-plaintext highlighter-rouge notranslate">matchFeatures</code> fields are available for the template engine. Matched features can be referenced with <codeclass="language-plaintext highlighter-rouge notranslate">{{ .<feature-name> }}</code> in the template, and the available data could be described in yaml as follows:</p><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="s">.</span>
</code></pre></div></div><p>That is, the per-feature data is a list of objects whose data fields depend on the type of the feature:</p><ul><li>for <em>flag</em> features only ‘Name' is available</li><li>for <em>value</em> features ‘Name' and ‘Value' are available</li><li>for <em>instance</em> features all attributes of the matched instance are available</li></ul><p>A simple example of a template utilizing name and value from an <em>attribute</em> feature: <!----></p><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="na">labelsTemplate</span><spanclass="pi">:</span><spanclass="pi">|</span>
<spanclass="s">{{ range .system.osrelease }}system-{{ .Name }}={{ .Value }}</span>
</code></pre></div></div><!----><p><strong>NOTE</strong> In case of matchAny is specified, the template is executed separately against each individual <codeclass="language-plaintext highlighter-rouge notranslate">matchFeatures</code> field and the final set of labels will be superset of all these separate template expansions. E.g. consider the following:</p><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="pi">-</span><spanclass="na">name</span><spanclass="pi">:</span><spanclass="s"><name></span>
</code></pre></div></div><p>In the example above (assuming the overall result is a match) the template would be executed on matcher#1 as well as on matcher#2 and/or matcher#3 (depending on whether both or only one of them match). All the labels from these separate expansions would be created, i.e. the end result would be a union of all the individual expansions.</p><p>Rule templates use the Golang <ahref="https://pkg.go.dev/text/template">text/template</a> package and all its built-in functionality (e.g. pipelines and functions) can be used. An example template taking use of the built-in <codeclass="language-plaintext highlighter-rouge notranslate">len</code> function, advertising the number of PCI network controllers from a specific vendor: <!----></p><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="na">labelsTemplate</span><spanclass="pi">:</span><spanclass="pi">|</span>
</code></pre></div></div><!----><p>Imaginative template pipelines are possible, but care must be taken in order to produce understandable and maintainable rule sets.</p><h3id="backreferences">Backreferences</h3><p>Rules support referencing the output of preceding rules. This enables sophisticated scenarios where multiple rules are combined together to for more complex heuristics than a single rule can provide. The labels and vars created by the execution of preceding rules are available as a special <codeclass="language-plaintext highlighter-rouge notranslate">rule.matched</code> feature.</p><p>Consider the following configuration:</p><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="pi">-</span><spanclass="na">name</span><spanclass="pi">:</span><spanclass="s2">"</span><spanclass="s">my</span><spanclass="nv"></span><spanclass="s">kernel</span><spanclass="nv"></span><spanclass="s">label</span><spanclass="nv"></span><spanclass="s">rule"</span>
</code></pre></div></div><p>The <codeclass="language-plaintext highlighter-rouge notranslate">feature.node.kubernetes.io/high-level-feature = true</code> label depends on thw two previous rules.</p><p>Note that when referencing rules across multiple <ahref="#nodefeaturerule-custom-resource"><codeclass="language-plaintext highlighter-rouge notranslate">NodeFeatureRule</code></a> objects attention must be paid to the ordering. <codeclass="language-plaintext highlighter-rouge notranslate">NodeFeatureRule</code> objects are processed in alphabetical order (based on their <codeclass="language-plaintext highlighter-rouge notranslate">.metadata.name</code>).</p><h3id="examples">Examples</h3><p>Some more configuration examples below.</p><p>Match certain CPUID features:</p><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="pi">-</span><spanclass="na">name</span><spanclass="pi">:</span><spanclass="s2">"</span><spanclass="s">example</span><spanclass="nv"></span><spanclass="s">cpuid</span><spanclass="nv"></span><spanclass="s">rule"</span>
</code></pre></div></div><p>Require a certain loaded kernel module and OS version:</p><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="pi">-</span><spanclass="na">name</span><spanclass="pi">:</span><spanclass="s2">"</span><spanclass="s">my</span><spanclass="nv"></span><spanclass="s">multi-feature</span><spanclass="nv"></span><spanclass="s">rule"</span>
</code></pre></div></div><p>Require a loaded kernel module and two specific PCI devices (both of which must be present):</p><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="pi">-</span><spanclass="na">name</span><spanclass="pi">:</span><spanclass="s2">"</span><spanclass="s">my</span><spanclass="nv"></span><spanclass="s">multi-device</span><spanclass="nv"></span><spanclass="s">rule"</span>
</code></pre></div></div><h2id="legacy-custom-rule-syntax">Legacy custom rule syntax</h2><p><strong>DEPRECATED</strong>: use the new rule syntax instead.</p><p>The <codeclass="language-plaintext highlighter-rouge notranslate">custom</code> source supports the legacy <codeclass="language-plaintext highlighter-rouge notranslate">matchOn</code> rule syntax for backwards-compatibility.</p><p>To aid in making the legacy rule syntax clearer, we define a general and a per rule nomenclature, keeping things as consistent as possible.</p><h3id="general-nomenclature-and-definitions">General nomenclature and definitions</h3><divclass="language-plaintext highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code>Rule :Represents a matching logic that is used to match on a feature.
Rule Input :The input a Rule is provided. This determines how a Rule performs the match operation.
Matcher :A composition of Rules, each Matcher may be composed of at most one instance of each Rule.
</code></pre></div></div><h3id="custom-features-format-using-the-nomenclature-defined-above">Custom features format (using the nomenclature defined above)</h3><p>Rules are specified under <codeclass="language-plaintext highlighter-rouge notranslate">sources.custom</code> in the nfd-worker configuration file.</p><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="na">sources</span><spanclass="pi">:</span>
</code></pre></div></div><p>The label is constructed by adding <codeclass="language-plaintext highlighter-rouge notranslate">custom-</code> prefix to the name field, label value defaults to <codeclass="language-plaintext highlighter-rouge notranslate">true</code> if not specified in the rule spec:</p><divclass="language-plaintext highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code>feature.node.kubernetes.io/custom-<name> = <value>
</code></pre></div></div><h3id="matching-process">Matching process</h3><p>Specifying Rules to match on a feature is done by providing a list of Matchers. Each Matcher contains one or more Rules.</p><p>Logical <em>OR</em> is performed between Matchers and logical <em>AND</em> is performed between Rules of a given Matcher.</p><h3id="rules">Rules</h3><h4id="pciid-rule">pciid rule</h4><h5id="nomenclature">Nomenclature</h5><divclass="language-plaintext highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code>Attribute :A PCI attribute.
Element :An identifier of the PCI attribute.
</code></pre></div></div><p>The PciId Rule allows matching the PCI devices in the system on the following Attributes: <codeclass="language-plaintext highlighter-rouge notranslate">class</code>,<codeclass="language-plaintext highlighter-rouge notranslate">vendor</code> and <codeclass="language-plaintext highlighter-rouge notranslate">device</code>. A list of Elements is provided for each Attribute.</p><h5id="format">Format</h5><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="na">pciId </span><spanclass="pi">:</span>
</code></pre></div></div><p>Matching is done by performing a logical <em>OR</em> between Elements of an Attribute and logical <em>AND</em> between the specified Attributes for each PCI device in the system. At least one Attribute must be specified. Missing attributes will not partake in the matching process.</p><h4id="usbid-rule">UsbId rule</h4><h5id="nomenclature-1">Nomenclature</h5><divclass="language-plaintext highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code>Attribute :A USB attribute.
Element :An identifier of the USB attribute.
</code></pre></div></div><p>The UsbId Rule allows matching the USB devices in the system on the following Attributes: <codeclass="language-plaintext highlighter-rouge notranslate">class</code>,<codeclass="language-plaintext highlighter-rouge notranslate">vendor</code>, <codeclass="language-plaintext highlighter-rouge notranslate">device</code> and <codeclass="language-plaintext highlighter-rouge notranslate">serial</code>. A list of Elements is provided for each Attribute.</p><h5id="format-1">Format</h5><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="na">usbId </span><spanclass="pi">:</span>
</code></pre></div></div><p>Matching is done by performing a logical <em>OR</em> between Elements of an Attribute and logical <em>AND</em> between the specified Attributes for each USB device in the system. At least one Attribute must be specified. Missing attributes will not partake in the matching process.</p><h4id="loadedkmod-rule">LoadedKMod rule</h4><h5id="nomenclature-2">Nomenclature</h5><divclass="language-plaintext highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code>Element :A kernel module
</code></pre></div></div><p>The LoadedKMod Rule allows matching the loaded kernel modules in the system against a provided list of Elements.</p><h5id="format-2">Format</h5><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="na">loadedKMod </span><spanclass="pi">:</span><spanclass="pi">[</span><spanclass="nv"><kernel module></span><spanclass="pi">,</span><spanclass="nv">...</span><spanclass="pi">]</span>
</code></pre></div></div><p>Matching is done by performing logical <em>AND</em> for each provided Element, i.e the Rule will match if all provided Elements (kernel modules) are loaded in the system.</p><h4id="cpuid-rule">CpuId rule</h4><h5id="nomenclature-3">Nomenclature</h5><divclass="language-plaintext highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code>Element :A CPUID flag
</code></pre></div></div><p>The Rule allows matching the available CPUID flags in the system against a provided list of Elements.</p><h5id="format-3">Format</h5><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="na">cpuId </span><spanclass="pi">:</span><spanclass="pi">[</span><spanclass="nv"><CPUID flag string></span><spanclass="pi">,</span><spanclass="nv">...</span><spanclass="pi">]</span>
</code></pre></div></div><p>Matching is done by performing logical <em>AND</em> for each provided Element, i.e the Rule will match if all provided Elements (CPUID flag strings) are available in the system.</p><h4id="kconfig-rule">Kconfig rule</h4><h5id="nomenclature-4">Nomenclature</h5><divclass="language-plaintext highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code>Element :A Kconfig option
</code></pre></div></div><p>The Rule allows matching the kconfig options in the system against a provided list of Elements.</p><h5id="format-4">Format</h5><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="na">kConfig</span><spanclass="pi">:</span><spanclass="pi">[</span><spanclass="nv"><kernel config option ('y' or 'm') or '=<value>'></span><spanclass="pi">,</span><spanclass="nv">...</span><spanclass="pi">]</span>
</code></pre></div></div><p>Matching is done by performing logical <em>AND</em> for each provided Element, i.e the Rule will match if all provided Elements (kernel config options) are enabled (<codeclass="language-plaintext highlighter-rouge notranslate">y</code> or <codeclass="language-plaintext highlighter-rouge notranslate">m</code>) or matching <codeclass="language-plaintext highlighter-rouge notranslate">=<value></code> in the kernel.</p><h4id="nodename-rule">Nodename rule</h4><h5id="nomenclature-5">Nomenclature</h5><divclass="language-plaintext highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code>Element :A nodename regexp pattern
</code></pre></div></div><p>The Rule allows matching the node's name against a provided list of Elements.</p><h5id="format-5">Format</h5><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="na">nodename</span><spanclass="pi">:</span><spanclass="pi">[</span><spanclass="nv"><nodename regexp pattern></span><spanclass="pi">,</span><spanclass="nv">...</span><spanclass="pi">]</span>
</code></pre></div></div><p>Matching is done by performing logical <em>OR</em> for each provided Element, i.e the Rule will match if one of the provided Elements (nodename regexp pattern) matches the node's name.</p><h3id="legacy-custom-rule-example">Legacy custom rule example</h3><divclass="language-yaml highlighter-rouge notranslate"><divclass="highlight"><preclass="highlight"><code><spanclass="na">custom</span><spanclass="pi">:</span>
</code></pre></div></div><p><strong>In the example above:</strong></p><ul><li>A node would contain the label: <codeclass="language-plaintext highlighter-rouge notranslate">feature.node.kubernetes.io/custom-my.kernel.feature=true</code> if the node has <codeclass="language-plaintext highlighter-rouge notranslate">kmod1</code><em>AND</em><codeclass="language-plaintext highlighter-rouge notranslate">kmod2</code> kernel modules loaded.</li><li>A node would contain the label: <codeclass="language-plaintext highlighter-rouge notranslate">feature.node.kubernetes.io/custom-my.pci.feature=true</code> if the node contains a PCI device with a PCI vendor ID of <codeclass="language-plaintext highlighter-rouge notranslate">15b3</code><em>AND</em> PCI device ID of <codeclass="language-plaintext highlighter-rouge notranslate">1014</code><em>OR</em><codeclass="language-plaintext highlighter-rouge notranslate">1017</code>.</li><li>A node would contain the label: <codeclass="language-plaintext highlighter-rouge notranslate">feature.node.kubernetes.io/custom-my.usb.feature=true</code> if the node contains a USB device with a USB vendor ID of <codeclass="language-plaintext highlighter-rouge notranslate">1d6b</code><em>AND</em> USB device ID of <codeclass="language-plaintext highlighter-rouge notranslate">0003</code>.</li><li>A node would contain the label: <codeclass="language-plaintext highlighter-rouge notranslate">feature.node.kubernetes.io/custom-my.combined.feature=true</code> if <codeclass="language-plaintext highlighter-rouge notranslate">vendor_kmod1</code><em>AND</em><codeclass="language-plaintext highlighter-rouge notranslate">vendor_kmod2</code> kernel modules are loaded <strong>AND</strong> the node contains a PCI device with a PCI vendor ID of <codeclass="language-plaintext highlighter-rouge notranslate">15b3</code><em>AND</em> PCI device ID of <codeclass="language-plaintext highlighter-rouge notranslate">1014</code><em>or</em><codeclass="language-plaintext highlighter-rouge notranslate">1017</code>.</li><li>A node would contain the label: <codeclass="language-plaintext highlighter-rouge notranslate">vendor.feature.node.kubernetes.io/accumulated.feature=true</code> if <codeclass="language-plaintext highlighter-rouge notranslate">some_kmod1</code><em>AND</em><codeclass="language-plaintext highlighter-rouge notranslate">some_kmod2</code> kernel modules are loaded <strong>OR</strong> the node contains a PCI device with a PCI vendor ID of <codeclass="language-plaintext highlighter-rouge notranslate">15b3</code><em>AND</em> PCI device ID of <codeclass="language-plaintext highlighter-rouge notranslate">1014</code><em>OR</em><codeclass="language-plaintext highlighter-rouge notranslate">1017</code>.</li><li>A node would contain the label: <codeclass="language-plaintext highlighter-rouge notranslate">feature.node.kubernetes.io/custom-my.kernel.featureneedscpu=true</code> if <codeclass="language-plaintext highlighter-rouge notranslate">KVM_INTEL</code> kernel config is enabled <strong>AND</strong> the node CPU supports <codeclass="language-plaintext highlighter-rouge notranslate">VMX</code> virtual machine extensions</li><li>A node would contain the label: <codeclass="language-plaintext highlighter-rouge notranslate">feature.node.kubernetes.io/custom-my.kernel.modulecompiler=true</code> if the in-tree <codeclass="language-plaintext highlighter-rouge notranslate">kmod1</code> kernel module is loaded <strong>AND</strong> it's built with <codeclass="language-plaintext highlighter-rouge notranslate">GCC_VERSION=100101</code>.</li><li>A node would contain the label: <codeclass="language-plaintext highlighter-rouge notranslate">profile.node.kubernetes.io/my-datacenter=datacenter-1</code> if the node's name matches the <codeclass="language-plaintext highlighter-rouge notranslate">node-datacenter1-rack.*-server.*</code> pattern, e.g. <codeclass="language-plaintext highlighter-rouge notranslate">node-datacenter1-rack2-server42</code></li></ul></div></div><divclass="navigation-bottom
