Developer guide

Table of contents

1. Building from source
   1. Download the source code
   2. Docker build
   3. Docker multi-arch builds with buildx
   4. Deployment
   5. Building locally
   6. Customizing the build
   7. Testing
   8. NFD-Master
   9. NFD-Worker
   10. NFD-Topology-Updater
2. Running with Tilt
   1. Prerequisites
   2. Environment variables
3. Documentation

Building from source

Download the source code

git clone https://github.com/kubernetes-sigs/node-feature-discovery
cd node-feature-discovery

Docker build

Build the container image

See customizing the build below for altering the container image registry, for example.

make image

This builds the minimal image variant, <IMAGE_REGISTRY>/node-feature-discovery:<IMAGE_TAG_NAME> (also tagged with a -minimal suffix), and the full variant, <IMAGE_REGISTRY>/node-feature-discovery:<IMAGE_TAG_NAME>-full. It also generates a kustomization.yaml in the root of the source tree (see Deployment below).

Push the container image

Optional, this example with Docker.

docker push <IMAGE_TAG>

Docker multi-arch builds with buildx

The architectures enabled by default for multi-arch builds are linux/amd64, linux/arm64, linux/arm/v7, linux/s390x and linux/ppc64le. If more architectures are needed one can override the IMAGE_ALL_PLATFORMS variable with a comma separated list of OS/ARCH tuples.

Build the manifest-list with a container image per arch

make image-all

make image-all builds with the nfd-builder buildx builder (docker-container driver, set up by hack/init-buildx.sh) and passes neither --load nor --push, so the images stay in the build cache and are not shown by docker images. Use make push-all to publish them (loading multi-platform images into the local image store requires the containerd image store).

Push the manifest-list with container image per arch

make push-all

The resulting container image can be used in the same way on each arch by pulling e.g. node-feature-discovery:v0.20.0 without specifying the architecture. The manifest-list will take care of providing the right architecture image.

Change the job spec to use your custom image (optional)

To use your published image from the step above instead of the registry.k8s.io/nfd/node-feature-discovery image, edit image attribute in the spec template(s) to the new location (<registry-name>/<image-name>[:<version>]).

Deployment

The yamls make target generates a kustomization.yaml matching your locally built image and using the deployment/overlays/default deployment. See build customization below for configurability, e.g. changing the deployment namespace.

K8S_NAMESPACE=my-ns make yamls
kubectl apply -k .

You can use alternative deployment methods by modifying the auto-generated kustomization file.

Building locally

You can also build the binaries locally

make build

This will compile binaries under bin/

Customizing the build

There are several Makefile variables that control the build process and the name of the resulting container image. The following are targeted targeted for build customization and they can be specified via environment variables or makefile overrides.

For example, to use a custom registry:

make image IMAGE_REGISTRY=<my custom registry uri>

Or to specify a build tool different from Docker, It can be done in 2 ways:

1. via environment

    IMAGE_BUILD_CMD="buildah bud" make image
   

2. by overriding the variable value

    make image IMAGE_BUILD_CMD="buildah bud"
   

Testing

Unit tests are not run as part of the container image build (CI runs them via scripts/test-infra/verify.sh). Run them in the source code tree with:

make test

End-to-end tests are built on top of the e2e test framework of Kubernetes, and, they required a cluster to run them on. For running the tests on your test cluster you need to specify the kubeconfig to be used:

make e2e-test KUBECONFIG=$HOME/.kube/config

There are several environment variables that can be used to customize the e2e-tests:

NFD-Master

For development and debugging it is possible to run nfd-master as a stand-alone binary outside the cluster. The -no-publish flag can be used to prevent nfd-master making changes to the nodes. If -no-publish is not set, nfd-master also requires the NODE_NAME environment variable to be set for cleaning up stale annotations.

make build
NODE_NAME=<EXISTING_NODE> ./bin/nfd-master -no-publish -kubeconfig ~/.kube/config

NFD-Worker

For development and debugging it is possible to run nfd-worker as a stand-alone binary outside the cluster. The -no-publish flag can be used to prevent nfd-worker from creating NodeFeature objects in the target cluster. If the -no-publish is not set, nfd-worker also requires the NODE_NAME and KUBERNETES_NAMESPACE environment variables to be defined to create the NodeFeature object in the target cluster.

make build
KUBERNETES_NAMESPACE=default NODE_NAME=nonexistent-node ./bin/nfd-worker -kubeconfig ~/.kube/config

NOTE: Running nfd-worker locally this way discovers and publishes features of the local development system you're running nfd-worker on.

NFD-Topology-Updater

For development and debugging it is possible to run nfd-topology-updater as a stand-alone binary outside the cluster. However, it requires access to the kubelet's local pod-resources socket and the kubelet http api so in practice it needs to be run on a host acting as a Kubernetes node and thus running kubelet. The kubelet configuration is read from the URI given with -kubelet-config-uri, which supports file:// (e.g. file:///var/lib/kubelet/config.yaml) and https:// (the kubelet /configz endpoint on the secure port 10250, authenticated with the token from -api-auth-token-file). http:// URIs such as the kubelet read-only port are not supported, and the read-only port does not serve /configz. Also, the -no-publish flag can be used to prevent nfd-topology-updater from creating NodeResourceTopology objects in the target cluster. If the -no-publish is not set, nfd-topology-updater also requires the NODE_NAME and KUBERNETES_NAMESPACE environment variables to be defined.

make build
KUBERNETES_NAMESPACE=default NODE_NAME=<this-node> ./bin/nfd-topology-updater -kubeconfig ~/.kube/config -kubelet-config-uri file:///var/lib/kubelet/config.yaml

Running with Tilt

Another option for building NFD locally is via Tilt tool, which can build container images, push them to a local registry and reload your Kubernetes pods automatically. When using Tilt, you don't have to build container images and re-deploy your pods manually but instead let the Tilt take care of it. Tiltfile is a configuration file for the Tilt and is located at the root directory. To develop NFD with Tilt, follow the steps below.

NOTE: The Tiltfile sets default_registry('ttl.sh'). If Tilt detects a local registry in the cluster (the local-registry-hosting ConfigMap in kube-public, which kind's local registry setup creates), it pushes images there and ignores default_registry. Without such a registry, Tilt loads the image straight into the cluster where the cluster supports it (for example kind or Docker Desktop). Only when neither works, for example on a kubeadm cluster (context kubernetes-admin@kubernetes, which the Tiltfile allows), does Tilt push images built from your working tree to the public, anonymous ttl.sh registry. Edit default_registry() in the Tiltfile if that is not acceptable.

Prerequisites

1. Install Docker
2. Setup Docker as a non-root user.
3. Install kubectl
4. Install kustomize
5. Install tilt
6. Create a local Kubernetes cluster
   • Create a local image registry first, for example with kind's local registry setup
   • Create a Kubernetes cluster. Please note that docker containers will be served as controller node and worker nodes, and NFD-worker will run as a DaemonSet in nested container. Therefore, to make sure the NFD-worker can discover the host features, the host folders "/boot" and "/lib" should be mounted into worker node docker containers when creating the Kubernetes cluster.

7. Start up node feature discovery development environment To start up your Tilt development environment, run at the root of your local NFD codebase.

    tilt up
   

   Tilt will start a web interface in the localhost and port 10350. From the web interface, you are able to see how NFD worker and master are progressing, watch their build and runtime logs. Once your code changes are saved locally, Tilt will notice it and re-build the container image from the current code, push the image to the registry and re-deploy NFD pods with the latest container image.

Environment variables

To override environment variables used in the Tiltfile during image build, export them in your current terminal before starting Tilt.

export IMAGE_TAG_NAME="v1"
tilt up

This will override the default value(master) of IMAGE_TAG_NAME variable defined in the Tiltfile.

Documentation

All documentation resides under the docs directory in the source tree. It is designed to be served as a html site by GitHub Pages.

Building the documentation is containerized to fix the build environment. The recommended way for developing documentation is to run:

make site-serve

This will build the documentation in a container and serve it under localhost:4000/ making it easy to verify the results. Any changes made to the docs/ will automatically re-trigger a rebuild and are reflected in the served content and can be inspected with a browser refresh.

To just build the html documentation run:

make site-build

This will generate html documentation under docs/_site/.