turboprop/README.rst

.. vim: set et sw=2:

#########
Turboprop
#########

Problem: You have twenty or thirty Helm releases, all of which you template semi-manually to `retain WYSIWYG control`_. Deploying new applications involves tremendous amounts of copy-pasta.

Solution: Use Nix. With Nix, you can `ensure chart integrity`_, `generate repetitive data`_ in `subroutines`_, and `easily reuse variable data`_.

Turboprop templates your Helm charts for you, making an individual Nix derivation of each one; each of these derivations is then gathered into a mega-derivation complete with Kustomizations for every namespace and service. In short, you're two commands away from full cluster reconciliation::

  nix build && kubectl diff -k ./result

.. _retain WYSIWYG control: https://github.com/kubernetes-sigs/kustomize/blob/bfb00ecb2747dc711abfc27d9cf788ca1d7c637b/examples/chart.md#best-practice
.. _ensure chart integrity: https://git.sr.ht/~goorzhel/kubernetes/tree/f3cba6831621288228581b7ad7b6762d6d58a966/item/charts/intel/device-plugins-gpu/default.nix#L5
.. _generate repetitive data: https://git.sr.ht/~goorzhel/kubernetes/tree/f3cba6831621288228581b7ad7b6762d6d58a966/item/services/svc/gateway/default.nix#L25-26
.. _subroutines: https://git.sr.ht/~goorzhel/kubernetes/tree/f3cba6831621288228581b7ad7b6762d6d58a966/item/services/svc/gateway/default.nix#L8-10
.. _easily reuse variable data: https://git.sr.ht/~goorzhel/kubernetes/tree/f3cba6831621288228581b7ad7b6762d6d58a966/item/system/kube-system/csi-driver-nfs/default.nix#L16

***************
Acknowledgments
***************

Without `Vladimir Pouzanov`_'s "`Nix and Kubernetes\: Deployments Done Right`_" (and `its notes`_), this project would not exist.

I also used `heywoodlh's Kubernetes flake`_ as a starting point early on.

.. _Vladimir Pouzanov: https://github.com/farcaller
.. _Nix and Kubernetes\: Deployments Done Right: https://media.ccc.de/v/nixcon-2023-35290-nix-and-kubernetes-deployments-done-right
.. _its notes: https://gist.github.com/farcaller/c87c03fbb55eaeaeb840b938455f37ff
.. _heywoodlh's Kubernetes flake: https://github.com/heywoodlh/flakes/blob/aa5a52a/kube/flake.nix


**********************
Usage and architecture
**********************

Getting started
===============

To start, add this flake to your flake's inputs, along with ``nixpkgs`` and ``flake-utils``:

.. code-block:: nix

  {
    inputs = {
      nixpkgs.url = "github:NixOS/nixpkgs";
      flake-utils.url = "github:numtide/flake-utils";
      turboprop = {
        url = "sourcehut:~goorzhel/turboprop";
        inputs.nixpkgs.follows = "nixpkgs";
      };
    };
    <...>
  }


Next, split your flake's output into two sections:

#. One running in pure Nix that "rakes" your module data into a *tree* (more on that later); and
#. one that builds derivations from your module data, using your current system's ``nixpkgs``.

.. code-block:: nix

  {
    <...>
    outputs = {self, nixpkgs, flake-utils, turboprop}:
      let rake = turboprop.rake in
    {
      # I'll explain the distinction in a later chapter
      systemServiceData = rake.leaves ./system;
      serviceData = rake.leaves ./services;

      repos = rake.leaves ./charts;

      namespaces = rake.namespaces {
        roots = [./system ./services];
        extraMetadata = import ./namespaces.nix;
      };
    }
    // flake-utils.lib.eachDefaultSystem (system: let
      pkgs = import nixpkgs {
        inherit system;
        overlays = [devshell.overlays.default];
      };

      turbo = turboprop.packages.${system};

      # We'll get back to this too
      flakeBuilders = turbo.flakeBuilders {};
      namespaces = <...>;
      paths = <...>;
    in {
      packages.default = turbo.mkDerivation {
          # This too
        };
      }
    );
  }

Now set that aside for the time being.

Example service module
======================

This is a module that defines a *service derivation*:

.. code-block:: nix

    { charts, lib, user, ... }: {  # 1
      builder = lib.builders.helmChart; # 1.2; 2.1
      args = {  # < - - - - - - - - - - - 2.2
        chart = charts.jetstack.cert-manager; # 1.1
        values = {
          featureGates = "ExperimentalGatewayAPISupport=true";
          installCRDs = true;
          prometheus = {

            enabled = true;
            servicemonitor = {
              enabled = true;
              prometheusInstance = "monitoring";
            };
          };
          startupapicheck.podLabels."sidecar.istio.io/inject" = "false";
        };
      };
      extraObjects = [  # 2.3
        {
          apiVersion = "cert-manager.io/v1";
          kind = "ClusterIssuer";
          metadata.name = user.vars.k8sCert.name; # 1.3
          spec.ca.secretName = user.vars.k8sCert.name;
        }
      ];
    }


1. The module takes as input:

  #. A tree of *chart derivations*;
  #. the Turboprop library; and
  #. user data specific to your flake. You may `omit any of these input variables`_ if they're not used.

2. The module has the output signature ``{builder, args, extraObjects}``.

  #. ``builder`` is the Turboprop builder that will create your derivation. Most often, you will use ``helmChart``; other builders exist for scenarios such as deploying a `collection of Kubernetes objects`_ or a `single remote YAML file`_. You may even `define your own builder`_.
  #. ``args`` are arguments passed to the builder. Refer to each builder's signature below.
  #. ``extraObjects`` are objects to deploy alongside the chart.

.. _omit any of these input variables: https://git.sr.ht/~goorzhel/kubernetes/tree/f3cba6831621288228581b7ad7b6762d6d58a966/item/system/gateway-system/gateway-api/default.nix#L1
.. _collection of Kubernetes objects: https://git.sr.ht/~goorzhel/kubernetes/tree/f3cba6831621288228581b7ad7b6762d6d58a966/item/services/svc/gateway/default.nix#L12
.. _single remote YAML file: https://git.sr.ht/~goorzhel/kubernetes/tree/f3cba6831621288228581b7ad7b6762d6d58a966/item/system/gateway-system/gateway-api/default.nix#L2
.. _define your own builder: https://git.sr.ht/~goorzhel/kubernetes/tree/f3cba6831621288228581b7ad7b6762d6d58a966/item/services/svc/breezewiki/default.nix#L6

Trees of Nix modules
====================

Turboprop's operates on *trees* of Nix modules, both in the filesystem sense (nested directories) and the Nix sense (nested attrsets). A service tree, then, consists of

#. an arbitrarily-named root, such as ``./services``, which contains
#. directories representing Kubernetes namespaces, which each contain
#. Nix modules representing a templated deployment.

This metaphor extends to charts. Both Turboprop and nixhelm, from which Turboprop borrows heavily, contain a chart tree:

#. an arbitrarily-named root, ``./charts``, which contains
#. directories representing Helm repositories, which each contain
#. Nix modules representing a(n untemplated) chart.

In practice::

  $ nix run nixpkgs#tree -- ~/src/kubernetes/{chart,service}s --noreport
  /home/ag/src/kubernetes/charts
  ├── kubernetes-dashboard
  │   └── kubernetes-dashboard
  │       └── default.nix
  <...>
  /home/ag/src/kubernetes/services
  <...>
  ├── istio-system
  │   ├── 1-18-1
  │   │   └── default.nix
  │   └── kiali
  │       └── default.nix
  └── svc
      ├── breezewiki
      │   └── default.nix
      <...>
      └── vaultwarden
          └── default.nix

You may have noticed that, if neither Nixhelm nor Turboprop provide a chart you need, you may define it within your flake. (PRs welcome, though.)

Builders and flake builders
===========================

*Builders* are Nix functions that build a derivation, be it of a chart, a service, or a service's extra objects.
*Flake builders* are Nix functions that contribute toward the flake's output, using the modules' defined builders and
your flake's directory structure to render a tree of Kustomizations.

Recall the definition of ``flakeBuilders`` in the system-specific let-in midway through your flake's outputs:

.. code-block:: nix

  {
    <...>
    outputs = {<...>, turboprop}:
    <...>
    // flake-utils.lib.eachDefaultSystem (system: let
      <...>
      turbo = turboprop.packages.${system};

      flakeBuilders = turbo.flakeBuilders {};
      namespaces = <...>;
      paths = <...>;
    in { <...> });
  }

.. TODO: Defining multiple flake builders sounds like boilerplate.
   Can't all of that be wrapped into turbo.mkDerivation?



Services and system services
============================



.. TODO
- Helm cannot see your cluster from inside the sandbox.
- Custom APIs must be gathered and passed explicitly.
- This can't be done in one step due to infinite recursion.

Finally: your flake's output
============================

*********
Reference
*********

Utilities
=========

Charts
======

Turboprop overlays `its own charts`_ atop `Nixhelm's`_.

.. _its own charts: https://git.sr.ht/~goorzhel/turboprop/tree/main/item/charts
.. _Nixhelm's: https://github.com/farcaller/nixhelm/tree/master/charts


Fetchers
========

gitChart
--------
``{name, version, url, hash, chartPath, vPrefixInRef?} -> <derivation: a dir containing a Helm chart>``

Fetch a Helm chart from a Git repository. Useful in the absence of a published Helm repo.

- **name** (str): The name of the Git repo.
- **version** (str): The tag to check out, which should resemble ``1.0.0``.
- **url** (str): The URL of the Git repo.
- **vPrefixInRef** (bool, default: ``false``): Whether the Git tag begins with an utterly redundant ``v``.
- **chartHash** (str): An `SRI-style hash`_.

helmChart
---------
``{repo, chart, version, chartHash?} -> <derivation: a dir containing a Helm chart>``

Re-export of `kubelib.downloadHelmChart`_.

- **repo** (str): The repository from which to download the chart.
- **chart** (str): The name of the chart.
- **version** (str): The version of the chart, which will also be the derivation's version.
- **chartHash** (str, default: `fakeHash`_): An `SRI-style hash`_.

.. _kubelib.downloadHelmChart: https://github.com/farcaller/nix-kube-generators/blob/cdb5810a8d5d553cdd0d04fa53378d5105b529b2/lib/default.nix#L49
.. _fakeHash: https://github.com/NixOS/nixpkgs/blob/5b528f99f73c4fad127118a8c1126b5e003b01a9/lib/deprecated.nix#L304

remoteYAMLFile
--------------
``{version, url, hash} -> <derivation: a YAML file>``

Fetch a remote file. Useful for applications distributed as a YAML stream, e.g., the `Gateway API`_.

- **version** (str): The version of the application, which will also be the derivation's version.
- **url** (str): The URL from which to fetch the file.
- **hash** (str): An `SRI-style hash`_.

.. _Gateway API: https://github.com/kubernetes-sigs/gateway-api/releases/tag/v1.0.0

Builders
========

Builder functions build a service derivation.

Builders receive ``name`` and ``namespace`` through the flake builders, so these two variables will be documented once:

- **name** (str): The name of the service. Usually reflected in the label ``app.kubernetes.io/instance``, as well as the derivation's name.
- **namespace** (str): The namespace into which to deploy the service.

derivation
----------
``{name, namespace, src, ...} -> <derivation>``

Copy a derivation verbatim. Useful in conjunction with a fetcher that produces a single file, like ``lib.fetchers.remoteYAMLFile``.

- **src** (derivation): The derivation to copy.

helmChart
---------
``{name, namespace, chart, values?, includeCRDs?, kubeVersion?, apiVersions?} -> <derivation: a YAML file of Helm output>``

Wrapped re-export of `kubelib.fromHelm`_ that sets ``metadata.namespace`` on all templated objects lacking it. As such, its signature is identical to `kubelib.buildHelmChart`_.

- **chart** (derivation): The chart from which to build.
- **values** (attrset, default: ``{}``): Values to pass into the chart.
- **includeCRDs** (bool, default: ``true``): Whether to include CustomResourceDefinitions in the template output.
- **kubeVersion** (str, default: ``pkgs.kubernetes.version``): The Kubernetes version to target.
- **apiVersions** ([str], default: ``[]``): Sets `Capabilities.APIVersions`_.

.. _kubelib.fromHelm: https://github.com/farcaller/nix-kube-generators/blob/cdb5810a8d5d553cdd0d04fa53378d5105b529b2/lib/default.nix#L123
.. _kubelib.buildHelmChart: https://github.com/farcaller/nix-kube-generators/blob/cdb5810a8d5d553cdd0d04fa53378d5105b529b2/lib/default.nix#L82-L90
.. _Capabilities.APIVersions: https://helm.sh/docs/chart_template_guide/builtin_objects/#helm

Flake builders
==============


.. _SRI-style hash: https://nixos.wiki/wiki/Nix_Hash

## Architecture

Services expected to provide custom APIs (e.g.: Gateway API,
Istio, Longhorn) go in ``./system``. All others in ``./services``,
including system-service charts dependent on other APIs.
This prevents infinite recursion when gathering APIs.

Each of the leaves of the ``services`` attrsets is a derivation
(explained better in ``lib/flake-builders.nix``).
Here, they are gathered into one mega-derivation, with Kustomizations
at each level for usage with ``k apply -k $path``.

### namespaces

Assign extra metadata in ``namespaces.nix``. For example,
``svc = {labels."istio.io/rev" = "1-18-1"}``
is the equivalent of
``k label ns/svc istio.io/rev=1-18-1``