Kubernetes with jsonnet and Kapitan

In my previous posts, I explained the basic of Kapitan and how it can be used to drive complex deployments of Kubernetes. This post will pick up where we left off on the Introduction to Kapitan, and go more into details on how to use jsonnet to manage Kubernetes configurations.

https://jsonnet.org

To recap on how to use Kapitan, please refer to my previous post.

Introduction to Kapitan

Please star https://github.com/deepmind/kapitan and leave a comment!

So why jsonnet, or rather: why not jinja/go templates?

In the initial deployment scripts we were using before Kapitan, we were using jinja2 to generate templates. This was before we learnt about Helm, which uses go templates but I think from that point of view shares the same issues.

(Forgive me but I do not have anymore jinja2 templates examples so I will use golang examples instead taken from random helm repositories)

I came to dislike “text based” for the following reasons:

YAML relies on indentation

This forces you to use things like the following

{{- toYaml .Values.nodeSelector | nindent 8 }}

Quite frankly, the need/use of whitespace control is simply revolting, let alone the fact that you need to count how much to indent something by.

Generated YAML might not be well formed

Text based templates do not understand what they are generating, and they have no way to understand whether their output is well formed YAML.

Reusability: You cannot share common bits

Or rather, it is very difficult. Using templates you can achieve that somehow but it is all very clunky. Not practical and difficult to read.

Repetition: Making global changes to all your manifests.

Same as before really, but repeating it again for another reason. If you want to add something to all your manifests, something like an annotation, you would need to edit all your files. No thanks.

Ultimately, this is what you end up with: statefulset.yaml. This might be ok if you never have to adapt it and you are fine with the default/off-the-shelf version, but if you need to extend it then you are out of luck.

Using jsonnet as an alternative

Jsonnet offers more flexibility than text templating systems. It is a language that supports object orientation, error handling, imports, functions.

JSON is used to describe data literally. Jsonnet extends JSON. Jsonnet has relaxed syntax and comments to make it easier for humans to write. Jsonnet also adds constructs for generating, translating and refining data. Because Jsonnet is an extension of JSON, valid JSON is also valid Jsonnet that just emits that JSON unchanged. (from jsonnet comparison)

Let’s see some examples, then we can dive straight back into Kapitan.

Let’s consider the JSON version of a deployment, in this case, the cod deployment from our Kapitan examples.

{
  "apiVersion": "apps/v1beta1",
  "kind": "Deployment",
  "metadata": {
    "name": "cod",
    "namespace": "dev-sea"
  },
  "spec": {
    "replicas": 1,
    "template": {
      "metadata": {
        "labels": {
          "app": "cod"
        }
      },
      "spec": {
        "containers": [
          {
            "args": [
              "--verbose=True"
            ],
            "image": "alledm/cod:v2.0.0",
            "name": "cod",
          }
        ]
      }
    }
  }
}

This is already valid jsonnet! Obviously, if I was to ask you to use it like this, you would immediately walk away from this post. Please don’t! Hang on!

If we were to compile the above, we would get this YAML equivalent.

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: cod
  namespace: dev-sea
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: cod
    spec:
      containers:
        - args:
            - '--verbose=True'
          image: 'alledm/cod:v2.0.0'
          name: cod

Even just with this simple example, we have an immediate improvement over the text based templating. Because the YAML generated by jsonnet will always be well formed!

Let’s spice it up, gently. Let’s move all the important bits out into variables.

local name = "cod";
local image = "alledm/cod:v2.0.0";
local replicas = 1;
local namespace = "dev-sea";
local args = ["--verbose=True"];
{
  "apiVersion": "apps/v1beta1",
  "kind": "Deployment",
  "metadata": {
    "name": name,
    "namespace": namespace
  },
  "spec": {
    "replicas": replicas,
    "template": {
      "metadata": {
        "labels": {
          "app": name
        }
      },
      "spec": {
        "containers": [
          {
            "args": args,
            "image": image,
            "name": name,
          }
        ]
      }
    }
  }
}

As you can see, we have gently introduced variables into the jsonnet file.

Hooking up the Kapitan inventory

We can now go one extra step, and import from the Kapitan inventory, the content for those variables.

local kap = import "lib/kapitan.libjsonnet";
local inv = kap.inventory();
local namespace = inv.parameters.namespace;
local name      = "cod";
local image     = inv.parameters[name].image;
local replicas  = inv.parameters[name].replicas;
local args      = inv.parameters[name].args;
{
  "apiVersion": "apps/v1beta1",
  "kind": "Deployment",
  "metadata": {
    "name": name,
    "namespace": namespace
  },
  "spec": {
    "replicas": replicas,
    "template": {
      "metadata": {
        "labels": {
          "app": name
        }
      },
      "spec": {
        "containers": [
          {
            "args": args,
            "image": image,
            "name": name,
          }
        ]
      }
    }
  }
}

We are still not taking advantage of the Jsonnet language.
Using a library (like https://github.com/bitnami-labs/kube-libsonnet) we could make it even better. See an even better example here:

An Example of Real Kubernetes: Bitnami

local kube = import "lib/kube.libjsonnet";
local kap = import "lib/kapitan.libjsonnet";
local inv = kap.inventory();
local namespace = inv.parameters.namespace;
local name      = "cod";
local image     = inv.parameters[name].image;
local replicas  = inv.parameters[name].replicas;
local args      = inv.parameters[name].args;
local container = kube.Container(name) {
  image: image,
  args: args
};
kube.Deployment(name) {
  spec+: {
    replicas: replicas,
    template+: { 
      spec+: { 
        containers: [ container ]
      }
    }
  }
}

The +: field syntax overrides deeply nested fields. See the Tutorial

This can be improved further by expanding the library or by creating your own components. For instance, imagine something like:

local my_lib = import "lib/my_lib.libjsonnet";
local kap = import "lib/kapitan.libjsonnet";
local inv = kap.inventory();
local namespace = inv.parameters.namespace;
local name      = "cod";
local image     = inv.parameters[name].image;
local replicas  = inv.parameters[name].replicas;
local args      = inv.parameters[name].args;
local container = my_lib.Container(name, image, args);
my_lib.SuperSimpleDeployment(name, replicas, container)

Learn more about jsonnet on its official website: jsonnet.org

Please star https://github.com/deepmind/kapitan and leave a comment.
Also read my other posts!

Let kapitan take the helm of Kubernetes

Introduction to Kapitan

Additional posts about jsonnet

Simplify Your CI Pipeline Configuration with Jsonnet

New! 10x more awesome ways to write JSON configuration using Jsonnet.