How to Change Your Kubernetes Deployment File Dynamically Using Kustomize

Photo by Andrew Karn on Unsplash

There are tons of cases when we want to define our Kubernetes deployment file dynamically. For example, when working with multiple environments, we sometimes need different environment variables for each environment, different amounts of replicas, different CronJob scheduling, different domain, different environment variables and so on…

In this example, we will see how we can use Kustomize to define CronJob with a different environment variable and namespace for each environment.

To begin, we will create a deployment.yaml file and define our CronJob in it.

deployment.yaml defining a CronJob resource

When working with Kustomize, we can customize our raw YAML deployment file easily. Let’s start with defining a base folder and a folder for each environment, and place our deployment.yaml file in the base folder.

Basic folder structure when using kustomize

Kustomize works by having the following kustomization files:

• The base kustomization file, which contains a reference to all the resources files in our project, all the “regular” kubernetes deployment files. This file is placed in the “base” directory.

The base kustomization.yaml file

• The root kustomization.yaml file, which contains references to all the different available environments. In our example, this file is placed in the root deployment directory, my_project,

The root kustomization.yaml file

• The different environment kustomization files, which contain the path to the base directory, where the base kustomization file exists.
  Also, this is the place where we will define our specific configuration for each environment.
  These files are placed in the specific environment directory they belong to, for example, production or staging.

The production environment kustomization.yaml file

The staging environment kustomization.yaml file

Our directories and files structure should look as follows:

We would like to add an environment variable, which will contain a different value for each environment, prod for production and stg for staging.

We will use the Kustomize patchesJson6902 feature.

The JSON patch contains:

• The target field defines the resource that we want to patch. We would like to change our CronJob so we will define the kind:CronJob, the name will be the name of our CronJob, defining under metadata/name and apiVersion will be break into group and version.
• The path is the file name where we define the new needed environment variable.

The kustomization.yaml staging file

The JSON patch allows us to either add a new section using the add opcode, or replace the existing section using the replace opcode. Here we choose to use add as we want to add an additional environment variable to the environment variables list.

We will now create the env_variable.yaml file with contains the patch data:

• op: the opcode wanted, add or replace.
• path: the location of the environment variables list, the env. In our case it is under /spec/jobTemplate/spec/template/spec/containers/0/env/0 . The first zero represents the first container and the second zero represents the first element in the environment variables list.
• value: the value we want to add, the environment variable name and value.

And we are finally done! The only thing left to do is to deploy, using the commands kubectl apply -k staging for staging and kubectl apply -k production for production.

Good luck :-)