Complete your Kubernetes resources as code using helmfile and raw Helm chart

Helm is a great tool to deploy popular services and applications to your Kubernetes cluster, but from the moment I started using it I had a feeleng that something is missing. You could easily configure and install whatever, but each chart that you use is a separate unit and there is no code containing all resources. This practicaly means in case of disaster it was still not easy to recreate the cluster, at least not in a quick timeframe. Another contra is that in order to pin exact image versions, you would have to edit each values file every time you want to upgrade something.

Fortunatelly, there is a tool called helmfile which basically can integrate all your Helm charts allowing full control over installed resources, image versions and settings. You may say yeah, but there may be some more resources defined or applied by kubectl and this can't be defined neither in Helm nor in helmfile. Wrong, there is a Helm chart called raw, which is currently in incubator phase, but it allows documenting all additional resources like secrets, TLS certificates, external services, etc. Combining these two it is really possible to finally have all resources defined as code and even better, it allows full control over upgrades and current state of the cluster.

If you install Helm plugin diff, you may see the differences between the current cluster state and the desired state and decide before hand weather you want to sync or not. helmfile is stateless and it will always fetch the real state inside of Kube cluster. So, I strongly suggest to install this plugin and then you can use helmfile subcommand diff.

Another important thing is how to protect your secrets. Of course they should not finish in your Git repository. It is maybe worth to mention that there is a nice tool called gitleaks, which can check your Git repository for leaked secrets. But let's go back to the topic how to protect your secrets. You can do it more ways, of course and one of the best and probably most complicated would be using of Vault. If you are not runnig an enterprise Kubernetes cluster, it is probably good enough to use either helm-secrets plugin or environments variables managed by direnv.

The third option is the simplest. You just define your secrets as environment variables in a file called .envrc in your project directory and import them by:

direnv allow .

In your helmfiles you can then use these wherever you have value defined as:

{{ requiredEnv "ENV_VAR_NAME" }}

Here is an example of docker-registry declaration in helmfile:

  - name: docker-registry
    namespace: default
    chart: stable/docker-registry
    - name: secrets.htpasswd
      value: {{ requiredEnv "DOCKER_REGISTRY_BASIC_AUTH_PASSWD" }}

You can apply this by:

helmfile -f filename.yaml sync

And the final note is - don't forget tu put .envrc in your .gitignore file ?