Question: How to handle global variables across helmfiles

[AndresPineros]

Is it possible to share global variables between helmfiles in the following manner?

Let's say I have one global variables file called globals.yaml, that has this content:

spring:
  regex: "some value"

dotnet:
  regex: "some other value"

And multiple helmfiles that depending on what we want can grab the spring or the dotnet value (this is what I want to know if is possible):

....
releases:
  - name: myrelease1 
    namespace: default
    chart: mycharts/mychart
    version: 0.1.0
    values:
      - globals.yaml
      - regex: {{ spring.regex }}

and

....
releases:
  - name: myrelease2
    namespace: default
    chart: mycharts/mychart
    version: 0.1.0
    values:
      - globals.yaml
      - regex: {{ dotnet.regex }}

So basically for each release, I can pass the global variable I want to a variable that is already defined in the chart.
Is there a way to do this or simulate this behavior?

[mumoshu]

@AndresPineros Hey! Thanks for trying helmfile.

I've asked about global variables several times so far, and I do understand those and your use-cases are important.

Back to the question, one possible way would be to use an environment variable that points to a kind of "base directory" containing globals.yaml. That is you run this against the top-level helmfile:

$ SOME_BASE_DIR=$(PWD)/values/globals.yaml helmfile apply

Your second sub-helmfile would look like:

....
releases:
  - name: myrelease2
    namespace: default
    chart: mycharts/mychart
    version: 0.1.0
    values:
      - regex: {{ list (requiredEnv "SOME_BASE_DIR") "globals.yaml" | join "" | readFile | fromYaml | getOrNil "dotnet.regex" }}

You'll be interested in #245 or more extremely using an external tool like kapitan 140, for reducing the amount of boilerplate code like this.. 😃

[mumoshu]

More thoughts:

• Using envvars like I've described above makes your helmfile less portable. You need to figure out what envvar is required by running or reading the helmfile
• Adding a convention that globals.yaml is searched upward and recursively won't work when you want multiple globalvars files

[mumoshu]

How about adding a template function findUpward that is used like {{ findUpward "globals.yaml" | readFile }}?

With that you can place globals.yaml at your project root and your helmfile and subhelmfiles can just locate it via findUpward, without having to provide an envvar like SOME_BASE_DIR.

[mumoshu]

@sstarcher @osterman Do you any experience that made you wanting global variables? How do you deal with it?

[sstarcher]

Could this not be solved by environments?

So one issue I have with my current use of globals is that it does not play nice with something like Atlantis. If you change a global it effects all releases not just a local change.

[mumoshu]

Could this not be solved by environments?

Good point! environments helps it. The issue may be how you can easily reference the globals.yaml then. Sub-helmfiles that resides within a deeply nested directory would need:

environments:
  default:
    values:
    - ../../../globals.yaml

My idea is that findUpward simplifies it:

environments:
  default:
    values:
    - {{ findUpward "globals.yaml" }}

[mumoshu]

So one issue I have with my current use of globals is that it does not play nice with something like Atlantis. If you change a global it effects all releases not just a local change.

Similarly to what happens when you use terraform modules and updates to the modules doesn't trigger changes to dependent tf projects?

I was in the impression that atlantis suggests us to use when_modified:

version: 2
projects:
- dir: project1
  autoplan:
    when_modified: ["../modules/**/*.tf", "*.tf*"]

https://www.runatlantis.io/guide/atlantis-yaml-use-cases.html#configuring-autoplanning

[sstarcher]

I don't have that issue due to separating out our terraform modules. I do the following.

• Each module is in it's own github repo and has tests on commit
• A single repo initiates all of those different modules using source to point to the github repo with a specific tag

We don't allow floating things and everything is versioned so that atlantis workaround is not required and people know EXACTLY what is being updated instead of effecting multiple services at once. Which is the exact issue a global introduces where you make a change and it breaks something you know nothing about.

[mumoshu]

Makes sense!

So an equivalent way to manage globals in helmfile would be to version your gloabls.yaml with unique IDs.

A poor man's implementation would look like:

- helmfiles/
  - helmfile.yaml
- globals/
  - globals.v1.yaml
  - globals.v2.yaml

And you reference a single version of globals.yaml(v1 or v2 or whatever) explicitly in your helmfile.yaml, so that people know EXACTLY what's being updated and why.

[sstarcher]

I would lean toward recommending using templating and environments.

• Globals go in the environment
• Use a gotmpl to define it and insert it into your resource