Add parameter_files and parameters configuration options to stack definition

[stevehodgkiss]

Following on from an earlier spike, but without the stack definition moving to its own file for better backwards compatibility. #330.

Added parameters_dir config

This matches what we have for template_dir and is useful for the parameter_files feature.

parameter_files - an array of explicit parameter file locations based from parameter_dir

Adds the ability to explicitly define parameter file locations in stack definitions. It's more obvious this way, and it can also be used to massively reduce duplication between stacks that follow a blue/green setup. For example, if we have myapp-blue and myapp-green, they could share a parameter file for the majority of parameters, whereas currently that's not possible without duplicating all the parameters.

stacks:
  us-east-1:
    myapp-blue:
      parameter_files:
        - myapp.yml
        - myapp-blue.yml
      template: myapp.rb

parameters hash key can add parameters inline with the rest of the stack definition

This can be useful when switching to a new stack that only needs a small change in parameters:

stacks:
  us-east-1:
    aws-elasticsearch-green:
      parameter_files:
        - aws-elasticsearch.yml
      template: aws-elasticsearch.rb
    aws-elasticsearch-blue:
      parameter_files:
        - aws-elasticsearch.yml
      parameters:
        engine_version: 7.4
      template: aws-elasticsearch.rb

[petervandoros]

Nice one! 👍 with a couple comments for your consideration.

Also, could you update the readme to include info on the new functionality?

[petervandoros]

Why use the conditional assignment operator in the initializer? Is @parameters expected to be set before the initializer is run?

[stevehodgkiss]

The call to super above sets values from the yaml file, so it would either be set from that point or not, in which case the default is {}.

[patrobinson]

Not sure what this test achieves?

[stevehodgkiss]

It checks the contents match by comparing parsed hashes rather than strings. For some reason apply_with_s3.feature started failing, and master and has the same issue - https://travis-ci.org/github/envato/stack_master/jobs/692383878#L2514 Maybe it's related to a sparkleformation upgrade?

[orien]

Awesome stuff!

I notice that if I specify an explicit parameter file, but don't provide a parameter needed by a template I get the old validation error message:

Parameters will be read from files matching the following globs:
 - parameters/stackname.y*ml
 - parameters/us-east-1/stackname.y*ml

This'll likely cause confusion given those globs aren't used when explicitly specifying the parameter files. I think we should update this message to include only the defined parameter files.

[orien]

Looks great. Thanks for this.

Do you think we should also support compile-time parameters in the stack_master.yml?

[stevehodgkiss]

Do you think we should also support compile-time parameters in the stack_master.yml?

I think they would be supported already due to the way compile_time_parameters are implemented (as a special parameter).

parameters:
  param: 1
  compile_time_parameters:
    param: 1

[orien]

Oh I see. I tried it and you're quite right, it does just work!