RFC: new way of loading certificate files

[f1-outsourcing]

1. nothing searches in private
   #bind 0.0.0.0:8080 ssl crt "${HAPROXY_CRT_DIR}"/dev.marathon.mesos.crt ssl-load-extra-files key "${HAPROXY_KEY_DIR}"/dev.marathon.mesos.key
   #bind 0.0.0.0:8080 ssl crt "${HAPROXY_CRT_DIR}"/dev.marathon.mesos.crt ssl-load-extra-files key
   bind 0.0.0.0:8080 ssl crt "${HAPROXY_CRT_DIR}"/dev.marathon.mesos.crt

2. my key does not end on crt.key, just key (It is obvious from the name in front that it is already the key for that domain)

3. Why can't I specify like any other application my own crt and key file name? I don't get why you even want to automate this. Also the certs and private directories are different on distributions.

[ALERT] 210/124445 (30) : parsing [/etc/haproxy/haproxy.cfg:52] : 'bind 0.0.0.0:8080' : No Private Key found in '/etc/ssl/certs/dev.marathon.mesos.crt' or '/etc/ssl/certs/dev.marathon.mesos.crt.key'.

Is there an config option to specify a key dir and a cert dir?

[wlallemand]

Hello,

This is not a bug and works as designed. Indeed the "crt" option takes a basename which will be used to search other files by adding suffixes (".issuer", ".ocsp", ".key" etc.) It won't try to remove the extension of your basename. https://cbonte.github.io/haproxy-dconv/2.2/configuration.html#ssl-load-extra-files

Is there an config option to specify a key dir and a cert dir?

You can only do that with a cert dir with 'crt-base' but we could probably add a 'key-base' too, that could be backported in 2.2.

The problem with a "key" directive is that it will apply on the whole bind line, and that's not a good idea because it will apply on all crt instances. Only a "key" directive in a crt-list file makes sense.

[f1-outsourcing]

Yes would be very nice, I am now scripting to auto generate the pems. I

If you have key and cert dir and specifiy those files, you do not need to do anything more. Except for ca validation.

You should also not force people the use of names. Although most follow standards, it is just never 'one size fits all'. I don't think I have ever seen any application, forcing me to use basename.key.

I think it is even the opposite of logical what is now implemented. In my case all the file names are the same with a nice 3 letter different extension. Furthermore your basename usage is also inconsistent, logically you should apply .crt to it. So in my case it would be dev.marathon.mesos.crt.crt so I could configure it as dev.marathon.mesos and crt and key are appended.
But then you have the 3rd argument that your configuration file values are not exactly the same as files on disk.

So I would rethink this ssl configuration, stick to file names easier configuration, less coding, less options for mistakes.

[wlallemand]

I agree with you regarding the use of names, but we had some limitations:

1. You can have several "crt" directives on a bind line
2. The directives on a bind line are supposed to apply on the whole line, not in declaration order

This was the original design of the SSL configuration parsing in HAProxy 1.5, I honestly hates it as most people use separate files and directories instead of PEM. When the OCSP was implemented, we couldn't add a "ocsp" directive because of (2), so a .ocsp file is looked up using the basename in "crt". And every features followed the same design after that, becoming worse with the bundles, which could look after ".ecdsa.ocsp" for example.

I won't be able to change the way we look at files because of backward compatibility. But the introduction of the "ssl-load-extra-files" directive was a first step to disable the auto discover and be able to specify the files with new directives in the future.

Specifying the extra files manually by certificate won't be compatible with the "crt" directive, so we will need to use the "crt-list" keyword for that.

A lot of things changed internally in HAProxy, we now have the concept of certificate storage, which is not really visible in the configuration. But what HAProxy does is that it will store all files related to a "crt' in the certificate storage and then load this storage for each new bind line it is used. So maybe what we need is to be able to declare a new certificate storage in a dedicated section. I don't have any definitive proposal for this, I'll probably open a new ticket with some thoughts.

[f1-outsourcing]

Good, interesting background info!

[wlallemand]

After discussing this with several people here are more thoughts about this:

• the ssl-load-extra-files feature is convenient, people like being able to auto-load a .key but adding this at this end of the filename is not convenient. We could add an option to remove the current extension before looking for the .key.

• some people want to be able to search a key in a separate directory, so the "key-base" will be implemented

• other people would like to put their filenames manually without any auto-discovery, with a "key" and "chain" keyword. However this is not compatible with bind lines. I thought the crt-list was a good way to do it, but after some discussions on future improvements we conclude that it will make the configuration worse. What we need is a way to describe a certificate, and a new section will be dedicated to this.

This is an example of what it could be:

certificates
    crt foobar.crt key foobar.key chain foobar.chain
    crt /etc/ssl/certs/site1.crt key /etc/ssl/private/site1.key

listen
    bind *:443 ssl crt foobar.crt # use the crt described in the certificates section

[WRMSRwasTaken]

How about using the resolvers section as a model to be able to define a name per certificates section?

certificates foobar
    crt /etc/ssl/certs/foobar.crt
    key /etc/ssl/private/foobar.key

So you'd just have to pass foobar to other directives (like for example making it compatible with crt-list or being able to specify multiple named certificates to the bind line)

[wlallemand]

We thought about it first, but that's not really convenient for people having thousands of certificates.

What I suggested will be compatible with crt-list, however if we want to use a different name we could use an optional "name" parameter, for example:

certificates
    crt foobar.crt key foobar.key chain foobar.chain
    crt /etc/ssl/certs/site1.crt key /etc/ssl/private/site1.key name site1

listen
    bind *:443 ssl crt foobar.crt crt site1 # use the crt described in the certificates section

[wtarreau]

I'm overall fine with this but I'm convinced that we really need to be able to name the section and probably to make it mandatory, like this:

certificates mysite
    crt foobar.crt key foobar.key chain foobar.chain
    crt /etc/ssl/certs/site1.crt key /etc/ssl/private/site1.key name site1

listen
    bind *:443 ssl crt foobar.crt crt @mysite:site1 # use the crt described in this certificates section

This would allow to easily load all the certs mentioned in such a group (e.g. crt @mysite), and more importantly will ease config management at scale. I don't think it adds any complexity at all, and should avoid difficulties imposed by a centralized list when it comes to multiple applications having their own certs. Typically an application that needs to be deployed could simply be shipped with a single "certificates" section mentioning the various paths and certs.

What I like with this format is that we can later think about extending it, for example by adding:

• import : loads a file into the section
• crt-path : preset the path to load all "crt" files
• key-path : preset the path to load all "key" files
  etc.