Fixed usage of custom location for custom tables

[schlunma]

Description

When specifying a custom config-developer.yml file (config_developer_file: ...) that uses a custom location for custom CMOR tables with

custom:
  cmor_path: ~/my/own/custom_tables

we get the very confusing error

esmvalcore.exceptions.InvalidConfigParameter: `config_developer_file` is not a valid config parameter.

The reasons for this is that the custom: entry does not have a input_dir: since #1835 this leads to a KeyError, which eventually leads to this esmvalcore.exceptions.InvalidConfigParameter through a try/except.

This PR fixes that.

Closes #2051

---

Before you get started

• ☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• 🧪 The new functionality is relevant and scientifically sound
• 🛠 This pull request has a descriptive title and labels
• 🛠 Code is written according to the code quality guidelines
• 🧪 and 🛠 Documentation is available
• 🛠 Unit tests have been added
• 🛠 Changes are backward compatible
• 🛠 Any changed dependencies have been added or removed correctly
• 🛠 The list of authors is up to date
• 🛠 All checks below this pull request were successful

---

To help with the number pull requests:

• 🙏 We kindly ask you to review two other open pull requests in this repository

[codecov]

Codecov Report

Merging #2052 (d505932) into main (1c1e6f1) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main    #2052   +/-   ##
=======================================
  Coverage   92.94%   92.94%           
=======================================
  Files         235      235           
  Lines       12564    12564           
=======================================
+ Hits        11677    11678    +1     
+ Misses        887      886    -1     

Impacted Files	Coverage Δ
esmvalcore/config/_config.py	98.91% <100.00%> (ø)
esmvalcore/config/_validated_config.py	95.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

[bouweandela]

Thanks for looking into this @schlunma! I didn't know we had such a feature. Maybe we should discuss this as part of our wider discussion on configuration at the SMHI workshop ESMValGroup/Community#98. The current custom tables solution leads to all kinds of problems because it mixes CMIP5 and CMIP6 standards. It would be better if we had a per-project configuration of extensions of the CMOR tables, e.g. as an additional project key in config-developer.yml instead of a custom project that may be tacked on to other projects in mysterious ways but can also be used standalone in some cases.

However, if you're keen to have this as a bugfix release in v2.9 I would be happy to approve and we can improve the configuration for a next release.

[schlunma]

I fully agree that the way we use custom tables is certainly not optimal. However, this should be adressed for custom tables in general, i.e., also for those located in our repository.

This PR only fixes the usage of external custom tables (which is used by quite some people), so I would be very happy to have this merged.