We shouldn't require authors to be in config-references.yml to run a recipe

[ledm]

In the documentation section of recipes, there is an obligatory field: authors. This requires the names listed in the authors section of the recipe to be in the config-references.yml file.

Otherwise, you will get an error like this one:

ValueError: Tag 'lee' does not exist in section 'authors' of /users/modellers/ledm/workspace/ESMValToolTest/ESMValTool/esmvaltool/config-references.yml

This should 100% be a requirement for pushing recipes back to github.

However, this excludes new users from ownership of their own recipe if they don't have access to the source code. For instance they are running a shared installation or ESMValTool, or they are using the user installation instead of the develop installation.

Can we relax this requirement when running a recipe, but keep it for submitting recipes?

@valeriupredoi and I spoke about it earlier today. Any thoughts @mattiarighi, @bouweandela ?

[valeriupredoi]

Right on @ledm ! This is particularily needed when the tool is centrally installed on a large data-store cluster like Jasmin or DKRZ - brand new users will want to first run an existing recipe with that sourceable installation and then write their own recipe - they have the option to keep the provenance info on a given existing recipe (which is not nice) or they should be able to use a new_author or newbie or jasmin-user before they get more serious and get added as authors themselves

[mattiarighi]

What about adding an entry in config-references.yml for such cases, something like:

undef:
  name: undefined user
  institute: -

[valeriupredoi]

sounds good @mattiarighi - can it be Batman instead? 😁
@bouweandela you the main provenance person, what say you (about putting this in, not Batman)

[mattiarighi]

sounds good @mattiarighi - can it be Batman instead?

No, that might disappoint Marvel fans.

[bouweandela]

I'm not so sure about this issue: I think it is best if people check out the diagnostics repository (after the planned split) and start developing in there, this will make it much easier to later create a pull request to add what you're developing back to esmvaltool. If you just want to try out esmvaltool, I think it's not necessary to put yourself as an author on the recipe, it only makes sense to do that by the time you want to publish your results and by that time it would be good if your code is in a shape that a pull request can be made anyway. However, I'm not a scientist of course, so maybe I'm missing something here..

[ledm]

I've got to disagree with you @bouweandela. It takes several hours to install a personal copy of ESMValTool from scratch. When host a 4 hour ESMValTool tutorial, we don't want to spend half of it installing ESMVallTol. This means that nearly all new users will always use a central installation ESMValTool, at least on day one.

Say you're a new user: you take some one else's recipe, changes the few bits that you can and run it using a central installation of ESMValTool. Your recipe will then fail immediately because your name is not on the accepted authorship list. To that user, this will feel like we are purposely preventing them from contributing, and it will make them feel excluded and they will have less drive to continue using ESMValTool.

At the very least, we can make it so that the authors field is not a required in a recipe. People can then just leave it blank until they start a PR.

[bouweandela]

Say you're a new user: you take some one else's recipe, changes the few bits that you can and run it using a central installation of ESMValTool.

After we've split the core and diagnostics installation (in preparation to actually splitting the repositories), it should be easy to use a centrally installed version of esmvaltool with a custom diagnostics scripts location, so long as you're not changing the dependencies, because the diagnostic scripts do not actually need to be installed, just their dependencies.

It takes several hours to install a personal copy of ESMValTool from scratch

It' that's the case, this is a serious problem and something that probably needs fixing more urgently. It takes me about 15 minutes to install esmvaltool (or less, depending on how fast the machine is), but I guess I'm not a good example of a new user.

[ledm]

With regards to an installation from scratch, it takes a while to install on jasmin, as users need to install miniconda2 in addition to all the packages listed in environment.yml.

In all cases, a full installation can be a significant hurdle to jump for a new user.

We're going to be hosting an ESMValTool or UKESM users in mid-june so it would be great to have resolved this issue before then. (I assume that the split will occur in the first week of june at the DLR meeting.)

[valeriupredoi]

yes, I fully support a john-doe user group

[bouweandela]

I took this into account when splitting the packages. @ledm After ESMValGroup/ESMValTool#1148 is merged, can you please test if things now work as you would like?