Rerun experiments re-runs on all previous versions of model/protocol instead of just latest

[mirams]

When uploading a new version of a model/protocol the 're-run experiments using this version' is really nice, but it is re-running all the old protocols/models and not just the latest one, which is probably what we want the default behaviour to be.

[jonc125]

It already only re-runs experiments that involved the immediate parent of the new version, is that not what you mean?

Or is it that if you upload a new version of a protocol say, and the previous version was run with 3 versions of the same model, you'll get 3 new experiments submitted, whereas you want only 1?

[mirams]

Yeah, the second. So if there are 10 previous versions of a model and I upload a new protocol, all 10 will run instead of just most recent version of a model.

[jonc125]

Hmm, it doesn't look easy with the current DB structure to ensure you get just the latest version of the entity. If there are multiple versions however, it's easy to get the one for which the Experiment was created most recently, which is probably going to be the same almost all the time I guess? Would that be reasonable?

[mirams]

Doesn't (e.g.) this view always give you a list of them:
https://scrambler.cs.ox.ac.uk/entities/protocols/51/versions/cf3763cd8c295b3f093be42c5129f4353fb354cb/compare

[mirams]

Oh, no it doesn't! It's listing v3 of Decker when a newer one is available! That's not ideal...

and the Decker versions:

and this protocol has run recently with this newer version of Decker:
https://scrambler.cs.ox.ac.uk/experiments/3463/versions/3595/outputs_Relative_APD90_gnuplot_data.csv/displayPlotFlot

[jonc125]

Yes, you can get a list of all versions easily enough. It's just that in the place where it's finding experiments to run, it's looking at a table from which it's not easy to get to that list. So it would take a more substantial change (and thinking!) to do it. And possibly be slightly slower, though that shouldn't be too significant.

[jonc125]

It will only re-run experiments that the user submitting the new version can see, incidentally.

[jonc125]

This might be easier to change now #250 is done, actually.

[jonc125]

The simplest logic to implement, and possibly what users most expect, isn't strictly speaking to re-run experiments run with the immediately preceding version. Instead, we could:

1. Find all experiments run with the entity for which we're adding a new version (suppose it's a model for the sake of argument; if it's a protocol being updated the logic is analogous).
2. Get the list of corresponding protocol IDs.
3. We now know both our model ID, and all protocols that have previously had any version run on any version of this model.
4. Run our new model version under the latest (visible) version of all those protocols.

Does that seem reasonable @mirams ?

If so, we might want to update the checkbox description, although it could get quite long! Currently it's "Re-run experiments involving the previous version of this model". We could say "Re-run the latest versions of protocols previously run on this model" or somesuch? Or just keep as is?

Implementing the change would need to happen in entities.processing.record_experiments_to_run. For the logic above, we could do something like:

if entity.other_type == entity.ENTITY_TYPE_MODEL:
    other_model = ModelEntity
else:
    other_model = ProtocolEntity
search_args = {
    entity.other_type + '_experiments__' + entity.entity_type: entity,
}
other_entities = other_model.objects.visible_to_user(user).filter(**search_args)
for other_entity in other_entities:
    # Look for latest visible version
    for other_version in other_entity.cachedentity.versions.reverse():
        if other_entity.is_version_visible_to_user(other_version.sha, user):
            # Record the new experiment to run
            kwargs = {
                'submitter': user,
                entity.other_type: other_entity,
                other_entity + '_version': other_version.sha,
            }
            kwargs.update(new_version_kwargs)
            PlannedExperiment.objects.get_or_create(**kwargs)
            break

[mirams]

Yeah, I think so.

I'm pretty sure the old WL did this sort of thing, and had a button to select just the latest version or all versions e.g. under Compare Models on https://travis.cs.ox.ac.uk/FunctionalCuration/protocol/INaIVCurve/10/withvoltageclampplots/112

So this logic should appear on

• Selecting a model and comparing protocols
• Selecting a protocol and comparing models
• When uploading a new model/protocol version and deciding what experiments to re-run
  and probably some new fitting spec things too?

[jonc125]

We do have that kind of view already, e.g. https://scrambler.cs.ox.ac.uk/entities/protocols/2/versions/v1/compare

For re-running, do we want the option to re-run with every earlier combination? Or only the latest visible as described above?