Rename Experiment as Prediction

[jonc125]

Probably best done when all current PRs have been merged!

• Rename Django app
• Rename DB model and create/do migrations
• Rename corresponding template files, JS code, etc?
  • But also consider how much of these might be common with ExperimentalData and FittingResult
• Change text in pages, etc.
• Change comments in code, docstrings, etc.

[mirams]

Hmmm, I'm wondering whether we go with something that looks like:

Virtual Experiment
    /          \
Prediction      Fit

Since the separation of Fits and Predictions is one of my major reasons for the WL tracking model fitting and validation process...

[jonc125]

The idea from the workshop last year was to have it as

             Dataset
            /   |   \__ FittingResult
           /    \        
Prediction     ExperimentalData

[mirams]

OK, then you want to be able to compare ExperimentalData with FittingResults and Predictions, and indeed other ExperimentalData...

[jonc125]

You definitely want to be able to compare ExperimentalData with Prediction (and ED with ED, P with P) but probably not with FittingResult - since the latter will be parameter distributions. But (not for the workshop!) you'd want to be able to do forward simulation of a model, drawing parameters from a FittingResult (either single run, or lots of runs to produce histogram of outputs), and compare to either P or ED (or both).

[mirams]

Hmmm, if the fit is a distribution the predictions will be too, but makes sense to be able to view (say) the most likely parameter set in both fit and prediction. I guess this might be part of the fitting spec, but nice to be able to do it post-hoc too.

[jonc125]

I'm not sure I understand you. A prediction (even one that results in a distribution) wouldn't give you a most likely parameter set - it gives you the result of simulating (and post-processing) the model. This is a different category of 'thing' from a distribution over the model parameters. So I don't see why you'd want to compare them?

You want to be able to compare different parameter sets (whether different fits, pre-determined, etc) with each other. And you want to be able to compare different kinds of prediction against each other and experimental data. But I can't see why you'd want to compare something from one category against something from the other.

[mirams]

I think we are thinking of FittingResult as a different thing. Using your meaning, I am thinking of a structure like this:

                  ======Dataset==========
                 /               \      \
                /                 \       Experimental Data
               /      FittingResult [Param distbn]
   Virtual Experiment
         /   \
Prediction    Fit  

Where Fit is the outcome of a FittingSpec in that it is the simulated experiment(s) that the objective/MCMC uses to get the FittingResult parameter set(s). In that model development jargon sense it is no longer a [validation] Prediction, it's a [training] Fit. In the practical programming sense Prediction == Fit for all intents and purposes, as they are all just VirtualExperiments, but I'd like to track which is which and make it obvious!

[jonc125]

OK, I think I understand now. So you'd like to record the final forward simulation result used in the fitting process to compare against the experimental data? But presumably not any of the interim results from earlier iterations? And if you're doing particle swarm or ABC, you have multiple 'final' Fits (one per particle in the distribution) so would you keep all of them, or just do an extra simulation with the distribution means at the end?

[mirams]

Exactly! And when you happen to look at the "Prediction" for this model/protocol something knows it isn't a [validation] prediction but is really a [training] fit [matching up well by design]!