Restart hyperopt

[Cmurilochem]

This PR addresses the issue of restarting an hyperoptimization with the Hyperopt library as discussed in #1800.

[Radonirinaunimi]

The changes here seem to me more complicated than it should, but we can discuss further this afternoon. And btw, as it stands, I don't think this PR should point to master.

[RoyStegeman]

Hi @Cmurilochem the diffs are a bit difficult to understand, it seems that you have a lot of commits that are already included in master while you're trying to work/merge on top of a branch that does not have these commits?

Perhaps this should be pointed to master and @Radonirinaunimi change should be undone or mabye the hyperopt_loss branch should be rebased (though if this was branched off from master you may need to rebase this on top of hyperopt_loss if that's what you want)? Either way, before continuing your progress you may want to make sure the history makes sense and you will be able to safely merge by the time that's needed.

[Radonirinaunimi]

Hi @Cmurilochem the diffs are a bit difficult to understand, it seems that you have a lot of commits that are already included in master while you're trying to work/merge on top of a branch that does not have these commits?

Perhaps this should be pointed to master and @Radonirinaunimi change should be undone or mabye the hyperopt_loss branch should be rebased (though if this was branched off from master you may need to rebase this on top of hyperopt_loss if that's what you want)? Either way, before continuing your progress you may want to make sure the history makes sense and you will be able to safely merge by the time that's needed.

Indeed! Now it is very complicated to understand which changes genuinely regards the continuation of hyperopt. As I mentioned a while ago, you basically have two options:

• Point the following PR to master. This would be feasible because even the current master could benefit from this feature. But for this you'll have to undo the changes that are coming from the hyperopt_loss and other branches.
• Point this PR to hyperopt_loss branch but then never merge/rebase stuff from the various merges/fixes.

At this point I'd say the first option is the most ideal, and once merged could be propagated throughout the different branches. There should be only single (main) branch from which all the various changes are collected (if needed) but this should not bet that branch.

[Radonirinaunimi]

So, that is, the diffs here should only show the changes regarding the continuation of hyperopt and the documentation describing how it is used.