Update GS experiments with dvc exp init

[iesahin]

This one is migrated from Notion. I have added a hidden section on hyperparameters there.

I have modified here:

• It starts with dvc exp init now. Without mentioning any pipelines stuff or dvc.yaml

• I'm planning to remove "If you've run repro before" hidden section, the section where we tell how to prepare pipelines for arbitrary projects, and keep the hyperparameter stuff to a minimum.

Closes #3072
Closes #3118

This one will be closed after merging the following small PRs:

• start: add a section about metrics to experiments #3197
• start: use exp init instead of stage add #3196
• start: Add a parameters hidden section to experiments to merge params/metrics into experiments #3163

[dberenbaum]

Let me know if you think this direction makes sense @iesahin!

[iesahin]

I'm copying @jorgeorpinel 's comments in Notion:

TBH I'm not sure I love hidden sections for metrics/params. They're usually reserved for off-topic implementation details. Maybe some of the metrics info can be collapsed but a basic description that is relevant to the trail should be in a regular paragraph, I think.

About the sentence starting with "This experiment is then associated with the values in parameters file":

Parameters mention start all the way up here. If we're going to insert params info after the exp show table, is there any way to avoid the concept before it? Let's try to all of the params info until after the table so it's consolidated and more logical.

[iesahin]

It looks we still need to use dvc init before dvc exp init, right?

dvc exp init python3 src/train.py
ERROR: you are not inside of a DVC repository (checked up to mount point '/home/iex/work')

Is there any work to check whether we're inside a DVC repository and dvc init if we're not? @dberenbaum

[dberenbaum]

It looks we still need to use dvc init before dvc exp init, right?

dvc exp init python3 src/train.py
ERROR: you are not inside of a DVC repository (checked up to mount point '/home/iex/work')

Is there any work to check whether we're inside a DVC repository and dvc init if we're not? @dberenbaum

This has been deprioritized for now, so dvc init is still needed.

[iesahin]

This one is ready for review @shcheklein @dberenbaum @jorgeorpinel @julieg18

[iesahin]

Converted to draft. Will work on these in separate branches and this will be closed after other PRs are submitted.

[iesahin]

#3112 modified to include --drop (but not --keep) in experiments.md instead of --include-params etc. I'll base the modifications on the current master and won't deal with it again.

[iesahin]

This section is in #3163

[jorgeorpinel]

#3163 is merged. Should master be merged into this branch so this change goes away, and resolve the comment?

[jorgeorpinel]

#3051 is reviewed & merged bit by bit. It will have 2 or 3 more PRs that will apply all the changes.

Interesting method. Similar to a nested PR or PR series but less messy. However, it can also be confusing to review in 2 places.

What's the exact use of this draft if it's split into other PRs? Should anyone be looking at the changes? What parts)? Thanks

[iesahin]

This is for my own tracking. For longer documents, I can rebase time to time to see the remaining parts. For this one, I don't expect any review. It's like a guide to move the document.

[iesahin]

This one is completed, now GS:Experiments starts with dvc exp init. Thank you for your reviews @jorgeorpinel @dberenbaum @shcheklein