From b0b323c4fe30f702a5faea364915f668cbf9b2f3 Mon Sep 17 00:00:00 2001 From: Himanshu Nailwal Date: Sat, 25 May 2019 23:24:50 +0530 Subject: [PATCH 1/5] meta entry updated --- static/docs/user-guide/dvc-file-format.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/static/docs/user-guide/dvc-file-format.md b/static/docs/user-guide/dvc-file-format.md index c26765e797..bf79bb68fc 100644 --- a/static/docs/user-guide/dvc-file-format.md +++ b/static/docs/user-guide/dvc-file-format.md @@ -30,6 +30,10 @@ outs: xpath: AUC path: metrics.json locked: True +# Comments are also persisted between dvc repro command. +meta: # key to contain arbitary user data + name: John + email: john@xyz.com ``` ## Structure @@ -60,3 +64,10 @@ A metric entry consists of such fields: - `type`: type of the metrics file (e.g. raw/json/tsv/htsv/csv/hcsv); - `xpath`: path within the metrics file to the metrics data(e.g. `AUC.value` for `{"AUC": {"value": 0.624321}}`); + +A meta entry consists of `key:value` pairs such as `name: john`. A meta entry +can have any structure and contain any number of attributes. `"meta: string"` is +also possible, it doesn't necessarily need to contain dictionary always. + +Comments can be added to the .dvc file using `# comment` syntax. Comments are +preserved between multiple execution of `dvc repro ` command. From cb0770e36114a6a084800b712f9e59a01bf6edee Mon Sep 17 00:00:00 2001 From: Himanshu Nailwal Date: Mon, 27 May 2019 06:08:32 +0530 Subject: [PATCH 2/5] comments detail improved --- static/docs/user-guide/dvc-file-format.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/static/docs/user-guide/dvc-file-format.md b/static/docs/user-guide/dvc-file-format.md index bf79bb68fc..8002cb19af 100644 --- a/static/docs/user-guide/dvc-file-format.md +++ b/static/docs/user-guide/dvc-file-format.md @@ -69,5 +69,6 @@ A meta entry consists of `key:value` pairs such as `name: john`. A meta entry can have any structure and contain any number of attributes. `"meta: string"` is also possible, it doesn't necessarily need to contain dictionary always. -Comments can be added to the .dvc file using `# comment` syntax. Comments are -preserved between multiple execution of `dvc repro ` command. +Comments can be added to the .dvc file using `# comment` syntax. Comments +and meta values are preserved between multiple executions of `dvc repro` and +`dvc commit` commands. From 09c5c20e65902daa2bf19dd8532de0192a2b725d Mon Sep 17 00:00:00 2001 From: Himanshu Nailwal Date: Tue, 28 May 2019 13:11:11 +0530 Subject: [PATCH 3/5] updated dvc-file-format --- static/docs/user-guide/dvc-file-format.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/static/docs/user-guide/dvc-file-format.md b/static/docs/user-guide/dvc-file-format.md index 8002cb19af..d91619c2e8 100644 --- a/static/docs/user-guide/dvc-file-format.md +++ b/static/docs/user-guide/dvc-file-format.md @@ -30,8 +30,10 @@ outs: xpath: AUC path: metrics.json locked: True -# Comments are also persisted between dvc repro command. -meta: # key to contain arbitary user data + +#Comments are persisted between multiple executions of dvc repro/commit. +#Comments are not persisted between multiple executions of dvc run/add/import. + meta: # key to contain arbitary user data name: John email: john@xyz.com ``` @@ -69,6 +71,10 @@ A meta entry consists of `key:value` pairs such as `name: john`. A meta entry can have any structure and contain any number of attributes. `"meta: string"` is also possible, it doesn't necessarily need to contain dictionary always. -Comments can be added to the .dvc file using `# comment` syntax. Comments -and meta values are preserved between multiple executions of `dvc repro` and -`dvc commit` commands. +Comments can be added to the .dvc file using `# comment` syntax. + +Comments and meta values are preserved between multiple executions of `dvc repro` + and `dvc commit` commands. + +If user overwrites the file, comments and meta values are not preserved between +multiple executions of `dvc run`,`dvc add`,`dvc import` commands. From 8ebae87f6f977ff94cede322b4ec74df5b8832c0 Mon Sep 17 00:00:00 2001 From: Himanshu Nailwal Date: Tue, 28 May 2019 14:11:04 +0530 Subject: [PATCH 4/5] meta-comment info updated --- static/docs/commands-reference/add.md | 7 +++++++ static/docs/commands-reference/import.md | 2 ++ static/docs/commands-reference/run.md | 2 ++ 3 files changed, 11 insertions(+) diff --git a/static/docs/commands-reference/add.md b/static/docs/commands-reference/add.md index 5c6cc4a782..ed087c2e1c 100644 --- a/static/docs/commands-reference/add.md +++ b/static/docs/commands-reference/add.md @@ -126,11 +126,18 @@ outs: md5: d8acabbfd4ee51c95da5d7628c7ef74b metric: false path: data.xml.jpg +meta: #key to contain arbitary user data + name: John + email: john@xyz.com ``` This is a standard DVC stage file with only an `outs` entry. The checksum should correspond to an entry in the cache. +If user overwrites the `.dvc` file, comments and meta values are not preserved +between multiple executions of `dvc add` command. + + ```dvc $ file .dvc/cache/d8/acabbfd4ee51c95da5d7628c7ef74b diff --git a/static/docs/commands-reference/import.md b/static/docs/commands-reference/import.md index 9749ce0894..387427f0ce 100644 --- a/static/docs/commands-reference/import.md +++ b/static/docs/commands-reference/import.md @@ -195,6 +195,8 @@ The `etag` field in the DVC file contains the ETag recorded from the HTTP request. If the remote file changes, the ETag changes, letting DVC know when the file has changed. +While executing `dvc import` command, if user overwrites the `.dvc` file, +comments and meta values are not preserved between multiple executions. ## Example: Detecting remote file changes What if that remote file is one which will be updated regularly? The project diff --git a/static/docs/commands-reference/run.md b/static/docs/commands-reference/run.md index cfe9ffe1ad..b6006e8f86 100644 --- a/static/docs/commands-reference/run.md +++ b/static/docs/commands-reference/run.md @@ -159,6 +159,8 @@ To track the changes with git run: git add .gitignore metric.dvc ``` +While executing `dvc run` command, if user overwrites the `.dvc` file, comments +and meta values are not preserved between multiple executions. - Execute a Python script as a DVC pipeline step. Stage file name is not specified, so a `model.p.dvc` stage file is created: From e483bb498d838554e882b2820e74f1aae3d46b25 Mon Sep 17 00:00:00 2001 From: Himanshu Nailwal Date: Wed, 29 May 2019 08:50:18 +0530 Subject: [PATCH 5/5] changes requested --- static/docs/commands-reference/import.md | 3 ++- static/docs/user-guide/dvc-file-format.md | 11 +++++------ 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/static/docs/commands-reference/import.md b/static/docs/commands-reference/import.md index 387427f0ce..dafcd6034a 100644 --- a/static/docs/commands-reference/import.md +++ b/static/docs/commands-reference/import.md @@ -196,7 +196,8 @@ request. If the remote file changes, the ETag changes, letting DVC know when the file has changed. While executing `dvc import` command, if user overwrites the `.dvc` file, -comments and meta values are not preserved between multiple executions. +comments and meta values are not preserved between multiple executions. + ## Example: Detecting remote file changes What if that remote file is one which will be updated regularly? The project diff --git a/static/docs/user-guide/dvc-file-format.md b/static/docs/user-guide/dvc-file-format.md index d91619c2e8..1d5b35d142 100644 --- a/static/docs/user-guide/dvc-file-format.md +++ b/static/docs/user-guide/dvc-file-format.md @@ -31,8 +31,8 @@ outs: path: metrics.json locked: True -#Comments are persisted between multiple executions of dvc repro/commit. -#Comments are not persisted between multiple executions of dvc run/add/import. +# Comments are persisted between multiple executions of dvc repro/commit. +# Comments are not persisted between multiple executions of dvc run/add/import. meta: # key to contain arbitary user data name: John email: john@xyz.com @@ -71,10 +71,9 @@ A meta entry consists of `key:value` pairs such as `name: john`. A meta entry can have any structure and contain any number of attributes. `"meta: string"` is also possible, it doesn't necessarily need to contain dictionary always. -Comments can be added to the .dvc file using `# comment` syntax. - -Comments and meta values are preserved between multiple executions of `dvc repro` - and `dvc commit` commands. +Comments can be added to the .dvc file using `# comment` syntax. Comments and +meta values are preserved between multiple executions of `dvc repro` and +`dvc commit` commands. If user overwrites the file, comments and meta values are not preserved between multiple executions of `dvc run`,`dvc add`,`dvc import` commands.