From 702172e85dd23d6f1190726e3ac40aa06e4f48e0 Mon Sep 17 00:00:00 2001 From: karajan1001 Date: Sun, 26 Jul 2020 17:44:15 +0800 Subject: [PATCH 01/20] Docs for new command `dvc check-ignore` fix#1628 --- .../docs/command-reference/check-ignore.md | 82 +++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 content/docs/command-reference/check-ignore.md diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md new file mode 100644 index 0000000000..216d6efce1 --- /dev/null +++ b/content/docs/command-reference/check-ignore.md @@ -0,0 +1,82 @@ +# check-ignore + +Used for debugging `.dvcignore` / exclude files. + +## Synopsis + +```usage +usage: usage: dvc check-ignore [-h] [-q | -v] [-d] [-n] targets [targets ...] + +positional arguments: + targets Input files/directories to check ignore patterns. +``` + +## Description + +For each pathname given via the command-line , check whether the file is excluded +by `.dvcignore` output the path if it is excluded. + +## Options + +- `-h`, `--help` - prints the usage/help message, and exit. + +- `-q`, `--quiet` - do not write anything to standard output. Exit with 0 if no + problems arise, otherwise 1. + +- `-v`, `--verbose` - displays detailed tracing information. + +- `-d`, `--details` - Show the exclude pattern together with the path. + +- `-n`, `--non-matching` - Show given paths which don’t match any pattern. Only + used when --details is enabled. + +## Examples + +First, let's create a `.dvcignore` file with some patterns in it, and some files +to check. + +```dvc +$ echo "file*\n\!file2" >> .dvcignore +$ cat .dvcignore +file* +!file2 +$ touch file1 file2 other +$ ls +file1 file2 other +``` + +Then, let's check if they were excluded by `.dvcignore` + +```dvc +$ dvc check-ignore file1 +file1 +$ dvc check-ignore file2 +file2 +$ dvc check-ignore other +$ dvc check-ignore file* +file1 +file2 +``` + +If `--details` is specified, the output is a series of lines of the form: +` ` + +```dvc +$ dvc check-ignore -d file1 +.dvcignore:1:file* file1 +$ dvc check-ignore -d file2 +.dvcignore:2:!file2 file2 +$ dvc check-ignore -d other +$ dvc check-ignore -d file* +.dvcignore:1:file* file1 +.dvcignore:2:!file2 file2 +``` + +If `--non-matching` is specified, non-matching pathnames will also be +output, in which case all fields in each output record except for + will be empty + +```dvc +$ dvc check-ignore -d -n other +:: other +``` From 214131d1e88cbb087f926f88526d201349530d6a Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Tue, 28 Jul 2020 22:32:50 -0500 Subject: [PATCH 02/20] Update content/docs/command-reference/check-ignore.md --- content/docs/command-reference/check-ignore.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md index 216d6efce1..f11ce8a744 100644 --- a/content/docs/command-reference/check-ignore.md +++ b/content/docs/command-reference/check-ignore.md @@ -5,7 +5,8 @@ Used for debugging `.dvcignore` / exclude files. ## Synopsis ```usage -usage: usage: dvc check-ignore [-h] [-q | -v] [-d] [-n] targets [targets ...] +usage: usage: dvc check-ignore [-h] [-q | -v] [-d] [-n] + targets [targets ...] positional arguments: targets Input files/directories to check ignore patterns. From 20cbcf5b50667984f7bd360bac4a89ac65d33598 Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Tue, 28 Jul 2020 22:38:10 -0500 Subject: [PATCH 03/20] Update content/docs/command-reference/check-ignore.md --- content/docs/command-reference/check-ignore.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md index f11ce8a744..25285bed4b 100644 --- a/content/docs/command-reference/check-ignore.md +++ b/content/docs/command-reference/check-ignore.md @@ -26,7 +26,7 @@ by `.dvcignore` output the path if it is excluded. - `-v`, `--verbose` - displays detailed tracing information. -- `-d`, `--details` - Show the exclude pattern together with the path. +- `-d`, `--details` - show the exclude pattern together with the path. - `-n`, `--non-matching` - Show given paths which don’t match any pattern. Only used when --details is enabled. From ead5f4f3178ea0a31f3cf8b7a513512f20720aeb Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Tue, 28 Jul 2020 22:38:59 -0500 Subject: [PATCH 04/20] Update content/docs/command-reference/check-ignore.md --- content/docs/command-reference/check-ignore.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md index 25285bed4b..48fa5d67b9 100644 --- a/content/docs/command-reference/check-ignore.md +++ b/content/docs/command-reference/check-ignore.md @@ -26,7 +26,7 @@ by `.dvcignore` output the path if it is excluded. - `-v`, `--verbose` - displays detailed tracing information. -- `-d`, `--details` - show the exclude pattern together with the path. +- `-d`, `--details` - show the exclude pattern together with each target path. - `-n`, `--non-matching` - Show given paths which don’t match any pattern. Only used when --details is enabled. From 3b5e1a06e725abdea1e1949494a150b453f3c8cb Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Tue, 28 Jul 2020 22:40:08 -0500 Subject: [PATCH 05/20] Update content/docs/command-reference/check-ignore.md --- content/docs/command-reference/check-ignore.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md index 48fa5d67b9..8cd354eb01 100644 --- a/content/docs/command-reference/check-ignore.md +++ b/content/docs/command-reference/check-ignore.md @@ -28,8 +28,8 @@ by `.dvcignore` output the path if it is excluded. - `-d`, `--details` - show the exclude pattern together with each target path. -- `-n`, `--non-matching` - Show given paths which don’t match any pattern. Only - used when --details is enabled. +- `-n`, `--non-matching` - show the target paths which don’t match any pattern. + Only usable when `--details` is also employed ## Examples From 2fe85d3ee2b61d5ffd04844bb6f4faedf9119c81 Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Tue, 28 Jul 2020 22:40:48 -0500 Subject: [PATCH 06/20] Update content/docs/command-reference/check-ignore.md --- content/docs/command-reference/check-ignore.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md index 8cd354eb01..911399f009 100644 --- a/content/docs/command-reference/check-ignore.md +++ b/content/docs/command-reference/check-ignore.md @@ -34,7 +34,7 @@ by `.dvcignore` output the path if it is excluded. ## Examples First, let's create a `.dvcignore` file with some patterns in it, and some files -to check. +to check against it. ```dvc $ echo "file*\n\!file2" >> .dvcignore From 98add0ade050a6865fab37ed1d7088f327af2382 Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Tue, 28 Jul 2020 22:42:09 -0500 Subject: [PATCH 07/20] Update content/docs/command-reference/check-ignore.md --- content/docs/command-reference/check-ignore.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md index 911399f009..fee3be4060 100644 --- a/content/docs/command-reference/check-ignore.md +++ b/content/docs/command-reference/check-ignore.md @@ -1,6 +1,7 @@ # check-ignore -Used for debugging `.dvcignore` / exclude files. +Check whether any given files or directories are excluded from DVC +due to the patterns found in [`.dvcignore`](/doc/user-guide/dvcignore). ## Synopsis From feaa074912a38356c7e93ddd436f556d26ea458e Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Tue, 28 Jul 2020 22:43:49 -0500 Subject: [PATCH 08/20] Update content/docs/command-reference/check-ignore.md --- content/docs/command-reference/check-ignore.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md index fee3be4060..cd9d39ac45 100644 --- a/content/docs/command-reference/check-ignore.md +++ b/content/docs/command-reference/check-ignore.md @@ -47,7 +47,7 @@ $ ls file1 file2 other ``` -Then, let's check if they were excluded by `.dvcignore` +Then, let's check if these files would be excluded given our `.dvcignore` file: ```dvc $ dvc check-ignore file1 From 545b2630bdd1fcd850ab5814ee2e5d5fa97fd34a Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Tue, 28 Jul 2020 22:52:35 -0500 Subject: [PATCH 09/20] Update content/docs/command-reference/check-ignore.md --- content/docs/command-reference/check-ignore.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md index cd9d39ac45..88cedd757d 100644 --- a/content/docs/command-reference/check-ignore.md +++ b/content/docs/command-reference/check-ignore.md @@ -60,7 +60,7 @@ file1 file2 ``` -If `--details` is specified, the output is a series of lines of the form: +If the `--details` option is used, a series of lines are printed using this format: ` ` ```dvc From 34fdb5f25ccd336422653b7c3515dfffdd2083f7 Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Tue, 28 Jul 2020 22:59:10 -0500 Subject: [PATCH 10/20] Update content/docs/command-reference/check-ignore.md --- content/docs/command-reference/check-ignore.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md index 88cedd757d..a574479ca1 100644 --- a/content/docs/command-reference/check-ignore.md +++ b/content/docs/command-reference/check-ignore.md @@ -74,9 +74,9 @@ $ dvc check-ignore -d file* .dvcignore:2:!file2 file2 ``` -If `--non-matching` is specified, non-matching pathnames will also be -output, in which case all fields in each output record except for - will be empty +With the `--non-matching` option, non-matching `targets` will also be +included in the list. All fields in each line, except for ``, will +be empty. ```dvc $ dvc check-ignore -d -n other From 44277dc5d0965861564bffccc6eef179c4904316 Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Wed, 5 Aug 2020 23:23:43 -0500 Subject: [PATCH 11/20] Update content/docs/command-reference/check-ignore.md --- content/docs/command-reference/check-ignore.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md index a574479ca1..f2c27d4404 100644 --- a/content/docs/command-reference/check-ignore.md +++ b/content/docs/command-reference/check-ignore.md @@ -10,7 +10,7 @@ usage: usage: dvc check-ignore [-h] [-q | -v] [-d] [-n] targets [targets ...] positional arguments: - targets Input files/directories to check ignore patterns. + targets File or directory paths, or ignore patterns to check ``` ## Description From 12884071f3d40f852a1f366a5788bf650cdeb60f Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Wed, 5 Aug 2020 23:25:33 -0500 Subject: [PATCH 12/20] Update content/docs/command-reference/check-ignore.md --- content/docs/command-reference/check-ignore.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md index f2c27d4404..11ed398067 100644 --- a/content/docs/command-reference/check-ignore.md +++ b/content/docs/command-reference/check-ignore.md @@ -15,8 +15,9 @@ positional arguments: ## Description -For each pathname given via the command-line , check whether the file is excluded -by `.dvcignore` output the path if it is excluded. +This helper command checks whether the given `targets` are ignored by DVC +according to the [`.dvcignore` file](/doc/user-guide/dvcignore) (if any). The +ones that are indeed ignores are printed back. ## Options From d99e05c363f6f37d35a93e4d42551eeaf2637e1d Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Wed, 5 Aug 2020 23:28:13 -0500 Subject: [PATCH 13/20] Update content/docs/command-reference/check-ignore.md --- content/docs/command-reference/check-ignore.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md index 11ed398067..d49a6e22d1 100644 --- a/content/docs/command-reference/check-ignore.md +++ b/content/docs/command-reference/check-ignore.md @@ -21,6 +21,11 @@ ones that are indeed ignores are printed back. ## Options +- `-d`, `--details` - show the exclude pattern together with each target path. + +- `-n`, `--non-matching` - show the target paths which don’t match any pattern. + Only usable when `--details` is also employed + - `-h`, `--help` - prints the usage/help message, and exit. - `-q`, `--quiet` - do not write anything to standard output. Exit with 0 if no @@ -28,11 +33,6 @@ ones that are indeed ignores are printed back. - `-v`, `--verbose` - displays detailed tracing information. -- `-d`, `--details` - show the exclude pattern together with each target path. - -- `-n`, `--non-matching` - show the target paths which don’t match any pattern. - Only usable when `--details` is also employed - ## Examples First, let's create a `.dvcignore` file with some patterns in it, and some files From 3ae42a8281d398f8145d509261cf6c874fb9099a Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Wed, 5 Aug 2020 23:29:05 -0500 Subject: [PATCH 14/20] Update content/docs/command-reference/check-ignore.md --- content/docs/command-reference/check-ignore.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md index d49a6e22d1..05330fe0af 100644 --- a/content/docs/command-reference/check-ignore.md +++ b/content/docs/command-reference/check-ignore.md @@ -53,7 +53,8 @@ Then, let's check if these files would be excluded given our `.dvcignore` file: ```dvc $ dvc check-ignore file1 file1 -$ dvc check-ignore file2 +$ dvc check-ignore file1 file2 +file1 file2 $ dvc check-ignore other $ dvc check-ignore file* From c61c42a303f8ed8565dda25dcb651d9b7915fd18 Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Wed, 5 Aug 2020 23:30:02 -0500 Subject: [PATCH 15/20] Update content/docs/command-reference/check-ignore.md --- content/docs/command-reference/check-ignore.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md index 05330fe0af..dca589bb1e 100644 --- a/content/docs/command-reference/check-ignore.md +++ b/content/docs/command-reference/check-ignore.md @@ -57,6 +57,7 @@ $ dvc check-ignore file1 file2 file1 file2 $ dvc check-ignore other + # There's no command output, meaning `other` is not excluded. $ dvc check-ignore file* file1 file2 From f10806bcb2d869ed8188c04d4bf328bbcd4fc80c Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Wed, 5 Aug 2020 23:30:49 -0500 Subject: [PATCH 16/20] Update content/docs/command-reference/check-ignore.md --- content/docs/command-reference/check-ignore.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md index dca589bb1e..f1aa28573e 100644 --- a/content/docs/command-reference/check-ignore.md +++ b/content/docs/command-reference/check-ignore.md @@ -67,9 +67,8 @@ If the `--details` option is used, a series of lines are printed using this form ` ` ```dvc -$ dvc check-ignore -d file1 +$ dvc check-ignore -d file1 file2 .dvcignore:1:file* file1 -$ dvc check-ignore -d file2 .dvcignore:2:!file2 file2 $ dvc check-ignore -d other $ dvc check-ignore -d file* From 6d3930dc6cdc68bc47d9aa40485f2c4bb3e0b6a1 Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Thu, 6 Aug 2020 21:15:50 -0500 Subject: [PATCH 17/20] Update content/docs/command-reference/check-ignore.md --- content/docs/command-reference/check-ignore.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md index f1aa28573e..bcf74a12dd 100644 --- a/content/docs/command-reference/check-ignore.md +++ b/content/docs/command-reference/check-ignore.md @@ -64,7 +64,7 @@ file2 ``` If the `--details` option is used, a series of lines are printed using this format: -` ` +`:: | ` ```dvc $ dvc check-ignore -d file1 file2 From 1c32b8d4caaeb79cb4d65441208f300498467643 Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Thu, 6 Aug 2020 22:01:16 -0500 Subject: [PATCH 18/20] cmd: complete check-ignore ref. per https://github.com/iterative/dvc.org/pull/1629#issuecomment-668820551 and https://github.com/iterative/dvc.org/pull/1629#pullrequestreview-457171636 --- config/prismjs/dvc-commands.js | 1 + .../docs/command-reference/check-ignore.md | 18 ++-- content/docs/sidebar.json | 4 + content/docs/user-guide/contributing/docs.md | 2 +- content/docs/user-guide/dvcignore.md | 87 ++++++++++--------- 5 files changed, 62 insertions(+), 50 deletions(-) diff --git a/config/prismjs/dvc-commands.js b/config/prismjs/dvc-commands.js index 937f9da24e..4e877960f2 100644 --- a/config/prismjs/dvc-commands.js +++ b/config/prismjs/dvc-commands.js @@ -48,6 +48,7 @@ module.exports = [ 'config', 'commit', 'checkout', + 'check-ignore', 'cache dir', 'cache', 'add' diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md index bcf74a12dd..434804f08c 100644 --- a/content/docs/command-reference/check-ignore.md +++ b/content/docs/command-reference/check-ignore.md @@ -1,7 +1,7 @@ # check-ignore -Check whether any given files or directories are excluded from DVC -due to the patterns found in [`.dvcignore`](/doc/user-guide/dvcignore). +Check whether any given files or directories are excluded from DVC due to the +patterns found in [`.dvcignore`](/doc/user-guide/dvcignore). ## Synopsis @@ -48,7 +48,8 @@ $ ls file1 file2 other ``` -Then, let's check if these files would be excluded given our `.dvcignore` file: +Then, let's use `dvc check-ignore` to see which of these files would be excluded +given our `.dvcignore` file: ```dvc $ dvc check-ignore file1 @@ -60,11 +61,11 @@ $ dvc check-ignore other # There's no command output, meaning `other` is not excluded. $ dvc check-ignore file* file1 -file2 +file2 ``` -If the `--details` option is used, a series of lines are printed using this format: -`:: | ` +If the `--details` option is used, a series of lines are printed using this +format: `:: | ` ```dvc $ dvc check-ignore -d file1 file2 @@ -76,9 +77,8 @@ $ dvc check-ignore -d file* .dvcignore:2:!file2 file2 ``` -With the `--non-matching` option, non-matching `targets` will also be -included in the list. All fields in each line, except for ``, will -be empty. +With the `--non-matching` option, non-matching `targets` will also be included +in the list. All fields in each line, except for ``, will be empty. ```dvc $ dvc check-ignore -d -n other diff --git a/content/docs/sidebar.json b/content/docs/sidebar.json index 7535efb2b0..9cf5e1b783 100644 --- a/content/docs/sidebar.json +++ b/content/docs/sidebar.json @@ -163,6 +163,10 @@ } ] }, + { + "label": "check-ignore", + "slug": "check-ignore" + }, { "label": "checkout", "slug": "checkout" diff --git a/content/docs/user-guide/contributing/docs.md b/content/docs/user-guide/contributing/docs.md index 34cae8b53d..1b08f3c062 100644 --- a/content/docs/user-guide/contributing/docs.md +++ b/content/docs/user-guide/contributing/docs.md @@ -200,7 +200,7 @@ We also use "emoji" symbols sparingly for visibility on certain notes. Mainly: - 📖 For notes that link to other related documentation - ⚠️ Warnings about possible problems related to DVC usage (similar to **Note!** and "Note that..." notes) -- 💡 Useful tips related to external tools/integrations +- 💡 Useful tips related to related or external tools and integrations > Some other emojis currently in use here and there: ⚡✅🙏🐛⭐❗ (among > others). diff --git a/content/docs/user-guide/dvcignore.md b/content/docs/user-guide/dvcignore.md index 7305abe11b..40be689563 100644 --- a/content/docs/user-guide/dvcignore.md +++ b/content/docs/user-guide/dvcignore.md @@ -13,11 +13,10 @@ similar to `.gitignore` in Git. ## How does it work? -- You need to create the `.dvcignore` file. It can be placed in the root of the - project or inside any subdirectory (see also [remarks](#Remarks) below). -- Populate it with [patterns](https://git-scm.com/docs/gitignore) that you would - like to ignore. You can find useful templates - [here](https://github.com/github/gitignore). +- You need to create a `.dvcignore` file. These can be placed in the root of the + project, or in any subdirectory (see the [remarks](#Remarks) below). +- Populate it with [.gitignore patterns](https://git-scm.com/docs/gitignore). + You can find useful templates [here](https://github.com/github/gitignore). - Each line should contain only one pattern. - During execution of commands that traverse directories, DVC will ignore matching paths. @@ -28,87 +27,95 @@ Ignored files will not be saved in cache, they will be non-existent for DVC. It's worth to remember that, especially when ignoring files inside DVC-handled directories. -**It is crucial to understand, that DVC might remove ignored files upon -`dvc run` or `dvc repro`. If they are not produced by a -[pipeline](/doc/command-reference/dag) [stage](/doc/command-reference/run), they -can be deleted permanently.** +⚠️ Important! Note that `dvc run` and `dvc repro` might remove ignored files. If +they are not produced by a pipeline [stage](/doc/command-reference/run), they +can be lost permanently. -Keep in mind, that when you add to `.dvcignore` entries that affect one of the -existing outputs, its status will change and DVC will behave as if -that affected files were deleted. +Keep in mind, that when you add `.dvcignore` patterns that affect an existing +output, its status will change and DVC will behave as if that +affected files were deleted. + +💡 Note that you can use the `dvc check-ignore` command to check whether given +files or directories are ignored by the patterns in a `.dvcignore` file. If DVC finds a `.dvcignore` file inside a dependency or output directory, it raises an error. Ignoring files inside such directories should be handled from a `.dvcignore` in higher levels of the project tree. -## Syntax - -The same as for [`.gitignore`](https://git-scm.com/docs/gitignore). - ## Examples -Let's see what happens when we add a file to `.dvcignore`. +Let's see what happens when we add a file to `.dvcignore`: ```dvc $ mkdir data -$ echo data1 >> data/data1 -$ echo data2 >> data/data2 -$ tree . - +$ echo 1 > data/data1 +$ echo 2 > data/data2 +$ tree . └── data ├── data1 └── data2 ``` -We created the `data/` directory with two files. Let's ignore one of them, and -track the directory with DVC. +We created the `data/` directory with two data files. Let's ignore one of them, +and double check that it's being ignored by DVC: ```dvc $ echo data/data1 >> .dvcignore $ cat .dvcignore - data/data1 +$ dvc check-ignore data/* +data/data1 +``` -$ dvc add data +> Refer to `dvc check-ignore` for more details on that command. -$ tree .dvc/cache +## Example: Skip specific files when adding directories + +Let's now track the directory with `dvc add`, and see what happens in the +cache: +```dvc +$ dvc add data +... +$ tree .dvc/cache .dvc/cache -├── 54 -│   └── 40cb5e4c57ab54af68127492334a23.dir -└── ed - └── c3d3797971f12c7f5e1d106dd5cee2 +├── 26 +│   └── ab0db90d72e28ad0ba1e22ee510510 +└── ad + └── 8b0ddcf133a6e5833002ce28f97c5a.dir +$ md5 data/* +b026324c6904b2a9cb4b88d6d61c81d1 data/data1 +26ab0db90d72e28ad0ba1e22ee510510 data/data2 ``` -Only the hash values of a directory (`data/`) and one file have been -cached. This means that `dvc add` ignored one of the files -(`data1`). +Only the cache entries of the `data/` directory itself and one file have been +stored. Checking the hash value of the data files manually, we can see that +`data2` was cached. This means that `dvc add` did ignore `data1`. > Refer to > [Structure of cache directory](/doc/user-guide/dvc-files-and-directories#structure-of-the-cache-directory) > for more info. +## Example: Ignore file state changes + Now, let's modify file `data1` and see if it affects `dvc status`. ```dvc $ dvc status - Data and pipelines are up to date. -$ echo "123" >> data/data1 +$ echo "2345" >> data/data1 $ dvc status - Data and pipelines are up to date. ``` -`dvc status` also ignores `data1`. The same modification on a tracked file will -produce a different output: +`dvc status` ignores `data1`. Modifications on a tracked file produce a +different output: ```dvc -$ echo "123" >> data/data2 +$ echo "345" >> data/data2 $ dvc status - data.dvc: changed outs: modified: data From 524b01eedfbddca3a95f929b8eec3fb5562d0039 Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Thu, 6 Aug 2020 22:07:51 -0500 Subject: [PATCH 19/20] guide: anoter link from dvcignore guide to check-ignore cmd ref. --- content/docs/user-guide/dvcignore.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/content/docs/user-guide/dvcignore.md b/content/docs/user-guide/dvcignore.md index 40be689563..a9572b271e 100644 --- a/content/docs/user-guide/dvcignore.md +++ b/content/docs/user-guide/dvcignore.md @@ -8,8 +8,10 @@ project. For example, when working in a workspace directory with a large number of data files, you might encounter extended execution time for operations as simple as `dvc status`. In other case you might want to omit files or folders unrelated to the project (like `.DS_Store` on MacOS). To address -these scenarios, DVC supports optional `.dvcignore` files. `.dvcignore` works -similar to `.gitignore` in Git. +these scenarios, DVC supports optional `.dvcignore` files. + +`.dvcignore` is similar to `.gitignore` in Git, and can be tested with our +helper command `dvc check-ignore`. ## How does it work? From c759585568a8fa6e900bf912c60728a9e639bb3e Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Thu, 6 Aug 2020 22:12:51 -0500 Subject: [PATCH 20/20] cmd: update check-ignore targets arg desc per https://github.com/iterative/dvc/pull/4282#pullrequestreview-460270636 --- content/docs/command-reference/check-ignore.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/content/docs/command-reference/check-ignore.md b/content/docs/command-reference/check-ignore.md index 434804f08c..4a99f6d591 100644 --- a/content/docs/command-reference/check-ignore.md +++ b/content/docs/command-reference/check-ignore.md @@ -10,14 +10,18 @@ usage: usage: dvc check-ignore [-h] [-q | -v] [-d] [-n] targets [targets ...] positional arguments: - targets File or directory paths, or ignore patterns to check + targets File or directory paths to check (wildcards supported) ``` ## Description This helper command checks whether the given `targets` are ignored by DVC according to the [`.dvcignore` file](/doc/user-guide/dvcignore) (if any). The -ones that are indeed ignores are printed back. +ones that are ignored indeed are printed back. + +> Note that your shell may support path wildcards such as `dir/file*` and these +> can be fed as `targets` to `dvc check-ignore`, as shown in the +> [examples](#examples). ## Options