Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion public/static/docs/command-reference/add.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ usage: dvc add [-h] [-q | -v] [-R] [--no-commit] [-f FILE]
targets [targets ...]

positional arguments:
targets Input files/directories to add.
targets Input files/directories to add.
```

## Description
Expand Down
4 changes: 2 additions & 2 deletions public/static/docs/command-reference/checkout.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ usage: dvc checkout [-h] [-q | -v] [-d] [-R] [-f] [--relink]
[targets [targets ...]]

positional arguments:
targets DVC-files to checkout. Optional. (Finds all
DVC-files in the workspace by default.)
targets DVC-files to checkout. Optional. (Finds all
DVC-files in the workspace by default.)
```

## Description
Expand Down
4 changes: 2 additions & 2 deletions public/static/docs/command-reference/commit.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ usage: dvc commit [-h] [-q | -v] [-f] [-d] [-R]
[targets [targets ...]]

positional arguments:
targets DVC-files to commit. Optional. (Finds all
DVC-files in the workspace by default.)
targets DVC-files to commit. Optional. (Finds all DVC-files
in the workspace by default.)
```

## Description
Expand Down
4 changes: 2 additions & 2 deletions public/static/docs/command-reference/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ usage: dvc config [-h] [--global] [--system] [--local] [-q | -v] [-u]
name [value]

positional arguments:
name Option name.
value Option value.
name Option name.
value Option value.
```

## Description
Expand Down
7 changes: 3 additions & 4 deletions public/static/docs/command-reference/diff.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,9 @@ narrowed down to specific target files and directories under DVC control.
usage: dvc diff [-h] [-q | -v] [-t TARGET] a_ref [b_ref]

positional arguments:
a_ref Git reference from which diff calculates
b_ref Git reference until which diff calculates, if
omitted diff shows the difference between
current HEAD and a_ref
a_ref Git reference from which diff calculates
b_ref Git reference until which diff calculates, if omitted diff
shows the difference between current HEAD and a_ref
```

## Description
Expand Down
4 changes: 2 additions & 2 deletions public/static/docs/command-reference/fetch.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ usage: dvc fetch [-h] [-q | -v] [-j JOBS] [--show-checksums]
[targets [targets ...]]

positional arguments:
targets Limit command scope to these DVC-files. Using -R,
directories to search DVC-files in can also be given.
targets Limit command scope to these DVC-files. Using -R,
directories to search DVC-files in can also be given.
```

## Description
Expand Down
16 changes: 7 additions & 9 deletions public/static/docs/command-reference/get-url.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ Download a file or directory from a supported URL (for example `s3://`,
usage: dvc get-url [-h] [-q | -v] url [out]

positional arguments:
url (See supported URLs in the description.)
out Destination path to put data in.
url (See supported URLs in the description.)
out Destination path to put data to.
```

## Description
Expand Down Expand Up @@ -42,11 +42,6 @@ DVC supports several types of (local or) remote locations (protocols):
| `hdfs` | HDFS to file\* | `hdfs://user@example.com/path/to/data.csv` |
| `http` | HTTP to file\* | `https://example.com/path/to/data.csv` |

> \* HDFS and HTTP **do not** support downloading entire directories, only
> single files.

<!-- Separate MD quote: -->

> If you installed DVC via `pip` and plan to use cloud services as remote
> storage, you might need to install these optional dependencies: `[s3]`,
> `[azure]`, `[gdrive]`, `[gs]`, `[oss]`, `[ssh]`. Alternatively, use `[all]` to
Expand All @@ -55,8 +50,8 @@ DVC supports several types of (local or) remote locations (protocols):

<!-- Separate MD quote: -->

> See `dvc get` to download data/model files or directories from other <abbr>DVC
> repositories</abbr> (e.g. GitHub URLs).
\* HDFS and HTTP **do not** support downloading entire directories, only single
files.

Another way to understand the `dvc get-url` command is as a tool for downloading
data files. On GNU/Linux systems for example, instead of `dvc get-url` with
Expand All @@ -66,6 +61,9 @@ HTTP(S) it's possible to instead use:
$ wget https://example.com/path/to/data.csv
```

> See `dvc get` to download data/model files or directories from other <abbr>DVC
> repositories</abbr> (e.g. GitHub URLs).

## Options

- `-h`, `--help` - prints the usage/help message, and exit.
Expand Down
6 changes: 3 additions & 3 deletions public/static/docs/command-reference/get.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ repository into the current working directory.
usage: dvc get [-h] [-q | -v] [-o [OUT]] [--rev [REV]] url path

positional arguments:
url Location of DVC or Git repository to download from.
path Path to a file or directory within the repository.
url Location of DVC or Git repository to download from.
path Path to a file or directory within the repository.
```

## Description
Expand All @@ -37,7 +37,7 @@ The `path` argument of this command is used to specify the location of the
target to be downloaded within the source repository at `url`. It can point to
any file or directory in there, including <abbr>outputs</abbr> tracked by DVC,
as well as files tracked by Git. Note that for DVC repos, the target should be
found in one of the [DVC-files](/doc/user-guide/dvc-file-format) pf the project.
found in one of the [DVC-files](/doc/user-guide/dvc-file-format) of the project.
The project should also have a default
[DVC remote](/doc/command-reference/remote), containing the actual data.

Expand Down
56 changes: 25 additions & 31 deletions public/static/docs/command-reference/import-url.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ changes in the remote data source. Creates a DVC-file.
usage: dvc import-url [-h] [-q | -v] [-f FILE] url [out]

positional arguments:
url (See supported URLs in the description.)
out Destination path to put data in.
url (See supported URLs in the description.)
out Destination path to put files to.
```

## Description
Expand Down Expand Up @@ -44,45 +44,36 @@ determine whether the local copy is out of date.

DVC supports several types of (local or) remote locations (protocols):

| Type | Description | `url` format |
| -------- | ----------------------------------------------------- | ------------------------------------------ |
| `local` | Local path | `/path/to/local/data` |
| `s3` | Amazon S3 | `s3://mybucket/data` |
| `gs` | Google Storage | `gs://mybucket/data` |
| `ssh` | SSH server | `ssh://user@example.com:/path/to/data` |
| `hdfs` | HDFS to file\* | `hdfs://user@example.com/path/to/data.csv` |
| `http` | HTTP to file with _strong ETag_ (explanation below)\* | `https://example.com/path/to/data.csv` |
| `remote` | Remote path (see explanation below) | `remote://myremote/path/to/data` |

> \* HDFS and HTTP **do not** support downloading entire directories, only
> single files.

<!-- Separate MD quote: -->
| Type | Description | `url` format |
| -------- | --------------------------------------------------- | ------------------------------------------ |
| `local` | Local path | `/path/to/local/data` |
| `s3` | Amazon S3 | `s3://mybucket/data` |
| `gs` | Google Storage | `gs://mybucket/data` |
| `ssh` | SSH server | `ssh://user@example.com:/path/to/data` |
| `hdfs` | HDFS to file (explanation below) | `hdfs://user@example.com/path/to/data.csv` |
| `http` | HTTP to file with _strong ETag_ (explanation below) | `https://example.com/path/to/data.csv` |
| `remote` | Remote path (see explanation below) | `remote://myremote/path/to/data` |

> If you installed DVC via `pip` and plan to use cloud services as remote
> storage, you might need to install these optional dependencies: `[s3]`,
> `[azure]`, `[gdrive]`, `[gs]`, `[oss]`, `[ssh]`. Alternatively, use `[all]` to
> include them all. The command should look like this: `pip install "dvc[s3]"`.
> (This example installs `boto3` library along with DVC to support S3 storage.)

<!-- Separate MD quote: -->

> In case of HTTP,
> [strong ETag](https://en.wikipedia.org/wiki/HTTP_ETag#Strong_and_weak_validation)
> is necessary to track if the specified remote file (URL) changed to download
> it again.
Specific explanations:

<!-- Separate MD quote: -->
- HDFS and HTTP **do not** support downloading entire directories, only single
files.

> `remote://myremote/path/to/file` notation just means that a DVC
> [remote](/doc/command-reference/remote) `myremote` is defined and when DVC is
> running. DVC automatically expands this URL into a regular S3, SSH, GS, etc
> URL by appending `/path/to/file` to the `myremote`'s configured base path.
- In case of HTTP,
[strong ETag](https://en.wikipedia.org/wiki/HTTP_ETag#Strong_and_weak_validation)
is necessary to track if the specified remote file (URL) changed to download
it again.

<!-- Separate MD quote: -->

> See `dvc import` to download and tack data/model files or directories from
> other <abbr>DVC repositories</abbr> (e.g. GitHub URLs).
- `remote://myremote/path/to/file` notation just means that a DVC
[remote](/doc/command-reference/remote) `myremote` is defined and when DVC is
running. DVC automatically expands this URL into a regular S3, SSH, GS, etc
URL by appending `/path/to/file` to the `myremote`'s configured base path.

Another way to understand the `dvc import-url` command is as a short-cut for a
more verbose `dvc run` command. This is discussed in the
Expand Down Expand Up @@ -112,6 +103,9 @@ Note that import stages are considered always locked, meaning that if you run
`dvc repro`, they won't be updated. Use `dvc update` on them to bring the import
up to date from the external data source.

> See `dvc import` to download and tack data/model files or directories from
> other <abbr>DVC repositories</abbr> (e.g. GitHub URLs).

## Options

- `-f`, `--file` - specify name of the DVC-file it generates. By default the
Expand Down
4 changes: 2 additions & 2 deletions public/static/docs/command-reference/import.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ import.
usage: dvc import [-h] [-q | -v] [-o [OUT]] [--rev [REV]] url path

positional arguments:
url Location of DVC or Git repository to download from.
path Path to a file or directory within the repository.
url Location of DVC or Git repository to download from.
path Path to a file or directory within the repository.
```

## Description
Expand Down
4 changes: 1 addition & 3 deletions public/static/docs/glossary.js
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,7 @@ Initialized by running \`dvc init\` in the **workspace** (typically in a Git
repository). It will contain the
[\`.dvc/\` directory](/doc/user-guide/dvc-files-and-directories) and
[DVC-files](/doc/user-guide/dvc-file-format) created with commands such as
\`dvc add\` or \`dvc run\`. Some DVC commands require it to be in a Git repo –
explained in each [command reference](/doc/command-reference) but usually
obvious from the context.
\`dvc add\` or \`dvc run\`.
`
},
{
Expand Down