Skip to content

run/repro/stage add: regroup options/flags #7524

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
karajan1001 merged 1 commit into from
May 5, 2022
Merged

run/repro/stage add: regroup options/flags #7524

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 31 additions & 31 deletions dvc/commands/repro.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,33 +67,19 @@ def add_arguments(repro_parser):
help="Reproduce even if dependencies were not changed.",
)
repro_parser.add_argument(
"-s",
"--single-item",
action="store_true",
default=False,
help="Reproduce only single data item without recursive dependencies "
"check.",
)
repro_parser.add_argument(
"-m",
"--metrics",
action="store_true",
default=False,
help="Show metrics after reproduction.",
)
repro_parser.add_argument(
"--dry",
"-i",
"--interactive",
action="store_true",
default=False,
help="Only print the commands that would be executed without "
"actually executing.",
help="Ask for confirmation before reproducing each stage.",
)
repro_parser.add_argument(
"-i",
"--interactive",
"-s",
"--single-item",
action="store_true",
default=False,
help="Ask for confirmation before reproducing each stage.",
help="Reproduce only single data item without recursive dependencies "
"check.",
)
repro_parser.add_argument(
"-p",
Expand All @@ -118,18 +104,25 @@ def add_arguments(repro_parser):
help="Reproduce all stages in the specified directory.",
)
repro_parser.add_argument(
"--force-downstream",
"-m",
"--metrics",
action="store_true",
default=False,
help="Reproduce all descendants of a changed stage even if their "
"direct dependencies didn't change.",
help="Show metrics after reproduction.",
)
repro_parser.add_argument(
"--downstream",
action="store_true",
default=False,
help="Start from the specified stages when reproducing pipelines.",
)
repro_parser.add_argument(
"--force-downstream",
action="store_true",
default=False,
help="Reproduce all descendants of a changed stage even if their "
"direct dependencies didn't change.",
)
repro_parser.add_argument(
"--pull",
action="store_true",
Expand All @@ -139,6 +132,13 @@ def add_arguments(repro_parser):
"from the run-cache."
),
)
repro_parser.add_argument(
"--dry",
action="store_true",
default=False,
help="Only print the commands that would be executed without "
"actually executing.",
)


def add_parser(subparsers, parent_parser):
Expand All @@ -156,13 +156,10 @@ def add_parser(subparsers, parent_parser):
add_arguments(repro_parser)
# repro only args
repro_parser.add_argument(
Comment thread
dberenbaum marked this conversation as resolved.
"--no-run-cache",
"--glob",
action="store_true",
default=False,
help=(
"Execute stage commands even if they have already been run with "
"the same command/dependencies/outputs/etc before."
),
help="Allows targets containing shell-style wildcards.",
)
repro_parser.add_argument(
"--no-commit",
Expand All @@ -171,9 +168,12 @@ def add_parser(subparsers, parent_parser):
help="Don't put files/directories into cache.",
)
repro_parser.add_argument(
"--glob",
"--no-run-cache",
action="store_true",
default=False,
help="Allows targets containing shell-style wildcards.",
help=(
"Execute stage commands even if they have already been run with "
"the same command/dependencies/outputs/etc before."
),
)
repro_parser.set_defaults(func=CmdRepro)
20 changes: 10 additions & 10 deletions dvc/commands/run.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,19 +70,17 @@ def add_parser(subparsers, parent_parser):
"--file", metavar="<filename>", help=argparse.SUPPRESS
)
run_parser.add_argument(
"--no-exec",
"--single-stage",
action="store_true",
default=False,
help="Only create dvc.yaml without actually running it.",
help=argparse.SUPPRESS,
)
_add_common_args(run_parser)
Comment on lines 73 to 78
Copy link
Copy Markdown
Contributor Author

@jorgeorpinel jorgeorpinel Apr 11, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW shouldn't -n|--name be added in _add_common_args()? It's a common arg both in stage add and run. Cc @karajan1001

Copy link
Copy Markdown
Contributor Author

@jorgeorpinel jorgeorpinel Apr 12, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also (💅🏼) should add_arguments() in repro/exp run use a similar private method name instead? _add_common_args() as well, even

Copy link
Copy Markdown
Contributor

@karajan1001 karajan1001 Apr 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can

BTW shouldn't -n|--name be added in _add_common_args()? It's a common arg both in stage add and run. Cc @karajan1001

I think we can do that.

Also (💅🏼) should add_arguments() in repro/exp run use a similar private method name instead? _add_common_args() as well, even

I'm not quite understand here why this method to be a private method as it is used outside its file's scope.

Copy link
Copy Markdown
Contributor

@dberenbaum dberenbaum Apr 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW shouldn't -n|--name be added in _add_common_args()? It's a common arg both in stage add and run. Cc @karajan1001

Resolved.

Copy link
Copy Markdown
Contributor

@dberenbaum dberenbaum Apr 12, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never mind, rolling this change back. It breaks a bunch of tests. I think they were intentionally left separate to support some behavior that was deprecated but not actually removed (like dvc run --single-stage), where --name was not required in dvc run. Let's stick with the UI changes.

run_parser.add_argument(
"--no-run-cache",
"--no-exec",
action="store_true",
default=False,
help=(
"Execute the command even if this stage has already been run "
"with the same command/dependencies/outputs/etc before."
),
help="Only create dvc.yaml without actually running it.",
)
run_parser.add_argument(
"--no-commit",
Expand All @@ -91,10 +89,12 @@ def add_parser(subparsers, parent_parser):
help="Don't put files/directories into cache.",
)
run_parser.add_argument(
"--single-stage",
"--no-run-cache",
action="store_true",
default=False,
help=argparse.SUPPRESS,
help=(
"Execute the command even if this stage has already been run "
"with the same command/dependencies/outputs/etc before."
),
)
_add_common_args(run_parser)
run_parser.set_defaults(func=CmdRun)
84 changes: 42 additions & 42 deletions dvc/commands/stage.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,13 @@ def run(self):


def _add_common_args(parser):
parser.add_argument(
"-f",
"--force",
action="store_true",
default=False,
help="Overwrite existing stage",
)
parser.add_argument(
"-d",
"--deps",
Expand All @@ -145,6 +152,14 @@ def _add_common_args(parser):
help="Declare dependencies for reproducible cmd.",
metavar="<path>",
).complete = completion.FILE
parser.add_argument(
"-p",
"--params",
action="append",
default=[],
help="Declare parameter to use as additional dependency.",
metavar="[<filename>:]<params_list>",
).complete = completion.FILE
parser.add_argument(
"-o",
"--outs",
Expand All @@ -163,13 +178,36 @@ def _add_common_args(parser):
metavar="<filename>",
).complete = completion.FILE
parser.add_argument(
"-p",
"--params",
"-c",
"--checkpoints",
action="append",
default=[],
help="Declare parameter to use as additional dependency.",
metavar="[<filename>:]<params_list>",
help="Declare checkpoint output file or directory for 'dvc exp run'. "
"Not compatible with 'dvc repro'.",
metavar="<filename>",
).complete = completion.FILE
parser.add_argument(
"--external",
action="store_true",
default=False,
help="Allow outputs that are outside of the DVC repository.",
)
parser.add_argument(
"--outs-persist",
action="append",
default=[],
help="Declare output file or directory that will not be "
"removed upon repro.",
metavar="<filename>",
)
parser.add_argument(
"--outs-persist-no-cache",
action="append",
default=[],
help="Declare output file or directory that will not be "
"removed upon repro (do not put into DVC cache).",
metavar="<filename>",
)
parser.add_argument(
"-m",
"--metrics",
Expand Down Expand Up @@ -226,50 +264,12 @@ def _add_common_args(parser):
help="Directory within your repo to run your command in.",
metavar="<path>",
)
parser.add_argument(
"-f",
"--force",
action="store_true",
default=False,
help="Overwrite existing stage",
)
parser.add_argument(
"--outs-persist",
action="append",
default=[],
help="Declare output file or directory that will not be "
"removed upon repro.",
metavar="<filename>",
)
parser.add_argument(
"--outs-persist-no-cache",
action="append",
default=[],
help="Declare output file or directory that will not be "
"removed upon repro (do not put into DVC cache).",
metavar="<filename>",
)
parser.add_argument(
"-c",
"--checkpoints",
action="append",
default=[],
help="Declare checkpoint output file or directory for 'dvc exp run'. "
"Not compatible with 'dvc repro'.",
metavar="<filename>",
).complete = completion.FILE
parser.add_argument(
"--always-changed",
action="store_true",
default=False,
help="Always consider this DVC-file as changed.",
)
parser.add_argument(
"--external",
action="store_true",
default=False,
help="Allow outputs that are outside of the DVC repository.",
)
parser.add_argument(
"--desc",
type=str,
Expand Down