Skip to content

Update duplicate output error msg#9138

Merged
daavoo merged 3 commits into
mainfrom
dup-output-error
Mar 23, 2023
Merged

Update duplicate output error msg#9138
daavoo merged 3 commits into
mainfrom
dup-output-error

Conversation

@dberenbaum
Copy link
Copy Markdown
Contributor

Fixes #8986

Before this PR:

$ dvc stage add -f -n train -d train.py -o model python train.py
ERROR: output 'model' is already specified in stages:
        - model.dvc
        - train

After this PR:

$ dvc stage add -f -n train -d train.py -o model python train.py
ERROR: output 'model' is already specified in stage: 'model.dvc'.
Use `dvc remove model.dvc` to stop tracking the overlapping output.

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 8, 2023

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (40484e8) 92.86% compared to head (8ef7c62) 92.86%.

Additional details and impacted files
@@           Coverage Diff            @@
##             main    #9138    +/-   ##
========================================
  Coverage   92.86%   92.86%            
========================================
  Files         458      458            
  Lines       37013    37028    +15     
  Branches     4355     5350   +995     
========================================
+ Hits        34373    34387    +14     
  Misses       2114     2114            
- Partials      526      527     +1     
Impacted Files Coverage Δ
dvc/exceptions.py 95.83% <100.00%> (+0.14%) ⬆️
dvc/repo/stage.py 94.35% <100.00%> (+0.13%) ⬆️
tests/func/test_stage.py 100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@dberenbaum dberenbaum marked this pull request as draft March 8, 2023 22:06
@dberenbaum

This comment was marked as outdated.

@dberenbaum dberenbaum closed this Mar 8, 2023
@dberenbaum dberenbaum reopened this Mar 8, 2023
@dberenbaum dberenbaum marked this pull request as ready for review March 8, 2023 22:10
@dberenbaum dberenbaum requested a review from a team March 17, 2023 19:12
@omesser omesser self-requested a review March 19, 2023 21:03
Copy link
Copy Markdown
Contributor

@omesser omesser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be compacted a bit more maybe if we tradeoff some "accuracy", e.g.

  • dvc remove message can be unified
  • maybe the multiline '\t-' variant is not needed

But is fine either way imo, approving

@daavoo daavoo force-pushed the dup-output-error branch from a2927e5 to 9c82ebf Compare March 21, 2023 11:54
Comment thread dvc/repo/stage.py
Comment on lines +183 to +188
try:
self.repo.check_graph(stages={stage})
except OutputDuplicationError as exc:
# Don't include the stage currently being added.
exc.stages.remove(stage)
raise OutputDuplicationError(exc.output, exc.stages) from None
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice to add a test for this

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any suggestions for where to add it?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably in tests/fun/test_add, something like: https://github.com/iterative/dvc/blob/121d56b7391f5325e5eb6ff41448980f0ede00df/tests/func/test_add.py#L749-L753 or even just update that one and use match with the expected exception message

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the message vary if the order is different from the one in the test (i.e. you run dvc add first)?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it does. This PR doesn't change the error for dvc add, only for dvc stage add, so not sure if it makes sense to put it there. Had trouble finding any tests about dvc stage add beyond testing the cmd args.

Copy link
Copy Markdown
Contributor

@daavoo daavoo Mar 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. I understand.

From a quick look, I think I would personally add it to tests/func/test_stage.py then. Like:

def test_stage_add_duplicated_output(tmp_dir, dvc):
    tmp_dir.dvc_gen("foo", "foo") 
    dvc.add("foo")

    with pytest.raises(
        OutputDuplicationError,
        match="Use `dvc remove foo.dvc` to stop tracking the overlapping output."
    ):
        dvc.stage.add(name="duplicated", cmd="echo bar > foo", outs=["foo"])

But I am also confused about where the logic for StageLoad.create is/should be tested cc @skshetry

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the test you suggested @daavoo. It makes sense to me.

@dberenbaum
Copy link
Copy Markdown
Contributor Author

  • dvc remove message can be unified

Thanks @omesser. Tried to clean this up a bit.

@daavoo daavoo enabled auto-merge (squash) March 23, 2023 08:26
@daavoo daavoo disabled auto-merge March 23, 2023 09:04
@daavoo daavoo enabled auto-merge (squash) March 23, 2023 09:05
@daavoo daavoo force-pushed the dup-output-error branch 2 times, most recently from 7489ca4 to fbc86a1 Compare March 23, 2023 13:32
@daavoo daavoo force-pushed the dup-output-error branch from fbc86a1 to 8ef7c62 Compare March 23, 2023 15:21
@daavoo daavoo merged commit da3bbcc into main Mar 23, 2023
@daavoo daavoo deleted the dup-output-error branch March 23, 2023 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

stage add: improve error for overlapping outputs

3 participants