Skip to content

Test command doesn't run when defined as list #1525

@bhawkyard1

Description

@bhawkyard1

I've found that if I define a test command as a list like so:

tests={
    "unit": {
        "command": ["python3", "-m", "pytest", "{root}/tests"],
        "requires": ["pytest"],
        "run_on": ["default"]
    }
}

which I believe should be allowed (https://github.com/AcademySoftwareFoundation/rez/wiki/Package-Definition-Guide#tests), when I run rez test, the command is never run and the command reports a success status, even if running the tests would result in a failure.

I think this is caused because rez test constructs a map object here if the test command is a list, then evaluates it here which exhausts the iterator. The iterator then gets passed down, rez runs a blank shell command, and thinks everything is fine.

Environment

  • Ubuntu LTS 22.04
  • Rez 2.112.0
  • Rez python version 3.10.12

To Reproduce
Define a package like so:

name = "test_package"
version = "1.0.0"
build_command = ""
tests={
    "fail": {
        "command": "exit 1"
    },
    "fail_lst": {
        "command": ["exit", "1"]
    }
}

Build the package.
Run rez test test_package fail
Run rez test test_package fail_lst

Expected behavior
I would expect both of the rez test commands to report the following in their output:

Running test command: exit 1
18:17:04 WARNING  Test command exited with code 1


Test results:
--------------------------------------------------------------------------------
0 succeeded, 1 failed, 0 skipped

Actual behavior
rez test test_package fail_lst does not do this, it reports that the test passed:

Running test command: exit 1


Test results:
--------------------------------------------------------------------------------
1 succeeded, 0 failed, 0 skipped

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions