In the process of testing a new flopy feature I discovered two tests where a package fails to load but the tests are still passing. Note that neither of these tests are expecting a package load failure or are testing for that failure. These tests illustrate a broader issue that is worth discussing here.
It looks like flopy's default behavior allowed these bugs to go undetected for so long. When loading a model flopy's default setting is to have the forgive flag set to True and the verbose flag set to False. Flopy's default behavior is to report nothing and continue when a package fails to load. While the load_fail flag can be checked to see if any packages failed, our test cases fail to check this flag in 74 out of 96 loads. At least two of these cases result in a test passing when it should be failing. These two cases are:
t028_test.py file running the 5_saltlake test. This test fails to load the bcf package due to an error with flopy. The bcf file has the ltype field spread across multiple lines. Flopy correctly concatenates these lines together but fails to strip out the carriage return character which it later tries to convert to an integer and fails. The test does not catch this since it does not check the load_fail flag.
t012_test.py file running test_mf2000_HSSTest. This test has an invalid wel package file (missing ITMP NP line) and flopy fails to load it. This test was not designed to test for an invalid wel package and I suspect the invalid wel package is an oversight that was not caught due to the test case not checking the load_fail flag.
Before I update all test cases to check for load_fail, I first want to see if there is any interest in changing the default behavior so that the forgive flag defaults to False. If we are getting tripped up by not checking load_fail in our test cases I suspect end users are having similar problems. Also, I suspect users expect all packages to load in most use cases.
In the process of testing a new flopy feature I discovered two tests where a package fails to load but the tests are still passing. Note that neither of these tests are expecting a package load failure or are testing for that failure. These tests illustrate a broader issue that is worth discussing here.
It looks like flopy's default behavior allowed these bugs to go undetected for so long. When loading a model flopy's default setting is to have the forgive flag set to True and the verbose flag set to False. Flopy's default behavior is to report nothing and continue when a package fails to load. While the load_fail flag can be checked to see if any packages failed, our test cases fail to check this flag in 74 out of 96 loads. At least two of these cases result in a test passing when it should be failing. These two cases are:
t028_test.py file running the 5_saltlake test. This test fails to load the bcf package due to an error with flopy. The bcf file has the ltype field spread across multiple lines. Flopy correctly concatenates these lines together but fails to strip out the carriage return character which it later tries to convert to an integer and fails. The test does not catch this since it does not check the load_fail flag.
t012_test.py file running test_mf2000_HSSTest. This test has an invalid wel package file (missing ITMP NP line) and flopy fails to load it. This test was not designed to test for an invalid wel package and I suspect the invalid wel package is an oversight that was not caught due to the test case not checking the load_fail flag.
Before I update all test cases to check for load_fail, I first want to see if there is any interest in changing the default behavior so that the forgive flag defaults to False. If we are getting tripped up by not checking load_fail in our test cases I suspect end users are having similar problems. Also, I suspect users expect all packages to load in most use cases.