Add coverage on all files #853
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
While #852 adds code coverage, it misses coverage of 5 files:
This occurs because the Coverage module (and therefore SimpleCov) need to be started before any require statement to notice a file. SimpleCov itself requires JSON, which causes the above files to be skipped in coverage metrics.
From what I can tell, there are three workaround options:
Option 1
Don't use SimpleCov for the measurement. Just use Coverage module directly and use SimpleCov to only format the output afterward.
The downside to Option 1 is that it loses the ancillary features that SimpleCov provides, That said, the main advantage is that it's the least invasive option.
Option 2
Patch
Kernel.requireto skiprequire 'json'statements coming from SimpleCov, but otherwise behave normally.This option has the obvious drawback of (lightly) messing with
Kernel, but it cleans up the use of SimpleCov to its standard API. This might be better for future maintainers to control this tool's behaviour.Option 3
Mess around with defined constants,
$LOADED_FEATURES, and/or$LOAD_PATHto force Ruby to actually run the require twice.This is what Docile does to get around the same require-sequencing problem.
Some of my earlier experiments were to try to get SimpleCov to load the Ruby built-in JSON while the test runner used the head copy (using
$LOAD_PATHshenanigans andrequire_relative), but that ran into namespace collisions, overrides, and complaints from the C bindings about pointing at the wrong thing. This spooked me off of Option 3's approach; it might be able to be done, but I personally do not have enough experience with Ruby C bindings to have a good feel for how this could break. Docile doesn't use C bindings at all, so they don't have this concern; plus, it gets used in the setup phase, so Option 2 is unavailable to them anyway.I've implemented both Option 1 and Option 2 in 18150e0 and 11dde98, respectively, so that the core JSON team has a choice about which approach is preferred. Both of them will produce coverage on all files, solving the original problem.
Option 3 is not provided for the reasons cited above, but could be if someone more familiar with the C bindings can assure me that it wouldn't be fragile.