Skip to content

Comments

show code coverage with Codecov.io#4587

Merged
WalterBright merged 2 commits intodlang:masterfrom
wilzbach:test-codecov
Jul 28, 2016
Merged

show code coverage with Codecov.io#4587
WalterBright merged 2 commits intodlang:masterfrom
wilzbach:test-codecov

Conversation

@wilzbach
Copy link
Contributor

Hi all, the idea for this automation is simple, let's ensure (1) that no PR decreases the code coverage, (2) help reviewers judging code coverage improvements PRS, (3) provide simple tools for contributors to see which parts are uncovered.

You can preview this here.

That being said, this won't be perfect in the near future as Travis can only run Linux and Mac OS builds, so special versioned paths will not be treated right. The idea here is to get an easy 90% solution that already helps reviewers in most cases (=a lot better than 0%). In the future we might add Codecov reporting to the AutoTester.

Now to this PR:

  1. Thanks to the help from @stevepeak, Codecov.io is able to recognize the code coverage output from our builds
  2. I haven't figured out yet how I can get the test runner to report the test coverage, for now I settled with
    make -f posix.mak std.test etc.test
  3. I ran into a couple of issues during my first attempt, so don't merge yet. At the moment I was lazy and just commented them out, but I/we should definitely investigate the root cause of them:
  • entropy in std.numeric (and thus schwarzSort in std.algorithm.sorting)
  • utf-16le in std.encoding
  • iso-8859-1 not recognized for std.net.curl
  1. As the newest druntime and dmd are needed, we now have to move phobos in its own folder and then clone the other repos. It makes the Travis script a bit ugly, better ideas are of course welcome.

(if everything goes well, CodeCov should comment on this PR)

@codecov-io
Copy link

codecov-io commented Jul 10, 2016

Current coverage is 88.68% (diff: 100%)

No coverage report found for master at f9dd0f8.

Powered by Codecov. Last update f9dd0f8...d8c181b

std/numeric.d Outdated
}

@safe unittest
unittest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why turn off @safe?

@stevepeak
Copy link

@wilzbach Nice job :) Let me know if you have any questions about Codecov configuration.

@wilzbach wilzbach mentioned this pull request Jul 12, 2016
@wilzbach wilzbach force-pushed the test-codecov branch 2 times, most recently from f6ce3ec to 721bbce Compare July 18, 2016 17:33
.travis.yml Outdated
# test code coverage
- (cd dmd && make -f posix.mak)
- (cd druntime && make -f posix.mak)
- (cd phobos && make -f posix.mak $(find std etc -name "*.d" | sed "s/[.]d$/.test/" | grep -vE '(std.encoding|net.curl)' )
Copy link
Contributor Author

@wilzbach wilzbach Jul 18, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Encoding's aren't registered properly for single tests - see e.g. #4493 and the reported bug for more infos.

@wilzbach wilzbach changed the title [WIP] show code coverage with Codecov.io show code coverage with Codecov.io Jul 18, 2016
@wilzbach
Copy link
Contributor Author

Travis is passing finally 🎉, so this experiment is ready.

As mentioned earlier there are some weird bootstrapping issues with std.encoding, but they are reported and being worked on. Thus these modules are excluded currently from the coverage report.

You can preview the automatic coverage report here.

In case you are unfamiliar with CodeCov, it's a service that processes coverage reports and will hopefully help us to spot coverage regression and judge coverage improvements.

The behavior of CodeCov is highly configurable, so if anything annoys or will annoy you, we most likely will find a solution ;-)

@WalterBright WalterBright merged commit aa017e5 into dlang:master Jul 28, 2016
@wilzbach wilzbach deleted the test-codecov branch July 28, 2016 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants