CI: add codecov to Azure CI by tylerjereddy · Pull Request #2720 · MDAnalysis/mdanalysis

tylerjereddy · 2020-06-09T00:07:59Z

add a codecov upload to Azure CI (32-bit
Windows)
I was a bit surprised that we don't
seem to have a yaml file for codecov
settings (?), so I didn't add a lint
stage for our codecov config
reviewers should check carefully that the upload actually happens because the job passed without uploading on my fork (i.e., the upload utility ran, but then exited with Commit sha does not match Azure build. Please upload with the Codecov repository upload token to resolve issue. , hopefully just because it was on my fork...)

Changes made in this Pull Request:

PR Checklist

Tests?
Docs?
CHANGELOG updated?
Issue raised/referenced?

codecov · 2020-06-09T01:18:27Z

Codecov Report

Merging #2720 into develop will increase coverage by 0.74%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop    #2720      +/-   ##
===========================================
+ Coverage    91.43%   92.18%   +0.74%     
===========================================
  Files          178      183       +5     
  Lines        23479    23977     +498     
  Branches      3090     3090              
===========================================
+ Hits         21469    22104     +635     
- Misses        1421     1808     +387     
+ Partials       589       65     -524

Impacted Files	Coverage Δ
package/MDAnalysis/topology/tpr/obj.py	`96.77% <0.00%> (-3.23%)`	⬇️
...onality_reduction/DimensionalityReductionMethod.py	`97.05% <0.00%> (-2.95%)`	⬇️
package/MDAnalysis/tests/__init__.py	`100.00% <0.00%> (ø)`
package/MDAnalysis/due.py	`61.11% <0.00%> (ø)`
package/MDAnalysis/analysis/legacy/x3dna.py	`0.00% <0.00%> (ø)`
package/MDAnalysis/analysis/legacy/__init__.py	`0.00% <0.00%> (ø)`
package/MDAnalysis/tests/datafiles.py	`21.42% <0.00%> (ø)`
package/MDAnalysis/core/selection.py	`99.50% <0.00%> (+<0.01%)`	⬆️
...sis/analysis/encore/clustering/ClusteringMethod.py	`96.92% <0.00%> (+0.04%)`	⬆️
package/MDAnalysis/analysis/base.py	`95.45% <0.00%> (+0.05%)`	⬆️
... and 83 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3192968...497b1b6. Read the comment docs.

tylerjereddy · 2020-06-09T02:13:09Z

I still see Commit sha does not match Azure build. Please upload with the Codecov repository upload token to resolve issue. I suppose we could use the token if we really have to. The Azure blog suggests we don't have to, but the example is on Linux so..

orbeckst · 2020-06-10T09:01:44Z

How sensitive is the upload token? The codecov instructions (from the settings) say

Add to your codecov.yml

codecov:
  token:  xxxxx

or Set in your environment variables

CODECOV_TOKEN="xxxx"

On travis we could encrypt the token. Can we add it to Azure encrypted or via configuration?

tylerjereddy · 2020-06-12T02:57:54Z

I've saved the encrypted codecov token in Azure UI configuration, but I think it does not get automatically decrypted. If the CI fails to upload again this time I'll have to use the mapping techniques they describe to decrypt in the docs for Azure secret vars

tylerjereddy · 2020-06-12T12:31:17Z

Ok, I need to decrypt the token still it seems

orbeckst · 2020-06-12T17:53:07Z

Do you have access to the decrypted codecov token, i.e., when you go to https://codecov.io/gh/MDAnalysis/mdanalysis/settings and login with GitHub, can you see it?

tylerjereddy · 2020-06-12T19:53:19Z

Yeah, I just need to follow the Azure docs to use it properly. I'll be slow, but should get it done. Failing that, the new codecov team is pretty responsive to GitHub pings, but will try on my own first.

tylerjereddy · 2020-06-13T00:46:25Z

Now it says Provided token is not a UUID. @thomasrockhu Any hints on the Azure + Windows + codecov? I probably have something silly like needing to add quotes around the secret token I pasted in Azure UI or something.

thomasrockhu · 2020-06-13T03:56:16Z

Hmmm, I'm not sure why that is. I copy and pasted my token into this project here (and actually used your yaml as a template): https://dev.azure.com/tomhu1096/codecov-github-test/_build/results?buildId=8&view=logs&j=e4f8c82b-de6b-5575-3950-14744af47318&t=9fea38d8-bc43-58c0-468d-6e85974c7041

Are you adding quotes or spaces or anything like that?

thomasrockhu · 2020-06-13T14:20:37Z

As an example, this is what my settings page looks like in Codecov

And this is what I pasted into Azure

tylerjereddy · 2020-06-13T20:37:03Z

I checked the "Keep this value secret" option, so I think the decryption component of the workflow may be adding some additional complication.

Is there no reason to keep the codecov tokens secret? If not, then I'll just do away with that and maybe it will work like yours does.

thomasrockhu · 2020-06-13T23:31:49Z

@tylerjereddy apologies, I actually had the Keep this value secret checkbox clicked but wanted you to be able to see what the format looked like

tylerjereddy · 2020-06-13T23:51:18Z

My setup should have been the same--I've tried deleting and setting up the variable again with a fresh copy-paste of the token.

tylerjereddy · 2020-06-14T01:38:29Z

Same thing--possible differences with your setup might be:

yours is a direct push to master branch instead of a PR from a fork
we are using 32-bit Windows instead of the normal 64-bit used in your example

thomasrockhu · 2020-06-14T22:44:54Z

@tylerjereddy, I think I'm using 32-bit now and it's in a PR. https://dev.azure.com/tomhu1096/codecov-github-test/_build/results?buildId=13&view=logs&jobId=eb841e9f-6e1e-5c29-3ad5-74ec2c658069&j=d420aaaa-0ae7-5c90-2d94-1e61729d0f21&t=2ebca456-40b1-5871-95d6-ad048e69e584

thomasrockhu · 2020-06-16T03:24:41Z

@tylerjereddy, I see this is still an issue. Would you be able to echo $CODECOV_TOKEN and see what it's actually trying to spit it out as? (An admin can always and should regenerate that token on the Codecov UI, and leaking this secret has very low impact over a short period of time).

tylerjereddy · 2020-06-16T03:54:55Z

I'll see what I can do--I should maybe check if the token is used anywhere else first, but I think most CI services don't even need it for public repos. The most recent attempt was with quotes added--the first two had no quotes.

orbeckst · 2020-06-16T03:56:45Z

I don't think we use the token elsewhere. Just change it when done.

tylerjereddy · 2020-06-17T02:21:57Z

Ok, I've added an echo for the variable--we'll see what happens I guess

thomasrockhu · 2020-06-17T16:55:15Z

Hmmm... I'm going to fork the repo and make the change to see why this might be happening sometime tonight or tomorrow. Sorry this is blocking your usage of Codecov on AzureCI

thomasrockhu · 2020-06-24T14:49:20Z

@tylerjereddy, we did some work on the Azure pipelines side of things which may have helped with this issue. My build now seems to work, and you can see the build report here. Could you try running this again to see if it's been fixed?

* add a codecov upload to Azure CI (32-bit Windows) * I was a bit surprised that we don't seem to have a `yaml` file for codecov settings (?), so I didn't add a lint stage for our codecov config

tylerjereddy · 2020-06-25T03:41:34Z

@thomasrockhu Cool, I've rebased and force pushed to trigger the rebuild :)

thomasrockhu

Apologies, we fixed an issue with tokenless Azure uploads for public repositories, so you don't have to specify the token at all anymore

azure-pipelines.yml

Co-authored-by: Thomas Hu <thomasrockhu@users.noreply.github.com>

tylerjereddy · 2020-06-26T02:02:15Z

thanks, I committed your suggestions just now

thomasrockhu · 2020-06-26T02:29:44Z

@tylerjereddy success!

tylerjereddy · 2020-06-26T02:36:48Z

Great! The PR should be ready to merge when the rest of the CI completes. I assume we can ignore the complaint from curl in there since the upload seems to happen anyway.

thomasrockhu · 2020-06-26T04:42:45Z

@tylerjereddy, yes the curl complaint is fine (it's actually my fault and I need to find the time to understand why)

orbeckst · 2020-06-26T05:38:46Z

Well, that increased our coverage by almost 1%!

Thank you @thomasrockhu for helping, much appreciated! (And makes me happy to use codecov for other projects!)

orbeckst

I am very happy with getting coverage from Azure, too. Thank you @thomasrockhu and @tylerjereddy for figuring it out.

orbeckst · 2020-06-26T05:41:43Z

@tylerjereddy I‘ll leave it to you to squash and merge, just in case you want to refine anything.

thomasrockhu · 2020-06-26T13:28:48Z

@orbeckst happy to help!

* CI: add codecov to Azure CI * add a codecov upload to Azure CI (32-bit Windows) Co-authored-by: Thomas Hu <thomasrockhu@users.noreply.github.com>

tylerjereddy added OpSys-Windows testing 32 bit Continuous Integration labels Jun 9, 2020

tylerjereddy closed this Jun 12, 2020

tylerjereddy reopened this Jun 12, 2020

tylerjereddy force-pushed the add_azure_codecov branch from f276e64 to 4358b39 Compare June 13, 2020 00:10

tylerjereddy closed this Jun 13, 2020

tylerjereddy reopened this Jun 13, 2020

tylerjereddy closed this Jun 16, 2020

tylerjereddy reopened this Jun 16, 2020

tylerjereddy force-pushed the add_azure_codecov branch from 4358b39 to 123f88e Compare June 17, 2020 02:21

thomasrockhu mentioned this pull request Jun 18, 2020

Add Codecov to Azure pipelines #2771

Closed

CI: add codecov to Azure CI

904f162

* add a codecov upload to Azure CI (32-bit Windows) * I was a bit surprised that we don't seem to have a `yaml` file for codecov settings (?), so I didn't add a lint stage for our codecov config

tylerjereddy force-pushed the add_azure_codecov branch from 123f88e to 904f162 Compare June 25, 2020 03:40

thomasrockhu reviewed Jun 25, 2020

View reviewed changes

azure-pipelines.yml Outdated Show resolved Hide resolved

azure-pipelines.yml Outdated Show resolved Hide resolved

tylerjereddy and others added 2 commits June 25, 2020 20:01

Update azure-pipelines.yml

984e913

Co-authored-by: Thomas Hu <thomasrockhu@users.noreply.github.com>

Update azure-pipelines.yml

497b1b6

Co-authored-by: Thomas Hu <thomasrockhu@users.noreply.github.com>

orbeckst approved these changes Jun 26, 2020

View reviewed changes

orbeckst assigned tylerjereddy Jun 26, 2020

tylerjereddy merged commit dddb784 into MDAnalysis:develop Jun 26, 2020

orbeckst added this to the 1.0.x milestone Jun 26, 2020

orbeckst pushed a commit that referenced this pull request Jun 26, 2020

CI: add codecov to Azure CI (#2720)

04222d9

* CI: add codecov to Azure CI * add a codecov upload to Azure CI (32-bit Windows) Co-authored-by: Thomas Hu <thomasrockhu@users.noreply.github.com>

tylerjereddy deleted the add_azure_codecov branch June 27, 2020 19:46

orbeckst pushed a commit that referenced this pull request Jul 12, 2020

CI: add codecov to Azure CI (#2720)

dd4490a

* CI: add codecov to Azure CI * add a codecov upload to Azure CI (32-bit Windows) Co-authored-by: Thomas Hu <thomasrockhu@users.noreply.github.com>

fiona-naughton added the maintainability label Sep 26, 2023

Conversation

tylerjereddy commented Jun 9, 2020

Changes made in this Pull Request:

PR Checklist

Uh oh!

codecov bot commented Jun 9, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

tylerjereddy commented Jun 9, 2020

Uh oh!

orbeckst commented Jun 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tylerjereddy commented Jun 12, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tylerjereddy commented Jun 12, 2020

Uh oh!

orbeckst commented Jun 12, 2020

Uh oh!

tylerjereddy commented Jun 12, 2020

Uh oh!

tylerjereddy commented Jun 13, 2020

Uh oh!

thomasrockhu commented Jun 13, 2020

Uh oh!

thomasrockhu commented Jun 13, 2020

Uh oh!

tylerjereddy commented Jun 13, 2020

Uh oh!

thomasrockhu commented Jun 13, 2020

Uh oh!

tylerjereddy commented Jun 13, 2020

Uh oh!

tylerjereddy commented Jun 14, 2020

Uh oh!

thomasrockhu commented Jun 14, 2020

Uh oh!

thomasrockhu commented Jun 16, 2020

Uh oh!

tylerjereddy commented Jun 16, 2020

Uh oh!

orbeckst commented Jun 16, 2020

Uh oh!

tylerjereddy commented Jun 17, 2020

Uh oh!

thomasrockhu commented Jun 17, 2020

Uh oh!

thomasrockhu commented Jun 24, 2020

Uh oh!

tylerjereddy commented Jun 25, 2020

Uh oh!

thomasrockhu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tylerjereddy commented Jun 26, 2020

Uh oh!

thomasrockhu commented Jun 26, 2020

Uh oh!

tylerjereddy commented Jun 26, 2020

Uh oh!

thomasrockhu commented Jun 26, 2020

Uh oh!

orbeckst commented Jun 26, 2020

Uh oh!

orbeckst left a comment

Choose a reason for hiding this comment

Uh oh!

orbeckst commented Jun 26, 2020

Uh oh!

thomasrockhu commented Jun 26, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

codecov bot commented Jun 9, 2020 •

edited

Loading

orbeckst commented Jun 10, 2020 •

edited

Loading

tylerjereddy commented Jun 12, 2020 •

edited

Loading