Traffic Ops integration tests#4959
Conversation
|
How hard would it be to get junit reports for v1, v2, and v3 as artifacts? |
|
Probably not too hard. But I don't know how yet. |
|
Running v1, v2, and v3 in separate steps would be nice so we can tell at a glance which API version had failing tests. |
|
If they're separate steps, v2 and v3 tests won't run if the preceding step - v1 tests - fails. If they're separate jobs then we duplicate a lot of work, which can cause it to slow down because it's waiting for executors. I think a better solution would be to use the reporting API to provide detailed error descriptions. |
By default, sure, but |
|
We're about to find out |
|
I wound up separating out the database setup to alleviate what became a much longer runtime, so instead of |
zrhoffman
left a comment
zrhoffman
left a comment
There was a problem hiding this comment.
From my testing, checking for always() was still necessary if the tests failed for a previous API version.
| envsubst </cdn.json >cdn.conf | ||
| mv /database.json ./database.conf | ||
|
|
||
| ./traffic_ops_golang --cfg ./cdn.conf --dbcfg ./database.conf >out.log 2>err.log & |
There was a problem hiding this comment.
Should stdout and stderr go to the same file so we have context for errors and warnings? Errors will still be pretty easy to find by searching for ERROR:.
There was a problem hiding this comment.
stdout won't contain anything too helpful, I pretty much just do that to avoid writing to closed file descriptors, just in case. info, event, and debug logging is piped to /dev/null by the logging configuration. I tried including those in stderr as well, but the resulting logs after the tests finished were so large my browser refused to load the checks page that printed them. One day, we can put the entire log in a build artifact, so at that point we don't need to worry about what the GH UI can handle and can include everything in one stream.
zrhoffman
left a comment
zrhoffman
left a comment
There was a problem hiding this comment.
- API tests are in separate job steps and the results are easy to find without digging into the logs
- v2 and v3 will still run if a previous API test version failed
- JSON and YAML are now read from external files
We could always improve how we show the logs of TO itself, but none of that is directly related to the API tests and can be added in the future IMO.
Looks good to me, this will help stability and PR review considerably.
3 participants
What does this PR (Pull Request) do?
This PR adds running the API tests to Pull Requests, any time a source file for Traffic Ops, one of its Go clients, or the Go client/API integration tests changes.
Which Traffic Control components are affected by this PR?
None.
What is the best way to verify this PR?
Observe the test output.
The following criteria are ALL met by this PR