METRON-1340: Improve e2e tests for metron alerts #857
Conversation
|
I haven't looked at the code yet, but I was able to pull this down and run the e2e tests repeatedly without failures, so this is definitely great stuff. |
| export let DEFAULT_FACETS = ['source:type', 'ip_src_addr', 'ip_dst_addr', 'host', 'enrichments:geo:ip_dst_addr:country']; | ||
| export let DEFAULT_GROUPS = ['source:type', 'ip_src_addr', 'ip_dst_addr', 'host', 'enrichments:geo:ip_dst_addr:country']; | ||
| export let INDEXES = environment.indices ? environment.indices.split(',') : []; | ||
| export let POLLING_DEFAULT_STATE = environment.defaultPollingState; |
There was a problem hiding this comment.
What does the POLLING_DEFAULT_STATE setting do?
There was a problem hiding this comment.
Added info for this in Readme and CC
There was a problem hiding this comment.
Would it be possible to change this to a better name? Something like DISABLE_POLLING? Setting POLLING_DEFAULT_STATE to true to stop polling doesn't make sense to me.
|
Took a first pass at this and I feel like the e2e tests are much improved. Great progress and good job so far. I was able to get several successful runs whereas before it was difficult to get a single successful run. A couple of comments/suggestions:
|
Removed sleep for clickActionDropdownOption Removed the version check for node
|
|
@merrimanr renamed the variable POLLING_DEFAULT_STATE to DISABLE_POLLING |
Added e2e maven profile
|
I've hit the same intermittent errors trying this, e.g. One of the very first things that failed was Is it possible there's an intermittent error getting the tests initially lined up that just carries through everything? @iraghumitra Can you also update the README with the instructions for running this with the mvn profile + any troubleshooting or other steps that need to happen? |
|
Something I'm missing? |
|
Seems that my NodeJS and npm versions got mangled since last time I ran this. Perhaps migrating laptops, upgrading to High Sierra, and switching from MacPorts to HomeBrew might have something to do with it ;) |
|
A couple notes for the community - if your node/npm versions are out of date and you've already built the project, you'll run into trouble with versions not matching against the expected node version, e.g. |
|
Should we also run the tests using the new Maven profile? The README should probably be updated to reflect this as well. |
|
OK, I was able to get the tests to run. First run I get a single error. @merrimanr and I are talking through it now. |
|
I was reviewing the protractor documentation here and I noticed this:
I don't see that setting in protractor.conf.js. Shouldn't it be included? Have all the tests been updated to use async/await instead of web driver flow control? |
|
Follow up from @merrimanr and my work yesterday. We upped the versions of Node to 9.2.1. Per the doc, >8 is required to work with async/await. For good measure, I also set the NPM version to 5.6.0. We didn't touch Jasmine, but the Protractor docs also state that it should be > 2.7. Looks like we are currently using 2.5.2 per the package.json file. We may want to consider increasing that version as well. We added There are still a large number of failures due to disabling the promise manager, but still having code throughout the test suite that leverages the older style. It's unclear if this will resolve all stability issues, but I think this is moving in the right direction. |
|
@merrimanr @mmiklavc @iraghumitra What are the next steps towards being able to have this stable? It sounds like there was some playing around with versions, and that there may be some more structural changes to the code style. How do you propose we move forward? I'd really like to see this stuff made stable and I'm happy to help out with more testing or whatever else we need. |
|
@justinleet I believe @iraghumitra is still working on the SELENIUM_PROMISE_MANAGER change proposed above but I will defer to him. |
|
What is the status of this pr? it is 29 day without comment, and conflicted, literally, and perhaps figuratively |
|
Got busy with too many things happening around me, will yield this a conclusion next week. |
|
Please close this |
|
I also requested that this be closed in this ticket; https://issues.apache.org/jira/browse/INFRA-17251 |
7 participants
Contributor Comments
This PR aims to improve the existing e2e tests of metro-alerts in the areas mentioned below. The PR doesn't add/remove any functionality to the existing alerts ui. However, there are few changes in HTML templates where I a needed a new selector for tests.
The tests are written using:
OS: macOS High Sierra 10.13.1
nodejs: v8.9.1.
Chrome: 62.0.3202.94
The tests should run reasonably well on other platforms since I personally didn't test on them I would request you to report any tests failure with information about your OS, nodejs and browser version.
I bumped up the version of the protractor to 5.2.0, so please run npm install before testing the PR. Please let me know if you need any more details about the PR.
Testing
If all the e2e tests run fine the PR is good.
Documentation
No new documentation is needed specifically for this PR. The steps to run the e2e tests remains the way they are before. I added a couple of caveats in Readme about deleting 'metron_update' table and headless chrome. Please let me know if we can improve our docs for e2e tests I would be glad to add it.
Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our Development Guidelines for the complete guide to follow for contributions.
Please refer also to our Build Verification Guidelines for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
For all changes:
For code changes:
Have you included steps to reproduce the behavior or problem that is being changed or addressed?
Have you included steps or a guide to how the change may be verified and tested manually?
Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
Have you written or updated unit tests and or integration tests to verify your changes?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?
For documentation related changes:
Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via
site-book/target/site/index.html:Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that travis-ci is set up for your personal repository such that your branches are built there before submitting a pull request.