METRON-2150: [UI] User not able to filter by multiple values of the same field on Alerts UI

[tiborm]

Contributor Comments

While I was testing filtering in Alerts UI I found a few broken use cases. These broken scenarios of filtering were fixed as a part of this PR. It also contains a few lines of code cleanup around the bugfixes.

Testing

Build a full dev from this branch or run the Alerts UI in this branch locally against an existing full dev. (The fix contains no backend changes.)
Make sure you have alert entries with status RESOLVE, DISMISS, NEW and OPEN.

Use the following filters to test:

Excluding filters: (currently only works with ElasticSearch)
-alert_status:DISMISS AND -alert_status:RESOLVE
-alert_status:(DISMISS OR RESOLVE)
Field grouping:
alert_status:(NEW OR OPEN)
Whitespace and wildcard:
' alert_status:OPEN'
(don't use the single quotes, but make sure you have a space at the beginning of the query)
'alert_status:OPEN '
(don't use the single quotes, but make sure you have a space at the end of the query)
* -alert_status:OPEN

None of these queries works with the Alerts UI in the Master. All should work with this fix.

Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.
Please refer to our Development Guidelines for the complete guide to follow for contributions.
Please refer also to our Build Verification Guidelines for complete smoke testing guides.

In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:

For all changes:

• Is there a JIRA ticket associated with this PR? If not one needs to be created at Metron Jira.
• Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
• Has your PR been rebased against the latest commit within the target branch (typically master)?

For code changes:

• Have you included steps to reproduce the behavior or problem that is being changed or addressed?

• Have you included steps or a guide to how the change may be verified and tested manually?

• Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:

  mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh 
  

• Have you written or updated unit tests and or integration tests to verify your changes?

• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

• Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?

For documentation related changes:

• Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via site-book/target/site/index.html:

  cd site-book
  mvn site
  

• Have you ensured that any documentation diagrams have been updated, along with their source files, using draw.io? See Metron Development Guidelines for instructions.

Note:

Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that travis-ci is set up for your personal repository such that your branches are built there before submitting a pull request.

[sardell]

@tiborm When I try to search with a provided time range, I get nothing returned unless I search for 'All Time'. I inspected the request payload going to the server and here is what it looks like when I search for 'This week'. Notice the query field:

facetFields: []
from: 0
indices: []
query: "(timestamp:[1560958398000 TO 1560959298000] OR metron_alert.timestamp:[1560958398000 TO 1560959298000]) AND (timestamp:[1560636000000 TO 1561240799000] OR metron_alert.timestamp:[1560636000000 TO 1561240799000])"
size: 25

When I perform the same search with the latest from master, this is what the request looks like:

tFields: []
from: 0
indices: []
query: "(timestamp:[1560636000000 TO 1561240799000] OR metron_alert.timestamp:[1560636000000 TO 1561240799000])"
size: 25

[tiborm]

@sardell Thank you for pointing out to this! I pushed the fix.

I also fixed another issue I found which was about filtering fields contains colon characters like source:type. I also covered these scenarios with unit tests to pin down proper functionality.

[sardell]

+1. I tested this on full dev it seems to work without a hitch for me. In my view, this really improves the capabilities of our search bar. I would like for another committer to take a quick glance before merging this into master, but it looks great to me.

[anandsubbu]

When testing this PR, I observed one case that does not behave as expected.

When a new alert is generated, the alert_status field is not set, but the default status is shown as 'NEW' in the UI. Attempting a filter on alerts without the field not being set does not hence yield the desired result.

So the following filter:

alert_status:(NEW OR OPEN)

will not display alerts which have their default status (i.e. NEW) set during creation time. However, if one manually set the status of an alert to NEW (e.g. switch from NEW (default) -> OPEN -> NEW), then the query works fine.

I have created METRON-2167 to track this issue separately.

[mmiklavc]

@anandsubbu is 2167 an issue introduced by this PR, or a pre-existing bug?

[tiborm]

@mmiklavc it's an existing issue but let's wait for @anandsubbu to confirm.

(With my last commit, I renamed a function to clarify it's purpose. No further changes coming.)

[anandsubbu]

@mmiklavc , @tiborm - it is a pre-existing issue.

[mmiklavc]

+1 by inspection, thanks @tiborm. Side note, thanks for taking the initiative to make minor improvements along the way, e.g. extracting strings like guid into constants. Good stuff!