METRON-155 Added query filtering capability for PCAP via Metron REST API

[mmiklavc]

https://issues.apache.org/jira/browse/METRON-155

Have not run the Vagrant deploy yet, but this is big PR so i wanted to get this reviewed asap.

• Added PCAP filter by query.
• Use Java's Stream, Function, and Predicate impls.
• Added QueryPcapFilterTest
• Added tests in REST api for new query option
• Added integration tests to PCAP topology
• Fixed bug in QueryParser - handle empty string as pass-through impl like WHERE TRUE

[cestella]

Could you change the javadocs here to reflect the method?

[mmiklavc]

Looks like older leftover auto-generated javadoc - I'll revise the packaging and make a proper javadoc for the methods in that class.

[cestella]

On the whole, this is great. Definitely a great feature and an impressive 2nd PR. Thanks for the contribution!

Please make sure you didn't inadvertently regress the existing pcap functionality on the full-dev-vagrant image by following the testing plan in the original PR to move PCAP to hbase here

[mmiklavc]

Ok, made a number of changes for review. Thanks for the feedback! Should have the remaining fixes/improvements avail soon and will also test per comments on full-dev-vagrant.

[nickwallen]

For some reason, I cannot comment on the original JIRA, so I will comment here.

I think using a custom created language for this purpose is going to be confusing for our users. Most network tools support what is called Berkely Packet Filter (BPF) syntax. The user community is going to be fairly familiar with this syntax. I would suggest that we use BPF syntax for this purpose.

[cestella]

@nickwallen Agreed we should support BPF. This PR makes the filter pluggable and we already have the query language. We can have a follow-on PR for BPF support IMO.

[nickwallen]

I think that makes sense, especially since this creates the 'hook' for later contributions of BPF.

The only downside is that we don't want to confuse users by having 8 ways to do things. But I don't think that will be a problem as long as we eventually get the BPF functionality.

[mmiklavc]

fyi, I was able to confirm no regressions in the existing PcapJob query via the instructions located here - #93

Note: I had to up map/reduce heap to 1GB and container mem to 1.2GB to avoid OOM errors.

[cestella]

+1, got my vote.

[dlyle65535]

+1 worked in EC2.