METRON-2012 Unable to Execute Stellar Functions Against HBase in the REPL #1345
Conversation
|
When did this start being an issue, or is it only an issue in full-dev? I am running e4d793a on a physical cluster and I have been using Just looking to see if this impacted any releases. |
|
I am not sure when it happened. |
|
Looks like there is another odd dependency impact on metron-elasticsearch now. Daft! |
…Elasticsearch's expection. Otherwise, chaos insues
nickwallen
force-pushed
the
METRON-2012
branch
from
9bf3148 to
982c7e7
Compare
|
Multiple conflicting versions of Netty were being pulled in by the integration tests. I had to alter the pom to ensure that Netty 4.1.13 is explicitly being pulled in for the Elasticsearch integration tests. |
| <dependency> | ||
| <groupId>org.apache.storm</groupId> | ||
| <artifactId>flux-core</artifactId> | ||
| <version>${global_flux_version}</version> |
There was a problem hiding this comment.
Most storm-based projects are getting this dependency through metron-common. Rather than that, let's be more explicit that we need it.
In the future, we need to get Storm out of metron-common anyways.
| <artifactId>transport-netty4-client</artifactId> | ||
| <version>${global_elasticsearch_version}</version> | ||
| <scope>test</scope> | ||
| </dependency> |
There was a problem hiding this comment.
While the compile time dependencies for Elasticsearch are walled-off in the elasticsearch-shaded project, the same is not done for the test dependencies, including the use of ElasticsearchComponent. This is why we have this specific dependency on Elastic here.
Other changes in this PR had an unexpected ripple effect causing multiple, incompatible versions of Netty to get pulled in. I had to explicitly exclude that from some dependencies, then create an explicit dependency on the version of Netty we need.
Order also matters here. If this is too low in the dependency list, the integration tests will fail.
|
Hey @JonZeolla - Thanks for the reference to what version you are running and NOT seeing the problem. That has helped. The bug might have been introduced here during the parser reorganization. And if that is the case, then the issue is not in any Apache release yet. |
...-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/WindowLookback.java
Show resolved
Hide resolved
|
I noticed another problem after I reverted what I thought was an unnecessary change. After reverting these changes, I noticed an unrelated integration test failed. What I thought was an unnecessary change, relocated Guava to a unique path for I think what is happening is that in many places we are relocating guava to the same path, something like |
| <relocation> | ||
| <pattern>com.google.common</pattern> | ||
| <shadedPattern>org.apache.metron.guava</shadedPattern> | ||
| <shadedPattern>org.apache.metron.guava.${guava_version}</shadedPattern> |
There was a problem hiding this comment.
In all the places where Guava is relocated, I used this standard prefix, which includes the version number. org.apache.metron.guava.${guava_version}
This ensures that multiple projects pulling in different versions of Guava do not overwrite one another. This also ensures that multiple copies of a specific Guava version will not be included in the jar.
I followed this pattern throughout. The specific Guava version necessary is declared at the top. I then ensure we have an explicit dependency on Guava, and make sure that Guava is relocated to a path that includes the version number. Using a property for the Guava version makes sure the relocation and the dependency line up correctly.
The resulting JARs will look like the following.
$ jar -tvf metron-platform/metron-common/target/metron-common-0.7.1.jar | grep Stopwatch
1088 Tue Mar 05 13:11:28 EST 2019 org/apache/metron/guava/17/0/base/Stopwatch$1.class
4002 Tue Mar 05 13:11:28 EST 2019 org/apache/metron/guava/17/0/base/Stopwatch.class
|
Unrelated web test failure/ |
|
+1, thanks for the contribution @nickwallen ! Ran up full dev and validated the following: |
4 participants
When running any functions that attempt to access HBase from the REPL, an IllegalAccessError exception is thrown. This can be replicated with Stellar functions like ENRICHMENT_GET and PROFILE_GET that attempt to read from HBase.
To replicate, start the Stellar REPL with HBase and Zookeeper running and accessible.
Changes
The Stellar REPL is being launched in a manner that pulls in multiple conflicting versions of Guava. Guava 17 was being used by
metron-managementandmetron-parsers-storm, but themetron-profiler-clientwas unexpectedly pulling in Guava 12 through themetron-hbaseproject.org.apache.metron.guava.${guava_version}What's Lacking?
Testing
End to End
Streaming Enrichment
Create a Streaming Enrichment by following these instructions.
Define the streaming enrichment and save it as a new source of telemetry.
Go to the Management UI and start the new parser called 'user'.
Create some test telemetry.
Ensure that the enrichments are persisted in HBase.
Profiler
Test a profile in the REPL according to these instructions.
Deploy that profile to the Streaming Profiler.
Wait for the Streaming Profiler in Storm to flush and retrieve the measurement from HBase.
For the impatient, you can reset the period duration to 1 minute. Alternatively, you can allow the Profiler topology to work for a minute or two and then kill the
profilertopology which will force it to flush a profile measurement to HBase.Retrieve the measurement from HBase. Prior to this PR, it was not possible to query HBase from the REPL.
Install Spark using Ambari.
Stop Storm, YARN, Elasticsearch, Kibana, and Kafka.
Install Spark2 using Ambari.
Ensure that Spark can talk with HBase.
Use the Batch Profiler to back-fill your profile. To do this, follow the direction provided here.
Retrieve the entire profile, including the back-filled data.
Pull Request Checklist