Speed up non-release builds

[kennknowles]

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

• Make sure the PR title is formatted like:
  [BEAM-<Jira issue #>] Description of pull request
• Make sure tests pass via mvn clean verify. (Even better, enable
  Travis-CI on your fork and ensure the whole test matrix passes).
• Replace <Jira issue #> in the title with the actual Jira issue
  number, if there is one.
• If this contribution is large, please file an Apache
  Individual Contributor License Agreement.

---

We can probably take this further, but this is a first step at isolating simple command-line mvn verify and mvn install from the extra artifacts that are important for release building & testing. We should keep the longer build for post-commit, maybe pre-commit, but definitely not for devs on the command line.

[kennknowles]

R: @davorbonaci

[lukecwik]

I think we need to keep this in the default profile since we have modules that depend on others modules test jars because of re-used testing utilities. Will need to factor those out to separate testing packages.

[kennknowles]

IMO such a dependency is a bug, for sure. Factoring out into a test utility is best, if possible. But here is a case where we cannot:

• class A depends on SDK main
• SDK test depends on class A
• non-SDK test depends on class A

In this case, class A must be in SDK main. I think many things fall into this category.

[kennknowles]

The main use we need to think about for the long term is the way that SDKs offer their own unit tests as test suites for runners. We may refactor but for now it makes sense for everything under sdks to offer the test jar.

[lukecwik]

looks good so far, several other pom files that could benefit from this as well

[lukecwik]

R: @lukecwik

[lukecwik]

Did you want to limit the scope of this change to what you have here or did you want to go through the other poms?

[kennknowles]

I'd like to get this in first. A number of these changes actually change it for everyone since they change parent pluginManagement. Submit & iterate.

[kennknowles]

One branch of the Travis run seems to have hung downloading deps.

[lukecwik]

LGTM, please self merge

[davorbonaci]

As far as I can see, this has reduced the build time from 15 minutes to 13 minutes (excluding outlays). Not a great win, IMO, with the given complexity. Perhaps we should chat separately if this is worth it.

[kennknowles]

We should just continue to find things we can omit. Certainly compile-and-test of 13 minutes is ridiculous for this project.

[kennknowles]

I didn't dig into too many modules, but mvn clean install -pl sdks/java/core went from ~3.5 minutes to ~2 minutes. To understand where the time goes, mvn test -pl sdks/java/core takes ~1.5 minutes.

So, taking all timings with a grain of salt, for the SDK this change reduced overall time by ~40% and reduced non-testing time by ~75%. I hope we can improve it further, and make similar improvements to all modules. We should also aim to reduce the testing time. This was a run with default settings, and it looked like parallel=none was set. Unless maven is inserting parallelism elsewhere, we should fix this.