[BEAM-3217] Jenkins job for HadoopInputFormatIOIT

[lgajowy]

This PR adds jenkins jobs for HadoopInputFormatIOIT.

Before this PR there was only one test that used postgres kubernetes scripts (JDBC IOIT). Right now, while having more than one test using those, there is a strong possiblility that two tests will try to create kubernetes resources (eg. service or replication controller) of the same name (say, "postgres"). This will result in one test being unable to run successfully (due to names collision).

This PR proposes a solution to above problem. Every kubernetes test resource is being created in it's own namespace. This avoids the name collisions and allows running them "in isolation".

This solution has some other (minor) advantages:

• if a Jenkins job gets killed and doesn't clean up it's Kubernetes resources on the cluster we don't have to manually delete them to be able to run other test (now the test fails due to name collisions with the "dangling" resources that are left there)
• we can easily clean up the cluster with one command. To delete every resource (service, rc, pods etc) it's sufficient to delete the namespace only (kubectl delete namespace foo) it deletes it's "inner resources" too.

Remarks:

• Port collisions are still possible even when using namespaces. If two services from different namespaces want bind to the same port we get an error from Kubernetes ("port is already allocated"). This doesn't happen while using load balancer service so it's not a problem for our IOIT tests now.

---

Follow this checklist to help us incorporate your contribution quickly and easily:

• Make sure there is a JIRA issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes.
• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue.
• Write a pull request description that is detailed enough to understand:
  • What the pull request does
  • Why it does it
  • How it does it
  • Why this approach
• Each commit in the pull request should have a meaningful subject line and body.
• Run mvn clean verify to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

[lgajowy]

Run seed job

[lgajowy]

Run Java HadoopInputFormatIO Performance Test

[lgajowy]

Run seed job

[lgajowy]

Run seed job

[lgajowy]

Run Java HadoopInputFormatIO Performance Test

[lgajowy]

Run seed job

[lgajowy]

Run Java HadoopInputFormatIO Performance Test

[lgajowy]

Run seed job

[lgajowy]

Run Java HadoopInputFormatIO Performance Test

[lgajowy]

@chamikaramj sorry for the above mess in comments - I had to run this on Jenkins properly. Now it's ready for review. Could you please take a look?

[lgajowy]

Run Java JdbcIO Performance Test

[lgajowy]

Run seed job

[lgajowy]

Run Java HadoopInputFormatIO Performance Test

[lgajowy]

Run Java JdbcIO Performance Test

[lgajowy]

I was able to confirm that HadoopInputFormat Performance test will fail due to dataflow runner issue as well. See the issue and [the logs].(https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/7/console)

@chamikaramj could you take a look either way?

[lgajowy]

Run seed job

[lgajowy]

Run Java JdbcIO Performance Test

[chamikaramj]

Run Java HadoopInputFormatIO Performance Test

[chamikaramj]

Run seed job

[lgajowy]

Retest this please

[chamikaramj]

LGTM. Merged.