METRON-2094 Create CentOS 7 Development Environment

[nickwallen]

This creates another development environment for CentOS 7. This work is a prerequisite for supporting HDP 3.1. This co-exists with the other existing environments. After some period of time, I see this replacing the CentOS 6 environment.

Known Issues

There is currently a bug with the Ubuntu 14 environment in apache/master. I have created METRON-2095 to track the problem. This prevents me from fully testing the Ubuntu environment. I have done what I can in this PR to ensure that the Ubuntu environment will continue to work once that bug is addressed.

Testing

1. Launch the CentOS 7 development environment.

   • Ensure alerts are visible in the Alerts UI
   • Ensure that the Metron Service Check completes successfully.

   cd metron-deployment/development/centos7
   vagrant up
   

2. Launch the CentOS 6 development environment.

   • Ensure alerts are visible in the Alerts UI
   • Ensure that the Metron Service Check completes successfully.

   cd metron-deployment/development/centos6
   vagrant up
   

3. Launch the Ubuntu 14 development environment.

   • The deployment will not complete successfully as you will encounter METRON-2095.

   cd metron-deployment/development/ubuntu14
   vagrant up
   

Pull Request Checklist

• Is there a JIRA ticket associated with this PR? If not one needs to be created at Metron Jira.
• Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
• Has your PR been rebased against the latest commit within the target branch (typically master)?
• Have you included steps to reproduce the behavior or problem that is being changed or addressed?
• Have you included steps or a guide to how the change may be verified and tested manually?
• Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
• Have you written or updated unit tests and or integration tests to verify your changes?
• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?

[ottobackwards]

I was unable to complete vagrant up, it froze at deploying cluster for hours.

[ottobackwards]

Any ideas?

[nickwallen]

@ottobackwards I have never seen that. Be sure to "vagrant destroy -f" in the directory, and even a rm -rf .vagrant for good measure, so you are starting with a completely fresh image.

What directory did you run that in; centos6 or centos7?

Does the same error happen for both 6 and 7 or just one of them?

And what box images are you using? I am running the following, but I may need to update. The command will show all versions that you have, but when you vagrant up it uses the latest version.

$ vagrant box list
...
centos/6           (virtualbox, 1804.02)
centos/7           (virtualbox, 1708.01)

[ottobackwards]

I ran from centos7 directory, in a fresh checkout with checkout-pr script.
Honestly, have leaving it for hours to complete, i didn't run centos6 to check, having just run that for the RC.

vagrant box list
bento/centos-6.7   (virtualbox, 2.2.7)
bento/centos-7.1   (virtualbox, 2.2.2)
bento/centos-7.3   (virtualbox, 2.3.5)
bento/centos-7.3   (virtualbox, 201708.22.0)
bento/ubuntu-14.04 (virtualbox, 2.3.5)
centos/6           (virtualbox, 1802.01)
centos/7           (virtualbox, 1902.01)
metron-hdp/base    (virtualbox, 0)
metron/centos_base (virtualbox, 6.7.1.0)
metron/hdp-base    (virtualbox, 0.0.3)
metron/quick_dev   (virtualbox, 0.1.0)
ubuntu/trusty64    (virtualbox, 20180122.0.2)

[ottobackwards]

I removed all my vagrant boxes, I'll run vagrant up again and see

[ottobackwards]

Still getting pending host assignment

[ottobackwards]

platform_info.txt

[nickwallen]

Would you be able to attach your ansible.log and your /var/log/ambari-server/ambari-server.log and /var/log/ambari-agent/ambari-agent.log from the VM?

I have updated my VM images and used the prepare-commit script to go through another round of tests to see if I can replicate your results.

I have run this up what feels like a million times without problems, so I am trying to close the gap on what is different between our environments. Thanks @ottobackwards

[ottobackwards]

Sure, I'll do a clean run, I already did a vagrant delete

[nickwallen]

Ok thanks. Also, try using prepare-commit rather than checkout-pr. The former will merge it with master and so the test run will be more like what the code would be like with this PR merged into master. That being said, I really doubt that is the cause of the problem, but just trying to reduce the number of variables here.

[nickwallen]

I was able to replicate on centos7 after updating my box images and using prepare-commit. So that's good news. I'll see what's going on.

[nickwallen]

The issue is that after the image update for centos/7 to 1902.01, I now need to force the Ambari Agent to use TLS v1.2. I am running up a fix for centos7 now.

• https://community.hortonworks.com/questions/134759/disabling-tlsv1-tls11-enabling-tlsv12.html

The centos6 build was not impacted by this.

[nickwallen]

I am also noticing that the swap space is not being set correctly in the latest CentOS 7 image. Daft

[ottobackwards]

Should I wait, or try current

[nickwallen]

Hold up until I get this swap space issue fixed. If you don't give it enough swap, services just die randomly, which is fun, but not something I want to put you through.

[ottobackwards]

Maybe that is why HBASE is having issues

[nickwallen]

The new CentOS 7 image comes with 2G of swap enabled. Our logic only enables swap, if it is off.

I needed to change the logic to resize the swap size because we need about 8G (1:1 with memory) for the development environment to behave reasonably.

[nickwallen]

I have fixed all the problems that were caused by the updates in the latest image of centos7; 1902.01. I have successfully re-tested all of the latest changes against the latest images for both 6 and 7.

[ottobackwards]

I'll re-test this morning

[ottobackwards]

+1, ran up both environments, checked all the ui and status.

[mmiklavc]

lgtm @nickwallen. Between your and @ottobackwards 's manual validation, I'm +1 via inspection.

[nickwallen]

Thanks for the reviews