CPVM: use X509ExtendedTrustManager to skip hostname verification #5419

weizhouapache · 2021-09-08T09:01:14Z

Description

This PR fixes #5413

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

weizhouapache · 2021-09-08T09:27:44Z

@blueorangutan package

blueorangutan · 2021-09-08T09:28:03Z

@weizhouapache a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-09-08T10:18:24Z

Packaging result: ✖️ el7 ✖️ el8 ✔️ debian ✔️ suse15. SL-JID 1175

weizhouapache · 2021-09-08T10:20:49Z

Hi @coreymr

can you please test with the following file in CPVM ?
cloud-console-proxy-4.15.1.0.zip

cd /usr/local/cloud/systemvm/
mv /usr/local/cloud/systemvm/cloud-console-proxy-4.15.1.0.jar /root/
wget https://github.com/apache/cloudstack/files/7128023/cloud-console-proxy-4.15.1.0.zip
mv cloud-console-proxy-4.15.1.0.zip cloud-console-proxy-4.15.1.0.jar
systemctl restart cloud

weizhouapache · 2021-09-08T10:21:02Z

@blueorangutan package

blueorangutan · 2021-09-08T10:22:03Z

@weizhouapache a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-09-08T10:43:18Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 1177

weizhouapache · 2021-09-08T12:45:29Z

@rhtyd @nvazquez
we maybe need this pr in 4.15.2.0

Mike has confirmed it fixes his console issue on vmware 7.

nvazquez

LGTM

nvazquez · 2021-09-08T12:50:16Z

Thanks @weizhouapache, @rhtyd can this be included on 4.15.2?

nvazquez · 2021-09-08T14:02:44Z

@blueorangutan package

blueorangutan · 2021-09-08T14:03:03Z

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-09-08T14:27:17Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 1184

nvazquez · 2021-09-08T14:27:50Z

@blueorangutan test centos7 vmware-70u1

blueorangutan · 2021-09-08T14:29:03Z

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + vmware-70u1) has been kicked to run smoke tests

nvazquez · 2021-09-08T15:56:46Z

@rhtyd @weizhouapache I think this is not a blocker, no need to include it on 4.15.2, do you agree?

blueorangutan · 2021-09-09T04:26:49Z

Trillian test result (tid-1992)
Environment: vmware-70u1 (x2), Advanced Networking with Mgmt server 7
Total time taken: 48769 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5419-t1992-vmware-70u1.zip
Smoke tests completed. 76 look OK, 11 have errors
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_create_lb_rule_src_nat	`Failure`	44.70	test_loadbalance.py
test_02_create_lb_rule_non_nat	`Failure`	34.30	test_loadbalance.py
test_assign_and_removal_lb	`Failure`	34.26	test_loadbalance.py
test_isolate_network_password_server	`Failure`	7.56	test_password_server.py
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80	`Failure`	259.56	test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80	`Failure`	328.93	test_internal_lb.py
test_02_vpc_privategw_static_routes	`Failure`	239.72	test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup	`Failure`	237.90	test_privategw_acl.py
test_04_rvpc_privategw_static_routes	`Failure`	390.03	test_privategw_acl.py
test_router_dhcphosts	`Failure`	7.56	test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPHosts>:teardown	`Error`	18.97	test_router_dhcphosts.py
test_router_dns_guestipquery	`Failure`	3.37	test_router_dnsservice.py
test_01_isolate_network_FW_PF_default_routes_egress_true	`Failure`	94.90	test_routers_network_ops.py
test_02_isolate_network_FW_PF_default_routes_egress_false	`Failure`	97.78	test_routers_network_ops.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true	`Failure`	143.43	test_routers_network_ops.py
test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false	`Failure`	147.34	test_routers_network_ops.py
test_04_change_offering_small	`Error`	147.80	test_service_offerings.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	`Failure`	563.56	test_vpc_redundant.py
test_02_redundant_VPC_default_routes	`Failure`	421.74	test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	`Failure`	520.69	test_vpc_redundant.py
test_02_VPC_default_routes	`Failure`	189.92	test_vpc_router_nics.py
test_01_redundant_vpc_site2site_vpn	`Error`	398.48	test_vpc_vpn.py
test_01_vpc_site2site_vpn_multiple_options	`Error`	263.16	test_vpc_vpn.py
test_01_vpc_site2site_vpn	`Error`	276.26	test_vpc_vpn.py

nvazquez · 2021-09-11T02:20:26Z

@blueorangutan test centos7 vmware-70u1

blueorangutan · 2021-09-11T02:22:03Z

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + vmware-70u1) has been kicked to run smoke tests

blueorangutan · 2021-09-14T04:02:10Z

Trillian test result (tid-2016)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 47889 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5419-t2016-xenserver-71.zip
Smoke tests completed. 88 look OK, 1 have errors
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_07_deploy_kubernetes_ha_cluster	`Failure`	85.99	test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown	`Error`	111.13	test_kubernetes_clusters.py

yadvr · 2021-09-14T05:52:01Z

cc @DaanHoogland @nvazquez @davidjumani - do we know what causes so many smoketest failures with Vmware7, can we fix them?

yadvr

LGTM - did not test it, this would require some manual testing + 100% smoketests pass before merging.

weizhouapache · 2021-09-14T11:13:21Z

@blueorangutan test centos7 vmware-65u2

blueorangutan · 2021-09-14T11:14:02Z

@weizhouapache a Trillian-Jenkins test job (centos7 mgmt + vmware-65u2) has been kicked to run smoke tests

nvazquez · 2021-09-14T15:06:07Z

@rhtyd @weizhouapache I've started a Vmware7 round of tests on the health check PR so we can compare if the failures are related to this PR or needs fixing

nvazquez · 2021-09-15T03:28:30Z

@rhtyd @weizhouapache this fix has been tested by @coreymr, do we need additional tests?

yadvr · 2021-09-15T09:21:54Z

@nvazquez yes my concern is not specifically VMware7, but other regression testing hypervisors/version combinations (with/without SSL enabled?) - if it breaks anything? Or do you or @weizhouapache think that's not necessary?

weizhouapache · 2021-09-15T09:34:02Z

@nvazquez yes my concern is not specifically VMware7, but other regression testing hypervisors/version combinations (with/without SSL enabled?) - if it breaks anything? Or do you or @weizhouapache think that's not necessary?

@rhtyd
I have tested with centos7 (ssl), ubuntu20 (non-ssl) and xenserver65 (non-ssl). no issues.

yadvr · 2021-09-15T09:40:42Z

Thanks for confirming @weizhouapache

yadvr · 2021-09-15T09:41:19Z

(just curious why XenServer65? and what about Vmware 6.5/6.7?)

weizhouapache · 2021-09-15T10:39:13Z

(just curious why XenServer65? and what about Vmware 6.5/6.7?)

@rhtyd sorry for typo.

centos7 (ssl), ubuntu20 (non-ssl), vmware65 (non-ssl) and xenserver71 (non-ssl).

weizhouapache · 2021-09-15T15:27:51Z

@blueorangutan test centos7 vmware-70u1 keepEnv

blueorangutan · 2021-09-15T15:29:03Z

@weizhouapache a Trillian-Jenkins test job (centos7 mgmt + vmware-70u1) has been kicked to run smoke tests

blueorangutan · 2021-09-16T05:52:50Z

Trillian test result (tid-2068)
Environment: vmware-70u1 (x2), Advanced Networking with Mgmt server 7
Total time taken: 50210 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5419-t2068-vmware-70u1.zip
Smoke tests completed. 77 look OK, 12 have errors
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80	`Failure`	317.27	test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80	`Failure`	430.32	test_internal_lb.py
test_01_create_lb_rule_src_nat	`Failure`	47.70	test_loadbalance.py
test_02_create_lb_rule_non_nat	`Failure`	37.43	test_loadbalance.py
test_assign_and_removal_lb	`Failure`	45.32	test_loadbalance.py
test_isolate_network_password_server	`Failure`	7.56	test_password_server.py
test_02_vpc_privategw_static_routes	`Failure`	364.22	test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup	`Failure`	349.69	test_privategw_acl.py
test_04_rvpc_privategw_static_routes	`Failure`	524.53	test_privategw_acl.py
test_router_dhcphosts	`Failure`	7.54	test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPHosts>:teardown	`Error`	17.82	test_router_dhcphosts.py
test_router_dns_guestipquery	`Failure`	3.41	test_router_dnsservice.py
test_01_isolate_network_FW_PF_default_routes_egress_true	`Failure`	125.98	test_routers_network_ops.py
test_02_isolate_network_FW_PF_default_routes_egress_false	`Failure`	129.80	test_routers_network_ops.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true	`Failure`	200.59	test_routers_network_ops.py
test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false	`Failure`	209.72	test_routers_network_ops.py
test_04_change_offering_small	`Error`	146.45	test_service_offerings.py
test_02_list_snapshots_with_removed_data_store	`Error`	55.10	test_snapshots.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	`Failure`	1079.67	test_vpc_redundant.py
test_02_redundant_VPC_default_routes	`Failure`	406.46	test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	`Failure`	504.67	test_vpc_redundant.py
test_02_VPC_default_routes	`Failure`	214.68	test_vpc_router_nics.py
test_01_redundant_vpc_site2site_vpn	`Error`	492.92	test_vpc_vpn.py
test_01_vpc_site2site_vpn_multiple_options	`Error`	329.31	test_vpc_vpn.py
test_01_vpc_site2site_vpn	`Error`	331.14	test_vpc_vpn.py

weizhouapache · 2021-09-16T10:53:20Z

@rhtyd @nvazquez
the test failures are caused by macchinina template we use for testing.

only the first ssh connection is ok, the next ssh attempts are refused. (it seems to be working in vmware65 environments)

root@r-463-VM:~# date;ssh 10.1.1.159
Thu 16 Sep 2021 09:55:47 AM UTC
The authenticity of host '10.1.1.159 (10.1.1.159)' can't be established.
ECDSA key fingerprint is SHA256:WQQZ6Sm2CxR5eDkbOw4Xwl+mTrE37fMmnR1tZigCQ8E.
Are you sure you want to continue connecting (yes/no/[fingerprint])? ^C

root@r-463-VM:~# date;ssh 10.1.1.159
Thu 16 Sep 2021 09:55:49 AM UTC
ssh: connect to host 10.1.1.159 port 22: Connection refused
root@r-463-VM:~# date;ssh 10.1.1.159
Thu 16 Sep 2021 09:55:52 AM UTC
ssh: connect to host 10.1.1.159 port 22: Connection refused

for default centos55 template, it works

root@r-463-VM:~# date;ssh -o KexAlgorithms=diffie-hellman-group1-sha1 10.1.1.120
Thu 16 Sep 2021 09:58:53 AM UTC
The authenticity of host '10.1.1.120 (10.1.1.120)' can't be established.
RSA key fingerprint is SHA256:gdLIogU+y021XvZfEGio/58rU5NrHOqHHLmA8T6z5G8.
Are you sure you want to continue connecting (yes/no/[fingerprint])? ^C
root@r-463-VM:~# 

root@r-463-VM:~# date;ssh -o KexAlgorithms=diffie-hellman-group1-sha1 10.1.1.120
Thu 16 Sep 2021 09:58:56 AM UTC
The authenticity of host '10.1.1.120 (10.1.1.120)' can't be established.
RSA key fingerprint is SHA256:gdLIogU+y021XvZfEGio/58rU5NrHOqHHLmA8T6z5G8.
Are you sure you want to continue connecting (yes/no/[fingerprint])? 
Host key verification failed.

When I use centos55 for testing, tests passed.

diff --git a/test/integration/smoke/test_loadbalance.py b/test/integration/smoke/test_loadbalance.py
index 53047f91f2..0a1b2ee5f4 100644
--- a/test/integration/smoke/test_loadbalance.py
+++ b/test/integration/smoke/test_loadbalance.py
@@ -41,7 +41,7 @@ class TestLoadBalance(cloudstackTestCase):
         cls.domain = get_domain(cls.apiclient)
         cls.zone = get_zone(cls.apiclient, testClient.getZoneForTests())
         cls.hypervisor = testClient.getHypervisorInfo()
-        template = get_test_template(
+        template = get_template(
                             cls.apiclient,
                             cls.zone.id,
                             cls.hypervisor)
diff --git a/test/integration/smoke/test_privategw_acl.py b/test/integration/smoke/test_privategw_acl.py
index 90596c6984..28cc0a3999 100644
--- a/test/integration/smoke/test_privategw_acl.py
+++ b/test/integration/smoke/test_privategw_acl.py
@@ -169,7 +169,7 @@ class TestPrivateGwACL(cloudstackTestCase):
         cls.zone = get_zone(cls.api_client, cls.testClient.getZoneForTests())
         cls.hypervisor = cls.testClient.getHypervisorInfo()
         cls.services['mode'] = cls.zone.networktype
-        cls.template = get_test_template(
+        cls.template = get_template(
             cls.api_client,
             cls.zone.id,
             cls.hypervisor)

nvazquez · 2021-09-16T19:25:10Z

@NuxRo any input on this strange behaviour with macchinina and Vmware 7?

sureshanaparti

code LGTM

weizhouapache · 2021-09-17T14:37:21Z

@NuxRo any input on this strange behaviour with macchinina and Vmware 7?

@NuxRo
if you have time, can you have a look at the macchinina template ?
on vmware7, ssh is unreachable after the first ssh attempt.
after executing "/usr/sbin/dropbear" in the vm, ssh works, but only once. it does not work again after a ssh attempt.
no issue on vmware65. it is strange.

weizhouapache · 2021-09-21T07:02:29Z

@nvazquez @sureshanaparti
can we merge this into 4.15 or master ? (4.15 might be better I think)

yadvr · 2021-09-21T07:21:57Z

@weizhouapache not before we have some agreement on the vmware failures (4.15 is fine, but I wonder if somebody will do a 4.15.3 as we'll soon have 4.16.0, 4.16.1 and 4.17.0 in the pipeline)

yadvr · 2021-09-21T07:22:31Z

ping @NuxRo @nvazquez @weizhouapache - kindly discuss and agree on whether this can be merged or some fixes in this PR or in trillian/lab env is needed.

weizhouapache · 2021-09-22T15:32:47Z

ping @NuxRo @nvazquez @weizhouapache - kindly discuss and agree on whether this can be merged or some fixes in this PR or in trillian/lab env is needed.

@rhtyd
the test failures are not related to this PR in my opinon.
but we can wait some days to see if macchinina template can be fixed on vmware7 @NuxRo
anyway, we should merge it before 4.16.0.0 RC. @nvazquez

nvazquez · 2021-09-22T18:46:48Z

Agree with @weizhouapache, the failures are not related to the PR as the same were seen in the health checks PR, lets merge it and fix the issues for macchinina and Vmware 7 on a separate PR

weizhouapache · 2021-09-23T07:20:09Z

For those who have same issue with 4.15.2.0, please use the jar below in SSVM
cloud-console-proxy-4.15.2.0.zip

cd /usr/local/cloud/systemvm/
mv /usr/local/cloud/systemvm/cloud-console-proxy-4.15.2.0.jar /root/
wget https://github.com/apache/cloudstack/files/7215623/cloud-console-proxy-4.15.2.0.zip
mv cloud-console-proxy-4.15.2.0.zip cloud-console-proxy-4.15.2.0.jar
systemctl restart cloud

CPVM: use X509ExtendedTrustManager

071ef49

weizhouapache changed the base branch from main to 4.15 September 8, 2021 09:29

weizhouapache marked this pull request as ready for review September 8, 2021 12:40

weizhouapache requested review from nvazquez and yadvr September 8, 2021 12:42

nvazquez approved these changes Sep 8, 2021

View reviewed changes

nvazquez mentioned this pull request Sep 8, 2021

[Vmware] Use host FQDN if host IP is not found on VNC command #5414

Closed

12 tasks

weizhouapache closed this Sep 8, 2021

weizhouapache reopened this Sep 8, 2021

weizhouapache mentioned this pull request Sep 8, 2021

Console Proxy & VMware 7 websocket issue #5413

Closed

weizhouapache changed the title ~~CPVM: use X509ExtendedTrustManager~~ CPVM: use X509ExtendedTrustManager to skip hostname verification Sep 10, 2021

yadvr added this to the 4.16.0.0 milestone Sep 10, 2021

yadvr added the status:needs-testing label Sep 14, 2021

yadvr approved these changes Sep 14, 2021

View reviewed changes

sureshanaparti approved these changes Sep 16, 2021

View reviewed changes

nvazquez merged commit 50a0e80 into apache:main Sep 22, 2021

DaanHoogland pushed a commit to shapeblue/cloudstack that referenced this pull request May 20, 2022

CPVM: use X509ExtendedTrustManager (apache#5419)

574c3af

weizhouapache deleted the 4.15-cpvm-X509ExtendedTrustManager branch December 9, 2022 08:45

shwstppr pushed a commit to shapeblue/cloudstack that referenced this pull request Jan 17, 2023

CPVM: use X509ExtendedTrustManager (apache#5419)

d317b02

CPVM: use X509ExtendedTrustManager to skip hostname verification #5419

CPVM: use X509ExtendedTrustManager to skip hostname verification #5419

Uh oh!

Conversation

weizhouapache commented Sep 8, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

weizhouapache commented Sep 8, 2021

Uh oh!

blueorangutan commented Sep 8, 2021

Uh oh!

blueorangutan commented Sep 8, 2021

Uh oh!

weizhouapache commented Sep 8, 2021

Uh oh!

weizhouapache commented Sep 8, 2021

Uh oh!

blueorangutan commented Sep 8, 2021

Uh oh!

blueorangutan commented Sep 8, 2021

Uh oh!

weizhouapache commented Sep 8, 2021

Uh oh!

nvazquez left a comment

Choose a reason for hiding this comment

Uh oh!

nvazquez commented Sep 8, 2021

Uh oh!

nvazquez commented Sep 8, 2021

Uh oh!

blueorangutan commented Sep 8, 2021

Uh oh!

blueorangutan commented Sep 8, 2021

Uh oh!

nvazquez commented Sep 8, 2021

Uh oh!

blueorangutan commented Sep 8, 2021

Uh oh!

nvazquez commented Sep 8, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

blueorangutan commented Sep 9, 2021

Uh oh!

nvazquez commented Sep 11, 2021

Uh oh!

blueorangutan commented Sep 11, 2021

Uh oh!

blueorangutan commented Sep 14, 2021

Uh oh!

yadvr commented Sep 14, 2021

Uh oh!

yadvr left a comment

Choose a reason for hiding this comment

Uh oh!

weizhouapache commented Sep 14, 2021

Uh oh!

blueorangutan commented Sep 14, 2021

Uh oh!

nvazquez commented Sep 14, 2021

Uh oh!

nvazquez commented Sep 15, 2021

Uh oh!

yadvr commented Sep 15, 2021

Uh oh!

weizhouapache commented Sep 15, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yadvr commented Sep 15, 2021

Uh oh!

yadvr commented Sep 15, 2021

Uh oh!

weizhouapache commented Sep 15, 2021

Uh oh!

weizhouapache commented Sep 8, 2021 •

edited

Loading

nvazquez commented Sep 8, 2021 •

edited

Loading

weizhouapache commented Sep 15, 2021 •

edited

Loading

weizhouapache commented Sep 17, 2021 •

edited

Loading