feat: Support ARM endpoint with non-trusted certificate

[honcao]

Reason for Change:

feat: Support ARM endpoint with non-trusted certificate

Issue Fixed:

Fixes #280

Requirements:

• uses conventional commit messages
• includes documentation
• adds unit tests
• tested upgrade from previous version

Notes:
Customer will provide the certificate to binding with external endpoint. e.g ARM endpoint.

The certificate could be non-trusted on the Linux Machine since customer could use cooperated signed certificate to deploy azure stack.

Right now, the k8s controller will be not be able to communicate back to ARM endpoint with non-trusted certificate.

AKS engine would need trusted the certificate and mount the certificate to the k8s controllers container.

[codecov]

Codecov Report

Merging #553 into master will increase coverage by 0.03%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #553      +/-   ##
==========================================
+ Coverage   54.75%   54.79%   +0.03%     
==========================================
  Files          97       97              
  Lines       14708    14716       +8     
==========================================
+ Hits         8054     8064      +10     
+ Misses       5980     5979       -1     
+ Partials      674      673       -1

[jackfrancis]

@honcao, let's do this, to avoid adding add'l ARM deployment payload unnecessarily to Azure Stack-only scenarios. See:

- path: /opt/azure/containers/provision_source.sh
  permissions: "0744"
  encoding: gzip
  owner: root
  content: !!binary |
    {{WrapAsVariable "provisionSource"}}

☝️ in the kubernetesagentcustomdata.yml file (cloud-init config). Basically:

1. Let's move all Azure Stack-specific shell functionality into a new shell script
2. Conditionally add it to the cloud-init configuration (the appropriate *customdata.yml files under parts/)

3. Conditionally source this new file (or files) into the kubernetescustomscript.sh implementation if the provision is operating in an Azure Stack context

Does the above make sense?

[mboersma]

@honcao I rebased and force-pushed this PR to reconcile it with #546 in master. It's no longer necessary to commit the generated files. Sorry about that!

[honcao]

@jackfrancis , I will make the changes to implement your proposal

[jackfrancis]

Are we sure we need this custom controller manager scaffolding on agent nodes? For regular (non-Azure Stack) scenarios, the controller manager implementation is only on master nodes.

[honcao]

It is also true that controller manager implementation is only on master nodes.
I updated ensureCertificates to check the existence kube controller file before update it.
The logic now are:

• on Master trust the certificate and update the kube controller config

• on Agent, only trust the certificate

[jackfrancis]

I don't see anything that will properly inject a value into <img>

[honcao]

src/github.com/Azure/aks-engine/parts/k8s/kubernetesmastercustomdata.yml
line 308 to 311
/opt/azure/containers/kubelet.sh will replace the

[jackfrancis]

Got it, you are re-using the kube-controller-manager.yaml filename for Azure Stack scenarios.

[jackfrancis]

I don't see anything that will properly inject a value into <args>

[honcao]

src/github.com/Azure/aks-engine/parts/k8s/kubernetesmastercustomdata.yml
line 319
/opt/azure/containers/kubelet.sh will replace the

[jackfrancis]

/lgtm

[acs-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: honcao, jackfrancis

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [jackfrancis]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment