fix tenant reconciliation

[eguzki]

Jira Issue: https://issues.redhat.com/browse/THREESCALE-8301

what

When a tenant is created declaratively, using the TenantCR, and afterwards it’s deleted from the 3scale admin portal UI, the operator starts creating tenants in an infinite loop.

how

The issue was caused by a wrong reconciliation logic that did not take into account the NOT FOUND response when reading an admin user. That returned an error, which turned up into a new reconciliation loop before the tenantID and AdminID was stored in the status section. Every new reconciliation loop ended up creatiing a new tenant.

The reconciliation loop logic has been refactored. Basically:

• Read tenant from status.TenantId
• If not found, create it and store in status new tenant ID together with 0 in the status.AdminId
• Read tenant user list looking for one user with Email and Username as specified in the tenant's spec.
• If not found any, create one
• reconcile the admin user -> make admin role and activated state in 3scale

Verification steps

Deploy 3scale with the APIManager

k apply -f - <<EOF
apiVersion: apps.3scale.net/v1alpha1
kind: APIManager
metadata:
  name: apimanager
spec:
  wildcardDomain: example.com
EOF

Wait for the deployment to be ready

oc wait --for=condition=available apimanager/apimanager --timeout=-1s

Deploy new tenant CR

k apply -f - <<EOF
apiVersion: capabilities.3scale.net/v1alpha1
kind: Tenant
metadata:
  name: ecorp-tenant
spec:
  username: admin
  systemMasterUrl: https://master.example.com
  email: admin@example.com
  organizationName: ECorp
  masterCredentialsRef:
    name: system-seed
  passwordCredentialsRef:
    name: ecorp-admin-secret
  tenantSecretRef:
    name: ecorp-tenant-secret
EOF

From the 3scale admin portal delete the tenant created (it will not be deleted, just scheduled for deletion)

Fully delete the tenant with the rails console:

open console

oc rsh -c system-master "$(oc get pods --selector deploymentconfig=system-app -o name)"
bundle exec rails console

delete with lines (takes few seconds)

tenant = Account.find(PROVIDER_ID)
# run several times
DeleteAccountHierarchyWorker.perform_later(tenant)

the tenant controller should not have notified the deletion of the tenant. We need to force tenant reconciliation for our tenant CR. Either stop and start the operator or update something in the TenantCR. I usually add harmless annotation

k annotate tenant ecorp-tenant newannotation=yes

The tenant controller should create (only) one new tenant with the CR spec.

[MStokluska]

👀

[MStokluska]

👀

[MStokluska]

Verified on the cluster, all looks good. Only one tenant is created afterward.

[MStokluska]

/lgtm
/approve

[MStokluska]

looks good, verified on cluster and the code looks good to me.

[qlty-cloud-legacy]

Code Climate has analyzed commit 8396019 and detected 4 issues on this pull request.

Here's the issue category breakdown:

Category	Count
Complexity	2
Duplication	2

View more on Code Climate.

[eguzki]

/retest