Inject hypervisor type and volume format on Quota tariffs #8138

winterhazel · 2023-10-24T17:23:18Z

Description

This PR adds two new preset variables into the Quota activation rules: hypervisor type and volume format. This allows operators to customize tariffs based on these two properties.

The respective preset variable will be injected into tariffs with the following usage types:

RUNNING_VM: value.hypervisorType;
ALLOCATED_VM: value.hypervisorType;
VM_SNAPSHOT: value.hypervisorType;
SNAPSHOT: value.hypervisorType;
VOLUME: value.volumeFormat;
VOLUME_SECONDARY: value.volumeFormat.

Example of an activation rule using value.hypervisorType:

if (value.hypervisorType=='KVM') {
  1
} else {
  2
}

Example of an activation rule using value.volumeFormat:

if (value.volumeFormat=='QCOW2') {
  3
} else {
  4
}

Furthermore, this PR also fixes the preset variable resourceType being injected into the JavaScript interpreter without quotes, which prevented the activation rules from executing.

2023-10-05 16:43:08,886 ERROR [o.a.c.q.QuotaManagerImpl] (qtp1955920234-22:ctx-d90594cb ctx-6d904941) (logid:5f086162) Failed to calculate the quota usage for account [{"accountName":"fabricio","domainId":1,"id":21,"uuid":"ae254524-854a-4049-9d1b-721cfaffdd71"}] due to [Unable to execute script [ account = {"role":{"type":"User","id":"64b4ca77-26de-11ec-8dcf-5254005dcdac","name":"User"},"id":"ae254524-854a-4049-9d1b-721cfaffdd71","name":"fabricio"}; domain = {"path":"\/","id":"52d83793-26de-11ec-8dcf-5254005dcdac","name":"ROOT"}; resourceType = KVM; value = {"accountResources":[{"domainId":"52d83793-26de-11ec-8dcf-5254005dcdac","zoneId":"8b2ceb16-a2f2-40ea-8968-9e08984bdb17"},{"domainId":"52d83793-26de-11ec-8dcf-5254005dcdac","zoneId":"8b2ceb16-a2f2-40ea-8968-9e08984bdb17"},{"domainId":"52d83793-26de-11ec-8dcf-5254005dcdac","zoneId":"8b2ceb16-a2f2-40ea-8968-9e08984bdb17"}],"computeOffering":{"customized":false,"id":"ab647165-7a0a-4984-8452-7bfceb036528","name":"Small Instance"},"computingResources":{"cpuNumber":1,"cpuSpeed":500,"memory":512},"host":{"tags":[],"id":"3d7d8532-d0cf-476c-a36e-1b936d780abb","name":"cloudstack-lab-host-2"},"hypervisorType":"KVM","osName":"Ubuntu 18.04 LTS","tags":{},"template":{"id":"bb4b7b6f-5249-4c18-a5f6-dd886683b702","name":"Ubuntu Bionic"},"id":"a83ece0c-58e7-4bc5-8564-f70fe8e34fc0","name":"t2"}; zone = {"id":"8b2ceb16-a2f2-40ea-8968-9e08984bdb17","name":"sc-floripa-01"}; if (value.hypervisorType=='KVM') { 1 } else { 2 }] due to [javax.script.ScriptException: ReferenceError: "KVM" is not defined in <eval> at line number 1]].

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

I created two tariffs through CloudMonkey:

(admin) 🐱 > quota tariffcreate activationrule="if (value.hypervisorType=='KVM') { 1 } else { 2 }" usagetype=1 value=0 name="hypervisor" description="hypervisor"
{
  "quotatariff": {
    "activationRule": "if (value.hypervisorType=='KVM') { 1 } else { 2 }",
    "currency": "$",
    "description": "hypervisor",
    "effectiveDate": "2023-10-04T21:46:19+0000",
    "name": "hypervisor",
    "tariffValue": 0,
    "usageDiscriminator": "None",
    "usageName": "RUNNING_VM",
    "usageType": 1,
    "usageTypeDescription": "Running Vm Usage",
    "usageUnit": "Compute*Month",
    "uuid": "85941d4b-cc6e-4e18-ba85-7708ea3a41eb"
  }
}


(admin) 🐱 > quota tariffcreate activationrule="if (value.volumeFormat=='QCOW2') { 3 } else { 4 }" usagetype=6 value=0 name="volume format" description="volume format"
{
  "quotatariff": {
    "activationRule": "if (value.volumeFormat=='QCOW2') { 3 } else { 4 }",
    "currency": "$",
    "description": "volume format",
    "effectiveDate": "2023-10-04T21:48:42+0000",
    "name": "volume format",
    "tariffValue": 0,
    "usageDiscriminator": "None",
    "usageName": "VOLUME",
    "usageType": 6,
    "usageTypeDescription": "Volume Usage",
    "usageUnit": "GB*Month",
    "uuid": "7b6c9eae-b369-40b7-adc3-0add4abf27bd"
  }
}

Then, I deployed a virtual machine using KVM, called quotaUpdate and verified through the logs that that the calculated value was 1 for the first tariff and 3 for the second.

DaanHoogland

clgtm

DaanHoogland · 2023-10-25T11:08:50Z

@blueorangutan package

blueorangutan · 2023-10-25T11:14:03Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

codecov · 2023-10-25T11:21:10Z

Codecov Report

Merging #8138 (3af5b4c) into main (543c54c) will increase coverage by 0.93%.
Report is 4 commits behind head on main.
The diff coverage is 57.14%.

@@             Coverage Diff              @@
##               main    #8138      +/-   ##
============================================
+ Coverage     28.15%   29.09%   +0.93%     
- Complexity    29181    30539    +1358     
============================================
  Files          5111     5111              
  Lines        360669   360696      +27     
  Branches      52700    52704       +4     
============================================
+ Hits         101562   104935    +3373     
+ Misses       245113   241331    -3782     
- Partials      13994    14430     +436

Flag	Coverage Δ
simulator-marvin-tests	`25.03% <0.00%> (+1.17%)`	⬆️
uitests	`4.79% <ø> (ø)`
unit-tests	`14.52% <76.19%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
.../org/apache/cloudstack/quota/QuotaManagerImpl.java	`50.00% <100.00%> (ø)`
...ck/quota/activationrule/presetvariables/Value.java	`100.00% <100.00%> (ø)`
...tionrule/presetvariables/PresetVariableHelper.java	`96.63% <58.33%> (-1.62%)`	⬇️
.../cloudstack/utils/jsinterpreter/JsInterpreter.java	`0.00% <0.00%> (ø)`

... and 252 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

GutoVeronezi

CLGTM

DaanHoogland · 2023-10-25T12:26:03Z

@blueorangutan test

blueorangutan · 2023-10-25T12:28:03Z

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

shwstppr

Code LGTM

blueorangutan · 2023-10-26T07:28:01Z

[SF] Trillian test result (tid-8095)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 45061 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8138-t8095-kvm-centos7.zip
Smoke tests completed. 111 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_02_list_cpvm_vm	`Failure`	0.05	test_ssvm.py
test_04_cpvm_internals	`Failure`	0.03	test_ssvm.py
test_12_destroy_cpvm	`Error`	3.15	test_ssvm.py
test_08_upgrade_kubernetes_ha_cluster	`Failure`	677.58	test_kubernetes_clusters.py

DaanHoogland · 2023-10-27T08:00:16Z

the gha failure is codecov upload. I don't think the ssvm errors are related. @shwstppr @GutoVeronezi Are we requiring 3rd party testing on this?

GutoVeronezi · 2023-10-27T17:02:27Z

the gha failure is codecov upload. I don't think the ssvm errors are related. @shwstppr @GutoVeronezi Are we requiring 3rd party testing on this?

It would be interesting to have 3rd party testing.

DaanHoogland · 2023-10-31T15:37:34Z

the gha failure is codecov upload. I don't think the ssvm errors are related. @shwstppr @GutoVeronezi Are we requiring 3rd party testing on this?

It would be interesting to have 3rd party testing.

I am not using quota, do you have a suggestion for who could test this, @GutoVeronezi ?

GutoVeronezi · 2023-10-31T21:08:55Z

the gha failure is codecov upload. I don't think the ssvm errors are related. @shwstppr @GutoVeronezi Are we requiring 3rd party testing on this?

It would be interesting to have 3rd party testing.

I am not using quota, do you have a suggestion for who could test this, @GutoVeronezi ?

@DaanHoogland, maybe @hsato03 could test it.

DaanHoogland · 2023-11-01T09:32:57Z

I am not using quota, do you have a suggestion for who could test this, @GutoVeronezi ?

@DaanHoogland, maybe @hsato03 could test it.

thanks guys

hsato03

I am sorry I could not test it before, guys.

I created an instance from a QCOW2 template using KVM and took a snapshot of it. Next, I created tariffs for RUNNING_VM, VOLUME and SNAPSHOT usage types and set the following activation rules:

RUNNING_VM

if (value.hypervisorType=='KVM') {
    333
} else {
    0
}

VOLUME

if (value.volumeFormat=='QCOW2') {
    444
} else {
    0
}

SNAPSHOT

if (value.hypervisorType=='KVM') {
    555
} else {
    0
}

Then, I called the quotaUpdate API via CloudMonkey and checked that the script result was 333 for the for the RUNNING_VM tariff, 444 for the VOLUME tariff and 555 for the SNAPSHOT tariff. I also verified that the exception thrown while injecting the resourceType preset variable was fixed.

I don't think it is necessary but it would be interesting to have these new preset variables in TEMPLATE usage type tariffs as well.

Inject hypervisor type and volume format on Quota tariffs

3af5b4c

boring-cyborg bot added the component:quota label Oct 24, 2023

DaanHoogland approved these changes Oct 25, 2023

View reviewed changes

DaanHoogland requested review from GutoVeronezi and shwstppr October 25, 2023 11:08

DaanHoogland added this to the 4.19.0.0 milestone Oct 25, 2023

GutoVeronezi approved these changes Oct 25, 2023

View reviewed changes

shwstppr approved these changes Oct 26, 2023

View reviewed changes

hsato03 approved these changes Nov 13, 2023

View reviewed changes

DaanHoogland merged commit df4cd2a into apache:main Nov 14, 2023

dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Nov 16, 2023

Inject hypervisor type and volume format on Quota tariffs (apache#8138)

82de527

Inject hypervisor type and volume format on Quota tariffs #8138

Inject hypervisor type and volume format on Quota tariffs #8138

Uh oh!

Conversation

winterhazel commented Oct 24, 2023

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland commented Oct 25, 2023

Uh oh!

blueorangutan commented Oct 25, 2023

Uh oh!

codecov bot commented Oct 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

GutoVeronezi left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland commented Oct 25, 2023

Uh oh!

blueorangutan commented Oct 25, 2023

Uh oh!

shwstppr left a comment

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Oct 26, 2023

Uh oh!

DaanHoogland commented Oct 27, 2023

Uh oh!

GutoVeronezi commented Oct 27, 2023

Uh oh!

DaanHoogland commented Oct 31, 2023

Uh oh!

GutoVeronezi commented Oct 31, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DaanHoogland commented Nov 1, 2023

Uh oh!

hsato03 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

codecov bot commented Oct 25, 2023 •

edited

Loading

GutoVeronezi commented Oct 31, 2023 •

edited

Loading

hsato03 left a comment •

edited

Loading