Skip to content

feat(vm): memory hotplug (phase 1)#2110

Merged
diafour merged 2 commits into
mainfrom
feat/vm/hotplug-memory
Apr 10, 2026
Merged

feat(vm): memory hotplug (phase 1)#2110
diafour merged 2 commits into
mainfrom
feat/vm/hotplug-memory

Conversation

@diafour
Copy link
Copy Markdown
Member

@diafour diafour commented Mar 16, 2026 •
edited by Isteb4k
Loading

Description

  • Change memory setting from domain.resources to domain.memory.guest.
  • Support old VMs to not require reboot on module update.

This PR is a base for #2147.

Why do we need it, and what problem does it solve?

It is a phase 1 of implementation of the memory hotplug support.

Limitations:

  • No VMClass settings, only spec.memory.size in the VM is supported for now.
  • maxGuest is hardcoded to 256Gi (we choose conservative 38 bit physical address).
  • Memory reducing forced to require reboot.
  • Memory hotplug is disabled by default, add a feature gate to ModuleConfig to enable: spec.settings.featureGates array should contain string HotplugMemoryWithLiveMigration.

What is the expected result?

VM is migrating and memory is changed when spec.memory field is changed.

384Mi -> 1384Mi (Crossing 1Gi threshold upward will enable hotplug and requires a reboot)

Restart initiated 

$ k -n vm get internalvirtualizationvirtualmachines.internal.virtualization.deckhouse.io vm-alpine-uefi -oyaml | grep ' memory:' -A 10
        memory:
          guest: 384Mi

$ k -n vm get all,po,intvirt,vmop -owide

NAME                                  READY   STATUS        RESTARTS   AGE    IP            NODE           NOMINATED NODE   READINESS GATES

pod/d8v-vm-vm-alpine-uefi-6tfh8       1/1     Terminating   0          5m6s   10.66.10.4    virtlab-mi-1              1/1


NAME                                                              PHASE     CORES   COREFRACTION   MEMORY   NEED RESTART   AGENT   MIGRATABLE   NODE           IPADDRESS    AGE

virtualmachine.virtualization.deckhouse.io/vm-alpine-uefi         Stopped   1       20%            384Mi                           True         virtlab-mi-1   10.66.10.4   10d


NAME                                                                                               AGE    PHASE       IP    NODENAME       READY   LIVE-MIGRATABLE   PAUSED

internalvirtualizationvirtualmachineinstance.internal.virtualization.deckhouse.io/vm-alpine-uefi   5m6s   Succeeded         virtlab-mi-1   False   True


NAME                                                                                             AGE   STATUS    READY

internalvirtualizationvirtualmachine.internal.virtualization.deckhouse.io/vm-alpine-uefi         10d   Stopped   False

After restart 

$ k -n vm get internalvirtualizationvirtualmachines.internal.virtualization.deckhouse.io vm-alpine-uefi -oyaml | grep ' memory:' -A 10
        memory:
          guest: 1384Mi
          maxGuest: 256Gi

$ k -n vm get all,po,intvirt,vmop -owide

NAME                                  READY   STATUS    RESTARTS   AGE   IP            NODE           NOMINATED NODE   READINESS GATES

pod/d8v-vm-vm-alpine-uefi-npxkw       1/1     Running   0          54s   10.66.10.4    virtlab-mi-2              1/1


NAME                                                              PHASE     CORES   COREFRACTION   MEMORY   NEED RESTART   AGENT   MIGRATABLE   NODE           IPADDRESS    AGE

virtualmachine.virtualization.deckhouse.io/vm-alpine-uefi         Running   1       20%            1384Mi                  True    True         virtlab-mi-2   10.66.10.4   10d


NAME                                                                                               AGE   PHASE     IP           NODENAME       READY   LIVE-MIGRATABLE   PAUSED

internalvirtualizationvirtualmachineinstance.internal.virtualization.deckhouse.io/vm-alpine-uefi   54s   Running   10.66.10.4   virtlab-mi-2   True    True


NAME                                                                                             AGE   STATUS    READY

internalvirtualizationvirtualmachine.internal.virtualization.deckhouse.io/vm-alpine-uefi         10d   Running   True

1384Mi -> 2384Mi (Hotplug enabled: add memory with live migration)

Migration initiated 

$ k -n vm get internalvirtualizationvirtualmachines.internal.virtualization.deckhouse.io vm-alpine-uefi -oyaml | grep ' memory:' -A 10
        memory:
          guest: 2384Mi
          maxGuest: 256Gi

$ k -n vm get all,po,intvirt,vmop -owide


NAME                                  READY   STATUS              RESTARTS   AGE    IP            NODE           NOMINATED NODE   READINESS GATES

pod/d8v-vm-vm-alpine-uefi-s2cgs       1/1     Running             0          117m   10.66.10.4    virtlab-mi-1              1/1

pod/d8v-vm-vm-alpine-uefi-wnptq       0/1     ContainerCreating   0          6s             virtlab-mi-3              0/1


NAME                                                              PHASE     CORES   COREFRACTION   MEMORY   NEED RESTART   AGENT   MIGRATABLE   NODE           IPADDRESS    AGE

virtualmachine.virtualization.deckhouse.io/vm-alpine-uefi         Running   1       20%            2384Mi                  True    True         virtlab-mi-1   10.66.10.4   10d


NAME                                                                                               AGE    PHASE     IP           NODENAME       READY   LIVE-MIGRATABLE   PAUSED

internalvirtualizationvirtualmachineinstance.internal.virtualization.deckhouse.io/vm-alpine-uefi   117m   Running   10.66.10.4   virtlab-mi-1   True    True


NAME                                                                                             AGE   STATUS    READY

internalvirtualizationvirtualmachine.internal.virtualization.deckhouse.io/vm-alpine-uefi         10d   Running   True


NAME                                                                          PHASE        TYPE    VIRTUALMACHINE   AGE

virtualmachineoperation.virtualization.deckhouse.io/hotplug-resources-46c8z   InProgress   Evict   vm-alpine-uefi   6s


NAME                                                                                                                      PHASE        VMI

internalvirtualizationvirtualmachineinstancemigration.internal.virtualization.deckhouse.io/vmop-hotplug-resources-46c8z   Scheduling   vm-alpine-uefi

After migrating 

$ k -n vm get internalvirtualizationvirtualmachines.internal.virtualization.deckhouse.io vm-alpine-uefi -oyaml | grep ' memory:' -A 10
        memory:
          guest: 2384Mi
          maxGuest: 256Gi

NAME                                  READY   STATUS      RESTARTS   AGE     IP            NODE           NOMINATED NODE   READINESS GATES

pod/d8v-vm-vm-alpine-uefi-s2cgs       0/1     Completed   0          119m    10.66.10.4    virtlab-mi-1              1/1

pod/d8v-vm-vm-alpine-uefi-wnptq       1/1     Running     0          2m20s   10.66.10.4    virtlab-mi-3              1/1

2384Mi -> 384Mi (Crossing 1Gi threshold downward will disable hotplug and requires reboot)

Reboot started 

$ k -n vm get internalvirtualizationvirtualmachines.internal.virtualization.deckhouse.io vm-alpine-uefi -oyaml | grep ' memory:' -A 10
        memory:
          guest: 2384Mi
          maxGuest: 256Gi

$ k -n vm get all,po,intvirt,vmop -owide


NAME                                  READY   STATUS        RESTARTS   AGE    IP            NODE           NOMINATED NODE   READINESS GATES

pod/d8v-vm-vm-alpine-uefi-wnptq       1/1     Terminating   0          3m5s   10.66.10.4    virtlab-mi-3              1/1


NAME                                                              PHASE     CORES   COREFRACTION   MEMORY   NEED RESTART   AGENT   MIGRATABLE   NODE           IPADDRESS    AGE

virtualmachine.virtualization.deckhouse.io/vm-alpine-uefi         Stopped   1       20%            2384Mi                          True         virtlab-mi-3   10.66.10.4   10d


NAME                                                                                               AGE    PHASE       IP    NODENAME       READY   LIVE-MIGRATABLE   PAUSED

internalvirtualizationvirtualmachineinstance.internal.virtualization.deckhouse.io/vm-alpine-uefi   120m   Succeeded         virtlab-mi-3   False   True


NAME                                                                                             AGE   STATUS    READY

internalvirtualizationvirtualmachine.internal.virtualization.deckhouse.io/vm-alpine-uefi         10d   Stopped   False

After reboot 

$ k -n vm get internalvirtualizationvirtualmachines.internal.virtualization.deckhouse.io vm-alpine-uefi -oyaml | grep ' memory:' -A 10
        memory:
          guest: 384Mi

$ k -n vm get all,po,intvirt,vmop -owide

NAME                                  READY   STATUS    RESTARTS   AGE   IP            NODE           NOMINATED NODE   READINESS GATES

pod/d8v-vm-vm-alpine-uefi-x2xvj       1/1     Running   0          26s   10.66.10.4    virtlab-mi-3              1/1


NAME                                                              PHASE     CORES   COREFRACTION   MEMORY   NEED RESTART   AGENT   MIGRATABLE   NODE           IPADDRESS    AGE

virtualmachine.virtualization.deckhouse.io/vm-alpine-uefi         Running   1       20%            384Mi                   False   True         virtlab-mi-3   10.66.10.4   10d


NAME                                                                                               AGE   PHASE     IP           NODENAME       READY   LIVE-MIGRATABLE   PAUSED

internalvirtualizationvirtualmachineinstance.internal.virtualization.deckhouse.io/vm-alpine-uefi   27s   Running   10.66.10.4   virtlab-mi-3   True    True


NAME                                                                                             AGE   STATUS    READY

internalvirtualizationvirtualmachine.internal.virtualization.deckhouse.io/vm-alpine-uefi         10d   Running   True

384Mi -> 512Mi (Hotplug disabled: memory size changing requires reboot)

Reboot started 

$ k -n vm get internalvirtualizationvirtualmachines.internal.virtualization.deckhouse.io vm-alpine-uefi -oyaml | grep ' memory:' -A 10
        memory:
          guest: 384Mi

$ k -n vm get all,po,intvirt,vmop -owide

Warning: kubevirt.io/v1 VirtualMachineInstancePresets is now deprecated and will be removed in v2.

NAME                                  READY   STATUS        RESTARTS   AGE    IP            NODE           NOMINATED NODE   READINESS GATES

pod/d8v-vm-vm-alpine-uefi-x2xvj       1/1     Terminating   0          118s   10.66.10.4    virtlab-mi-3              1/1


NAME                                                              PHASE     CORES   COREFRACTION   MEMORY   NEED RESTART   AGENT   MIGRATABLE   NODE           IPADDRESS    AGE

virtualmachine.virtualization.deckhouse.io/vm-alpine-uefi         Stopped   1       20%            384Mi                           True         virtlab-mi-3   10.66.10.4   10d


NAME                                                                                               AGE    PHASE       IP    NODENAME       READY   LIVE-MIGRATABLE   PAUSED

internalvirtualizationvirtualmachineinstance.internal.virtualization.deckhouse.io/vm-alpine-uefi   118s   Succeeded         virtlab-mi-3   False   True


NAME                                                                                             AGE   STATUS    READY

internalvirtualizationvirtualmachine.internal.virtualization.deckhouse.io/vm-alpine-uefi         10d   Stopped   False

After reboot 

$ k -n vm get internalvirtualizationvirtualmachines.internal.virtualization.deckhouse.io vm-alpine-uefi -oyaml | grep ' memory:' -A 10
        memory:
          guest: 512Mi

$ k -n vm get all,po,intvirt,vmop -owide

Warning: kubevirt.io/v1 VirtualMachineInstancePresets is now deprecated and will be removed in v2.

NAME                                  READY   STATUS    RESTARTS   AGE   IP            NODE           NOMINATED NODE   READINESS GATES

pod/d8v-vm-vm-alpine-uefi-grqdn       1/1     Running   0          33s   10.66.10.4    virtlab-mi-1              1/1


NAME                                                              PHASE     CORES   COREFRACTION   MEMORY   NEED RESTART   AGENT   MIGRATABLE   NODE           IPADDRESS    AGE

virtualmachine.virtualization.deckhouse.io/vm-alpine-uefi         Running   1       20%            512Mi                   False   True         virtlab-mi-1   10.66.10.4   10d


NAME                                                                                               AGE   PHASE     IP           NODENAME       READY   LIVE-MIGRATABLE   PAUSED

internalvirtualizationvirtualmachineinstance.internal.virtualization.deckhouse.io/vm-alpine-uefi   33s   Running   10.66.10.4   virtlab-mi-1   True    True


NAME                                                                                             AGE   STATUS    READY

internalvirtualizationvirtualmachine.internal.virtualization.deckhouse.io/vm-alpine-uefi         10d   Running   True

Checklist

  • The code is covered by unit tests.
  • e2e tests passed.
  • Documentation updated according to the changes.
  • Changes were tested in the Kubernetes cluster manually.

Changelog entries

section: vm
type: feature
summary: "Added initial support for changing virtual machine memory without manually stopping the virtual machine. The new `.spec.memory` value is applied via live migration. To enable this functionality, add `HotplugMemoryWithLiveMigration` to `.spec.settings.featureGates` in the `ModuleConfig` of the `virtualization` module."

@diafour diafour added this to the v1.8.0 milestone Mar 16, 2026
@diafour diafour force-pushed the feat/vm/hotplug-memory branch from 0f99db6 to 8ebdbe2 Compare March 16, 2026 11:46
@diafour diafour self-assigned this Mar 17, 2026
@diafour diafour modified the milestones: v1.8.0, v1.7.0 Mar 17, 2026
@diafour diafour force-pushed the feat/vm/hotplug-memory branch from fb903a5 to e174fb5 Compare March 17, 2026 18:40
@diafour diafour marked this pull request as draft March 17, 2026 18:40
@diafour diafour force-pushed the feat/vm/hotplug-memory branch 2 times, most recently from 6c201f9 to 4af4c78 Compare March 20, 2026 13:17
@diafour diafour marked this pull request as ready for review March 20, 2026 13:18
@diafour diafour requested a review from LopatinDmitr March 20, 2026 13:28
diafour
diafour commented Mar 20, 2026
View reviewed changes
Comment thread images/virt-artifact/werf.inc.yaml Outdated
@diafour diafour force-pushed the feat/vm/hotplug-memory branch from 4af4c78 to 4a4baaf Compare March 24, 2026 19:44
@nevermarine nevermarine modified the milestones: v1.7.0, v1.8.0 Mar 31, 2026
@diafour diafour force-pushed the feat/vm/hotplug-memory branch from 4a4baaf to fb28e31 Compare April 2, 2026 06:38
@diafour diafour removed request for fl64 and goganat April 2, 2026 07:13
@diafour diafour force-pushed the feat/vm/hotplug-memory branch 2 times, most recently from 31b93a2 to cc31668 Compare April 2, 2026 16:15
LopatinDmitr
LopatinDmitr reviewed Apr 4, 2026
View reviewed changes
Comment thread images/virtualization-artifact/pkg/controller/reconciler/reconciler.go
LopatinDmitr
LopatinDmitr previously approved these changes Apr 4, 2026
View reviewed changes
@LopatinDmitr LopatinDmitr self-requested a review April 4, 2026 20:02
@diafour diafour added the e2e/run Run e2e test on cluster of PR author label Apr 9, 2026
@deckhouse-BOaTswain
Copy link
Copy Markdown
Contributor

deckhouse-BOaTswain commented Apr 9, 2026 •
edited
Loading

Workflow has started.
Follow the progress here: Workflow Run

The target step completed with status: failure.

@deckhouse-BOaTswain deckhouse-BOaTswain removed the e2e/run Run e2e test on cluster of PR author label Apr 9, 2026
@diafour
feat(vm): memory hotplug (phase 1)
779ac27 
Simple implementation of the memory hotplug support.

- Change memory configuration: set it in domain.memory.guest instead of setting requests and limits in domain.resources.
- Support old VMs to not require reboot on module update.
- Support for enabling feature gates explicitly in ModuleConfig.

Dev notes:
- Adding detector of kvvm.spec changes to require reboot if kvbuilder changes something.
- New on-demand vmop migration prefixed as "hotplug-resource-".

Signed-off-by: Ivan Mikheykin <ivan.mikheykin@flant.com>
@diafour diafour force-pushed the feat/vm/hotplug-memory branch from 7d6a450 to ff15042 Compare April 9, 2026 19:14
@diafour diafour added the e2e/run Run e2e test on cluster of PR author label Apr 9, 2026
@deckhouse-BOaTswain
Copy link
Copy Markdown
Contributor

deckhouse-BOaTswain commented Apr 9, 2026 •
edited
Loading

Workflow has started.
Follow the progress here: Workflow Run

The target step completed with status: success.

@deckhouse-BOaTswain deckhouse-BOaTswain removed the e2e/run Run e2e test on cluster of PR author label Apr 9, 2026
@diafour diafour added the e2e/run Run e2e test on cluster of PR author label Apr 9, 2026
@deckhouse-BOaTswain
Copy link
Copy Markdown
Contributor

deckhouse-BOaTswain commented Apr 9, 2026 •
edited
Loading

Workflow has started.
Follow the progress here: Workflow Run

The target step completed with status: failure.

@deckhouse-BOaTswain deckhouse-BOaTswain removed the e2e/run Run e2e test on cluster of PR author label Apr 9, 2026
@diafour diafour added the e2e/run Run e2e test on cluster of PR author label Apr 9, 2026
@deckhouse-BOaTswain
Copy link
Copy Markdown
Contributor

deckhouse-BOaTswain commented Apr 9, 2026 •
edited
Loading

Workflow has started.
Follow the progress here: Workflow Run

The target step completed with status: success.

@deckhouse-BOaTswain deckhouse-BOaTswain removed the e2e/run Run e2e test on cluster of PR author label Apr 9, 2026
@diafour diafour added the e2e/run Run e2e test on cluster of PR author label Apr 9, 2026
@deckhouse-BOaTswain
Copy link
Copy Markdown
Contributor

deckhouse-BOaTswain commented Apr 9, 2026 •
edited
Loading

Workflow has started.
Follow the progress here: Workflow Run

The target step completed with status: failure.

@deckhouse-BOaTswain deckhouse-BOaTswain removed the e2e/run Run e2e test on cluster of PR author label Apr 9, 2026
@diafour
Copy link
Copy Markdown
Member Author

diafour commented Apr 10, 2026

e2e result "125 Passed | 4 Failed" can be considered a success:

  • Images provisioning is slow in this cluster, timeout is not sufficient. Same problem was with CVI in ComplexTest.
  • Cluster has no cluster network for the "VirtualMachineAdditionalNetworkInterfaces" test.

@diafour diafour force-pushed the feat/vm/hotplug-memory branch 2 times, most recently from 8463e7c to 9be6e63 Compare April 10, 2026 07:46
@LopatinDmitr LopatinDmitr self-requested a review April 10, 2026 07:50
@diafour
feat(vm): hotplug cpu (phase 1) (#2147)
7e47813 
* feat(vm): hotplug cpu (phase 1)

Simple implementation of CPU hotplug.

- Change cpu cores setting from domain.resources/limits to domain.cpu fields. Change cores count instead of sockets.
- Support old VMs to not require reboot on module update.
- Requires changes in Kubevirt: deckhouse/3p-kubevirt#82

---------

Signed-off-by: Ivan Mikheykin <ivan.mikheykin@flant.com>
@diafour diafour force-pushed the feat/vm/hotplug-memory branch from 9be6e63 to 7e47813 Compare April 10, 2026 07:51
@diafour diafour merged commit f389570 into main Apr 10, 2026
27 of 28 checks passed
@diafour diafour deleted the feat/vm/hotplug-memory branch April 10, 2026 08:04
@deckhouse-BOaTswain deckhouse-BOaTswain mentioned this pull request Apr 10, 2026
Merged
@deckhouse-BOaTswain deckhouse-BOaTswain mentioned this pull request Apr 23, 2026
Merged
This was referenced May 4, 2026
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@LopatinDmitr LopatinDmitr LopatinDmitr approved these changes

@Isteb4k Isteb4k Awaiting requested review from Isteb4k Isteb4k is a code owner

@yaroslavborbat yaroslavborbat Awaiting requested review from yaroslavborbat yaroslavborbat is a code owner

@universal-itengineer universal-itengineer Awaiting requested review from universal-itengineer

@nevermarine nevermarine Awaiting requested review from nevermarine

@prismagod prismagod Awaiting requested review from prismagod prismagod is a code owner

Assignees

@diafour diafour

Labels

None yet

Projects

None yet

Milestone

v1.8.0

Development

Successfully merging this pull request may close these issues.

4 participants

@diafour @deckhouse-BOaTswain @LopatinDmitr @nevermarine