Can't create or scale VM to more then 2 vcpus. XCP-NG.

[furyflash777]

ISSUE TYPE

• Bug Report

COMPONENT NAME

UI

CLOUDSTACK VERSION

4.18.0

CONFIGURATION

OS / ENVIRONMENT

XCP-NG 8.2.1

Global parameter xen.vm.vcpu.max set to 64

Host has 2 sockets and 16 cores each.

SUMMARY

Unable to start VM from different templates (Debian, Ubuntu) when vcpu more then 2.
VM with attached iso started successfully with more then 2 vcpu.

Task failed! Task record: uuid: 8d2ad021-9c6d-daaa-899a-a55867cd49e7
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Tue Mar 28 15:08:19 MSK 2023
finished: Tue Mar 28 15:08:19 MSK 2023
status: failure
residentOn: com.xensource.xenapi.Host@a937b48e
progress: 1.0
type:
result:
errorInfo: [VALUE_NOT_SUPPORTED, VCPUs-at-startup, 8, value greater than VCPUs-max]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []

    at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.checkForSuccess(CitrixResourceBase.java:508)
    at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:5143)
    at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:111)
    at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:57)
    at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
    at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1782)
    at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
    at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
    at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)

GUI error:

Unable to start a VM [e430f6d7-f13e-47b0-8f7c-d8fcc491e268] due to [Unable to create a deployment for VM instance {"id":148,"instanceName":"i-8-148-VM","type":"User","uuid":"e430f6d7-f13e-47b0-8f7c-d8fcc491e268"}].

STEPS TO REPRODUCE

Create VM with 3 or more CPUs

EXPECTED RESULTS

VM created successfully

ACTUAL RESULTS

VM has failed result

[sbrueseke]

I am not able to reproduce this error. I am able to start a VM with more than 3 vCPUs.
You can set the xen.vm.vcpu.max on Cluster level, too. Did you check the value there?

[DaanHoogland]

@furyflash777 I think you need to add more information.
for instance: can you deploy such a VM directly on Xen?
can you give the full stacktrace including any messages that have the same job- and/or ctx-?

[furyflash777]

@furyflash777 I think you need to add more information. for instance: can you deploy such a VM directly on Xen? can you give the full stacktrace including any messages that have the same job- and/or ctx-?

Hello,
New information :
Compute offerings without Dynamic Scale have not this problem.

2023-04-07 01:21:07,477 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-142:ctx-24901669) (logid:ecfc77dc) Seq 5-7472316206737908984: Executing request
2023-04-07 01:21:07,481 DEBUG [c.c.h.x.r.w.x.CitrixStartCommandWrapper] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) 1. The VM i-6-202-VM is in Starting state.
2023-04-07 01:21:07,482 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) no guest OS type, start it as HVM guest
2023-04-07 01:21:07,486 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) No recommended value found for dynamic min
2023-04-07 01:21:07,486 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) No recommended value found for dynamic max, setting static max and dynamic max equal
2023-04-07 01:21:07,497 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) Created VM 6c068ca5-e6fc-b712-19b9-908a6cc81268 for i-6-202-VM
2023-04-07 01:21:07,502 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) Setting boottype=BIOS and bootmode=null for VM: 6c068ca5-e6fc-b712-19b9-908a6cc81268
2023-04-07 01:21:07,511 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) VBD a8dba020-d853-2bac-455c-d337946533a7 created for com.cloud.agent.api.to.DiskTO@1a091b58
2023-04-07 01:21:07,520 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) VBD c98a6d77-647e-1472-97f8-cb20d43714bc created for com.cloud.agent.api.to.DiskTO@2839770
2023-04-07 01:21:07,520 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) Creating VIF for i-6-202-VM on nic [Nic:Guest-10.21.66.19-vlan://1836]
2023-04-07 01:21:07,521 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) Looking for network named trunk
2023-04-07 01:21:07,529 DEBUG [c.c.h.x.r.XsLocalNetwork] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) Found a network called trunk on host=F.A.K.E; Network=eed1eedf-986c-b05b-fd85-d91163e91343; pif=a9a5169b-163c-268d-c000-a827754c6542
2023-04-07 01:21:07,532 DEBUG [c.c.h.x.r.XsLocalNetwork] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) Found a network called VLAN-eed1eedf-986c-b05b-fd85-d91163e91343-1836 on host=F.A.K.E; Network=be8fa6aa-b6fb-a498-7381-f62f69337b98; pif=5f5219c1-4a9
a-2e73-9d8b-2f38b20f47e6
2023-04-07 01:21:07,538 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) Created a vif 809ceb04-8dc6-ec0b-5e07-686dbe4b8b1e on 0
2023-04-07 01:21:08,555 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) Task failed! Task record: uuid: 77dddee7-8a5c-cfb5-6e92-b5805af1dcb4
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 07 01:21:30 MSK 2023
finished: Fri Apr 07 01:21:30 MSK 2023
status: failure
residentOn: com.xensource.xenapi.Host@a937b48e
progress: 1.0
type:
result:
errorInfo: [VALUE_NOT_SUPPORTED, VCPUs-at-startup, 8, value greater than VCPUs-max]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []

2023-04-07 01:21:08,562 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) Unable to start VM(i-6-202-VM) on host(3605b9c3-e0b5-4a94-95b1-9220f426395d) due to Task failed! Task record: uuid: 77dddee7-8a5c-cfb5-
6e92-b5805af1dcb4
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 07 01:21:30 MSK 2023
finished: Fri Apr 07 01:21:30 MSK 2023
status: failure
residentOn: com.xensource.xenapi.Host@a937b48e
progress: 1.0
type:
result:
errorInfo: [VALUE_NOT_SUPPORTED, VCPUs-at-startup, 8, value greater than VCPUs-max]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []

Task failed! Task record: uuid: 77dddee7-8a5c-cfb5-6e92-b5805af1dcb4
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 07 01:21:30 MSK 2023
finished: Fri Apr 07 01:21:30 MSK 2023
status: failure
residentOn: com.xensource.xenapi.Host@a937b48e
progress: 1.0
type:
result:
errorInfo: [VALUE_NOT_SUPPORTED, VCPUs-at-startup, 8, value greater than VCPUs-max]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []

    at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.checkForSuccess(CitrixResourceBase.java:508)
    at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:5143)
    at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:111)
    at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:57)
    at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
    at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1782)
    at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
    at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
    at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)

2023-04-07 01:21:08,568 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) unable to destroy task(com.xensource.xenapi.Task@cfeb0182) on host(3605b9c3-e0b5-4a94-95b1-9220f426395d) due to You gave an invalid object reference.
The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
2023-04-07 01:21:08,570 WARN [c.c.h.x.r.w.x.CitrixStartCommandWrapper] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) Catch Exception: class com.cloud.utils.exception.CloudRuntimeException due to com.cloud.utils.exception.CloudRuntimeException: Unable to sta
rt VM(i-6-202-VM) on host(3605b9c3-e0b5-4a94-95b1-9220f426395d) due to Task failed! Task record: uuid: 77dddee7-8a5c-cfb5-6e92-b5805af1dcb4
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 07 01:21:30 MSK 2023
finished: Fri Apr 07 01:21:30 MSK 2023
status: failure
residentOn: com.xensource.xenapi.Host@a937b48e
progress: 1.0
type:
result:
errorInfo: [VALUE_NOT_SUPPORTED, VCPUs-at-startup, 8, value greater than VCPUs-max]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []

com.cloud.utils.exception.CloudRuntimeException: Unable to start VM(i-6-202-VM) on host(3605b9c3-e0b5-4a94-95b1-9220f426395d) due to Task failed! Task record: uuid: 77dddee7-8a5c-cfb5-6e92-b5805af1dcb4
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 07 01:21:30 MSK 2023
finished: Fri Apr 07 01:21:30 MSK 2023
status: failure
residentOn: com.xensource.xenapi.Host@a937b48e
progress: 1.0
type:
result:
errorInfo: [VALUE_NOT_SUPPORTED, VCPUs-at-startup, 8, value greater than VCPUs-max]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []

    at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:5162)
    at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:111)
    at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:57)
    at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
    at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1782)
    at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
    at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
    at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)

2023-04-07 01:21:08,571 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) Unable to start i-6-202-VM due to
com.cloud.utils.exception.CloudRuntimeException: Unable to start VM(i-6-202-VM) on host(3605b9c3-e0b5-4a94-95b1-9220f426395d) due to Task failed! Task record: uuid: 77dddee7-8a5c-cfb5-6e92-b5805af1dcb4
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 07 01:21:30 MSK 2023
finished: Fri Apr 07 01:21:30 MSK 2023
status: failure
residentOn: com.xensource.xenapi.Host@a937b48e
progress: 1.0
type:
result:
errorInfo: [VALUE_NOT_SUPPORTED, VCPUs-at-startup, 8, value greater than VCPUs-max]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []

    at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:5162)
    at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:111)
    at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:57)
    at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
    at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1782)
    at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
    at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
    at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)

2023-04-07 01:21:08,571 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) Unable to start i-6-202-VM due to
com.cloud.utils.exception.CloudRuntimeException: Unable to start VM(i-6-202-VM) on host(3605b9c3-e0b5-4a94-95b1-9220f426395d) due to Task failed! Task record: uuid: 77dddee7-8a5c-cfb5-6e92-b5805af1dcb4
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 07 01:21:30 MSK 2023
finished: Fri Apr 07 01:21:30 MSK 2023
status: failure
residentOn: com.xensource.xenapi.Host@a937b48e
progress: 1.0
type:
result:
errorInfo: [VALUE_NOT_SUPPORTED, VCPUs-at-startup, 8, value greater than VCPUs-max]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f

        subtasks: []

    at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:5162)
    at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:111)
    at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:57)
    at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
    at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1782)
    at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
    at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
    at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)

2023-04-07 01:21:08,591 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) Unable to clean up VBD due to
You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
at com.xensource.xenapi.Types.checkResponse(Types.java:693)
at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:457)
at com.xensource.xenapi.VBD.unplug(VBD.java:1109)
at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:3677)
at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:222)
at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:57)
at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1782)
at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2023-04-07 01:21:08,593 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) Unable to clean up VBD due to
You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
at com.xensource.xenapi.Types.checkResponse(Types.java:693)
at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:457)
at com.xensource.xenapi.VBD.unplug(VBD.java:1109)
at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:3677)
at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:222)
at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:57)
at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1782)
at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)

2023-04-07 01:21:08,597 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) Unable to cleanup VIF
You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
at com.xensource.xenapi.Types.checkResponse(Types.java:693)
at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:457)
at com.xensource.xenapi.VIF.unplug(VIF.java:921)
at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:3685)
at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:222)
at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:57)
at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1782)
at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2023-04-07 01:21:08,603 DEBUG [c.c.h.x.r.w.x.CitrixStartCommandWrapper] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) The VM is in stopped state, detected problem during startup : i-6-202-VM
2023-04-07 01:21:08,603 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) Seq 5-7472316206737908984: Response Received:
2023-04-07 01:21:08,604 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (DirectAgent-142:ctx-24901669) (logid:7a9ab01d) Seq 5-7472316206737908984: MgmtId 46327103290396: Resp: Routing to peer
2023-04-07 01:21:09,419 DEBUG [c.c.a.t.Request] (AgentManager-Handler-6:null) (logid:) Seq 5-7472316206737908985: Executing: { Cmd , MgmtId: 46327103290396, via: 5, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCle
anup":"false","forceStop":"false","vlanToPersistenceMap":{"1836":"true"},"volumesToDisconnect":[],"vmName":"i-6-202-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}] }

[weizhouapache]

I had a quick investigation

in PR #4672, the vcpu max is set to min of host cpu socket

cloudstack/plugins/hypervisors/xenserver/src/main/java/com/cloud/hypervisor/XenServerGuru.java

Lines 99 to 106 in 0574087

	List<HostVO> clusterHosts = hostDao.listByClusterAndHypervisorType(host.getClusterId(), host.getHypervisorType());
	HostVO hostWithMinSocket = clusterHosts.stream().min(Comparator.comparing(HostVO::getCpuSockets)).orElse(null);
	Integer vCpus = MaxNumberOfVCPUSPerVM.valueIn(host.getClusterId());
	if (hostWithMinSocket != null && hostWithMinSocket.getCpuSockets() != null &&
	hostWithMinSocket.getCpuSockets() < vCpus) {
	vCpus = hostWithMinSocket.getCpuSockets();
	}
	to.setVcpuMaxLimit(vCpus);

if dynamic scaling is enabled, the vcpu max is set to the above value, which might be smaller than the vcpus of vm instance.

cloudstack/plugins/hypervisors/xenserver/src/main/java/com/cloud/hypervisor/xenserver/resource/CitrixResourceBase.java

Lines 1331 to 1343 in 0574087

	if (isDmcEnabled(conn, host) && vmSpec.isEnableDynamicallyScaleVm()) {
	// scaling is allowed
	vmr.memoryStaticMin = getStaticMin(vmSpec.getOs(), vmSpec.getBootloader() == BootloaderType.CD, vmSpec.getMinRam(), vmSpec.getMaxRam(), recommendedMemoryMin);
	vmr.memoryStaticMax = getStaticMax(vmSpec.getOs(), vmSpec.getBootloader() == BootloaderType.CD, vmSpec.getMinRam(), vmSpec.getMaxRam(), recommendedMemoryMax);
	vmr.memoryDynamicMin = vmSpec.getMinRam();
	vmr.memoryDynamicMax = vmSpec.getMaxRam();
	if (guestOsTypeName.toLowerCase().contains("windows")) {
	vmr.VCPUsMax = (long)vmSpec.getCpus();
	} else {
	if (vmSpec.getVcpuMaxLimit() != null) {
	vmr.VCPUsMax = (long)vmSpec.getVcpuMaxLimit();
	}
	}

@shwstppr
can you please have look ?

[shwstppr]

@weizhouapache will have a look later this week or early next week

[weizhouapache]

@weizhouapache will have a look later this week or early next week

@shwstppr
cool, thanks.