GPU resumes to D0 immediately after ending transition to D3Cold (RTD3)

[imaGuru]

NVIDIA Open GPU Kernel Modules Version

575.64.03
575.64.05-1
580.76.05

Please confirm this issue does not happen with the proprietary driver (of the same version). This issue tracker is only for bugs specific to the open kernel driver.

• I confirm that this does not happen with the proprietary driver package.

Operating System and Version

Arch Linux

Kernel Release

6.15.7-arch1-1 #1 SMP PREEMPT_DYNAMIC Thu, 17 Jul 2025 21:05:29 +0000 x86_64 GNU/Linux
6.12.39-1
6.16.3-arch1-1 #1 SMP PREEMPT_DYNAMIC Sat, 23 Aug 2025 15:32:49 +0000 x86_64 GNU/Linux

Please confirm you are running a stable release kernel (e.g. not a -rc). We do not accept bug reports for unreleased kernels.

• I am running on a stable kernel release.

Hardware: GPU

NVIDIA GeForce RTX 4060 Laptop GPU (UUID: GPU-a0338cc2-52ca-fc18-d87f-0a5df1c5ca22)

Describe the bug

Laptop: Lenovo Legion Slim 5 16ahp9
CPU: AMD Ryzen 7 8845HS w/ Radeon 780M Graphics

dGPU can enter suspended state and turn off memory but won't stay cold. dGPU is woken up immediately after ending transition to D3cold. Waiting 15s causes GPU to sleep again, only to repeat the cycle. This behaviour persists even after killing all graphical interface (gdm/gnome/wayland/xorg) and no applications are using the GPU.

nvidia_drm module can be removed with modprobe -r when not running gnome (wayland/xorg). After removal the GPU suspends normally. Starting GDM and gnome with nvidia_drm removed works - running apps on dGPU also seems to work. Enabling nvidia-persistenced, nvidia-powerd or modprobing nvidia_drm brings back the bug with 15s turn on/off cycle

Suspending system to ram and waking up the laptop allows the GPU to power down correctly once for indefinite amount of time, until the first wakeup. After the bug manifests as usual.

These look like the same or very similar issues:

• https://forums.developer.nvidia.com/t/nvidia-gpu-fails-to-power-off-prime-razer-blade-14-2022/250023/40
• https://forums.developer.nvidia.com/t/4070-555-and-560-drivers-wont-stay-in-d3cold-lenovo-legion-slim-5/302967
• https://www.markwatkinson.com/knowledge/linux/nvidia-dgpu-power/#known-issues - exactly the same isssue

Best workaround which allows the dGPU to stay in D3Cold although with memory turned on, but in self refresh mode:

options nvidia NVreg_DynamicPowerManagement=0x02
options nvidia NVreg_DynamicPowerManagementVideoMemoryThreshold=0

# Enable runtime PM for NVIDIA VGA/3D controller devices on driver bind
ACTION=="bind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030000", TEST=="power/control", ATTR{power/control}="auto"
ACTION=="bind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030200", TEST=="power/control", ATTR{power/control}="auto"

# Disable runtime PM for NVIDIA VGA/3D controller devices on driver unbind
ACTION=="unbind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030000", TEST=="power/control", ATTR{power/control}="on"
ACTION=="unbind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030200", TEST=="power/control", ATTR{power/control}="on"

Windows doesn't have this issue.

To Reproduce

1. Boot into system gdm/gnome (wayland or xorg) (GPU goes into cold state and wakes up during booting into GUI - runtime_suspended_time > 0). If GPU was not d3cold, let it go into d3cold and wake it after up (eg. with nvidia-smi)
2. Wait 15s for GPU to go to sleep.
3. GPU wakes up immediately after suspending.
4. Cycle continues every 15s, gpu can no longer stay d3cold longer than 1s

Bug Incidence

Always

nvidia-bug-report.log.gz

nvidia-bug-report.log.gz

More Info

No response

[francoism90]

I have the same issue, and it causes my laptop to overheat! I had placed in my bag thinking the laptop was in sleep, but it almost set fire to it.. this should be fixed asap.

[Binary-Eater]

This sounds like its similar in nature to the problem I am trying to fix here: #759 (comment)

If you load nvidia_drm with the kernel parameters modeset=1 fbdev=0, are you able to reproduce the issue?

[imaGuru]

Unfortunately, yes. The issue persists, even with fbdev=0.

> cat /sys/module/nvidia_drm/parameters/modeset
Y
> cat /sys/module/nvidia_drm/parameters/fbdev 
N

Arch Wiki mentions:

Note: Kernels officially supported by Arch enable simpledrm, while NVIDIA driver requires efifb or vesafb when nvidia_drm.fbdev is disabled/unavailable (version < 545).

I do not know if this has any impact on your suggestion

Only semiworkable solution so far is suspending the laptop and after wakeup the first d3cold state holds correctly, subsequent do not.

[Binary-Eater]

Thanks @imaGuru. That suggests to me that there may be other paths in nvidia-drm where we may be making allocations that have certain requirements, preventing the GPU from being able to stay in RTD3. It would help me if we could conduct the following test.

1. Reload nvidia-drm with modeset=0 parameter
2. Launch GNOME (using Xorg)
3. Re-conduct the experiment to see if the GPU now stays in RTD3

If my theory is correct, I would expect this to enable the GPU to stay in RTD3.

[imaGuru]

Hey. Unfortunately, still the same result - changing modeset, or fbdev doesn't have any effect on my issue (d3cold holds only for 1s).

I did however check the issue with external monitors. After connecting external monitor and disconnecting it, GPU cannot enter d3cold at all (even the broken 1s d3colds). Reloading nvidia_drm (modeset, fbdev values don't matter) fixes that - GPU can enter d3cold, but because of my issue they only last 1s :(

[Binary-Eater]

Thanks @imaGuru ,

That's quite peculiar. The reason I wanted to run this experiment is because with modeset=0, nvidia-drm does almost nothing. Would you mind sharing a nvidia-bug-report.sh dump in the repro-ing case with modeset=0?

[imaGuru]

Sure, here you go modeset=0:

nvidia-bug-report.log.gz

To recap: first d3cold after boot or laptop suspend to ram holds, subsequent do not (last only 1s then wakeup). Removing nvidia_drm and stoping nvidia_powerd (removing just nvidia_drm with nvidia_powerd left running, keeps the gpu in the wakeup cycle) allows GPU to stay d3cold after first wakeup

[francoism90]

@imaGuru If I understand you correctly, it works when setting modeset=0?

I notice mime is set like this: https://rpmfusion.org/Howto/NVIDIA?highlight=%28%5CbCategoryHowto%5Cb%29#OSTree_.28Silverblue.2FKinoite.2Fetc.29

I'll try with modeset=0 to verify if this solves the issue, because it wakes my laptop every time after going to sleep (NVIDIA Optimus powered).

[imaGuru]

@francoism90 no, modset or fbdev do not change anything for me. I think I have tried every possible configuration of Nvidia driver, xorg and wayland, nothing works unfortunately. RTD3 works only the first time it triggers, after first wakeup it does enter rtd3cold but can never stay cold longer than 1s. The result is that it constantly powers off and on and seems to waste even more power, so it's better to disable rt3d or take great care not to wake the GPU after first sleep since boot/laptop suspend.

[francoism90]

@imaGuru Confirmation here - the nomodeset=0 doesn't change anything.

It's weird it seems to sleep fine when being connected on my dock (USB-C). But when I unplug it, and close the laptop - it wakes up again (but without you knowing the screen is back on).

[imaGuru]

@Binary-Eater anything interesting in the logs? The weird thing is that the problem persists even when nvidia_drm is unloaded but nvidia-powerd is running. That would suggest to me that the problem is somewhere deeper in the driver? Maybe dGPU is improperly put to sleep and after wakeup something is not setup right?

It seems that new lenovo legion's suffer from this. My old y540 with 1660ti and legion 5 with 3050ti all worked fine and GPUs stay d3cold with no problem on Linux.

I also get these error's from nvidia-powerd (maybe related to some improper power managment of the card?):

sie 04 10:31:12 sanctuary540 nvidia-powerd[895]: ERROR! Error in processParam call : -1 16   13
sie 04 10:31:12 sanctuary540 nvidia-powerd[895]: ERROR! Error in processParam call : -1 16   13
sie 04 10:31:12 sanctuary540 nvidia-powerd[895]: ERROR! Error in processParam call : -1 16   13

[francoism90]

@imaGuru I don't think the powered daemon is compatibel, you can find this error also on the issue tracker. I've disabled that service for now.

Sleep doesn't work at all. It's also dangerous, because you get the impression it goes to sleep, but it wakes up when you put your laptop in your bag.