Issue with CachyOS, 4090 and Framepack #790
Description
The issue:
Getting consistent OOM (out of memory) errors, even with a 4090 despite using every kind of option in the program. The strangest thing is. This software with the same hardware, worked fine with Nobara (Fedora Core).
So I am unsure what is going on.
Could this be an issue related to the usage of FISH vs BASH? ( Fish is the default of CachyOS ( Arch )).
Specs:
OS: CachyOS x86_64
Host: Z690 AORUS PRO (-CF)
kernel: Linux 6.17.9-2-cachyos
Shell: fish 4.2.1
Display (PHL 276E8V): 3840x2160 @ 1.25x in 27", 60 Hz [External] *
Display (PHL 276E8V): 3840x2160 @ 1.25x in 27", 60 Hz [External]
Display (HAT1560): 1920x1080 @ 1.25x in 16", 60 Hz [External]
DE: KDE Plasma 6.5.3
WM: KWin (Wayland)
WM Theme: Breeze
Terminal: konsole 25.8.3
CPU: Intel(R) Core(TM) i9-14900K (32) @ 6.00 GHz
GPU: NVIDIA GeForce RTX 4090 [Discrete]
Memory: 22.50 GiB / 62.61 GiB (36%)
Swap: 12.00 KiB / 62.61 GiB (0%)
Disk (/): 502.60 GiB / 929.51 GiB (54%) - btrfs
Disk (/mnt/4TBSea2): 292.60 GiB / 3.38 TiB (8%) - ext4
Disk (/mnt/4TBSeagate): 1.67 TiB / 3.44 TiB (49%) - ext4
Disk (/mnt/ADATA 1TB): 408.36 GiB / 816.24 GiB (50%) - ext4
Disk (/mnt/Crucial4TBSSD): 928.94 GiB / 3.58 TiB (25%) - ext4
Disk (/mnt/DOWNLOAD6TB): 2.70 TiB / 5.33 TiB (51%) - btrfs
Disk (/mnt/ForceMP500): 158.05 GiB / 219.00 GiB (72%) - ext4
Disk (/mnt/GigabyteGP500GB): 315.08 GiB / 457.38 GiB (69%) - ext4
Python:
Shell version Python : 3.13.7
Virtual Environment version: 3.11.14
Error Log:
Starting Environment
Currently enabled native sdp backends: ['flash', 'math', 'mem_efficient', 'cudnn']
Xformers is installed!
Flash Attn is not installed!
Sage Attn is installed!
Namespace(share=False, server='0.0.0.0', port=None, inbrowser=False)
Free VRAM 17.74957275390625 GB
High-VRAM Mode: False
Downloading shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 8268.71it/s]
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:01<00:00, 2.03it/s]
Fetching 3 files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 3639.84it/s]
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:05<00:00, 1.67s/it]
transformer.high_quality_fp32_output_for_inference = True
- Running on local URL: http://0.0.0.0:7860
To create a public link, set share=True in launch().
Unloaded DynamicSwap_LlamaModel as complete.
Unloaded CLIPTextModel as complete.
Unloaded SiglipVisionModel as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Unloaded DynamicSwap_HunyuanVideoTransformer3DModelPacked as complete.
Loaded CLIPTextModel to cuda:0 as complete.
Unloaded CLIPTextModel as complete.
Loaded AutoencoderKLHunyuanVideo to cuda:0 as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Loaded SiglipVisionModel to cuda:0 as complete.
latent_padding_size = 27, is_last_section = False
Unloaded SiglipVisionModel as complete.
Moving DynamicSwap_HunyuanVideoTransformer3DModelPacked to cuda:0 with preserved memory: 6 GB
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:54<00:00, 2.18s/it]
Offloading DynamicSwap_HunyuanVideoTransformer3DModelPacked from cuda:0 to preserve memory: 8 GB
Loaded AutoencoderKLHunyuanVideo to cuda:0 as complete.
Traceback (most recent call last):
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/demo_gradio.py", line 285, in worker
history_pixels = vae_decode(real_history_latents, vae).cpu()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/diffusers_helper/hunyuan.py", line 98, in vae_decode
image = vae.decode(latents.to(device=vae.device, dtype=vae.dtype)).sample
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/diffusers/utils/accelerate_utils.py", line 46, in wrapper
return method(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/diffusers/models/autoencoders/autoencoder_kl_hunyuan_video.py", line 868, in decode
decoded = self._decode(z).sample
^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/diffusers/models/autoencoders/autoencoder_kl_hunyuan_video.py", line 836, in _decode
return self._temporal_tiled_decode(z, return_dict=return_dict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/diffusers/models/autoencoders/autoencoder_kl_hunyuan_video.py", line 1052, in _temporal_tiled_decode
decoded = self.tiled_decode(tile, return_dict=True).sample
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/diffusers/models/autoencoders/autoencoder_kl_hunyuan_video.py", line 984, in tiled_decode
decoded = self.decoder(tile)
^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/diffusers/models/autoencoders/autoencoder_kl_hunyuan_video.py", line 618, in forward
hidden_states = up_block(hidden_states)
^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/diffusers/models/autoencoders/autoencoder_kl_hunyuan_video.py", line 408, in forward
hidden_states = upsampler(hidden_states)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/diffusers/models/autoencoders/autoencoder_kl_hunyuan_video.py", line 120, in forward
hidden_states = self.conv(hidden_states)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/diffusers/models/autoencoders/autoencoder_kl_hunyuan_video.py", line 79, in forward
return self.conv(hidden_states)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 717, in forward
return self._conv_forward(input, self.weight, self.bias)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 712, in _conv_forward
return F.conv3d(
^^^^^^^^^
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 14.34 GiB. GPU 0 has a total capacity of 23.51 GiB of which 3.00 GiB is free. Process 9854 has 14.10 MiB memory in use. Including non-PyTorch memory, this process has 15.25 GiB memory in use. Of the allocated memory 12.63 GiB is allocated by PyTorch, and 2.12 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
Unloaded AutoencoderKLHunyuanVideo as complete.
Unloaded DynamicSwap_LlamaModel as complete.
Unloaded CLIPTextModel as complete.
Unloaded SiglipVisionModel as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Unloaded DynamicSwap_HunyuanVideoTransformer3DModelPacked as complete.
Unloaded DynamicSwap_LlamaModel as complete.
Unloaded CLIPTextModel as complete.
Unloaded SiglipVisionModel as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Unloaded DynamicSwap_HunyuanVideoTransformer3DModelPacked as complete.
Loaded CLIPTextModel to cuda:0 as complete.
Unloaded CLIPTextModel as complete.
Loaded AutoencoderKLHunyuanVideo to cuda:0 as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Loaded SiglipVisionModel to cuda:0 as complete.
latent_padding_size = 27, is_last_section = False
Unloaded SiglipVisionModel as complete.
Moving DynamicSwap_HunyuanVideoTransformer3DModelPacked to cuda:0 with preserved memory: 6 GB
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:53<00:00, 2.12s/it]
Offloading DynamicSwap_HunyuanVideoTransformer3DModelPacked from cuda:0 to preserve memory: 8 GB
Loaded AutoencoderKLHunyuanVideo to cuda:0 as complete.
Traceback (most recent call last):
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/demo_gradio.py", line 285, in worker
history_pixels = vae_decode(real_history_latents, vae).cpu()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/diffusers_helper/hunyuan.py", line 98, in vae_decode
image = vae.decode(latents.to(device=vae.device, dtype=vae.dtype)).sample
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/diffusers/utils/accelerate_utils.py", line 46, in wrapper
return method(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/diffusers/models/autoencoders/autoencoder_kl_hunyuan_video.py", line 868, in decode
decoded = self._decode(z).sample
^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/diffusers/models/autoencoders/autoencoder_kl_hunyuan_video.py", line 836, in _decode
return self._temporal_tiled_decode(z, return_dict=return_dict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/diffusers/models/autoencoders/autoencoder_kl_hunyuan_video.py", line 1052, in _temporal_tiled_decode
decoded = self.tiled_decode(tile, return_dict=True).sample
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/diffusers/models/autoencoders/autoencoder_kl_hunyuan_video.py", line 984, in tiled_decode
decoded = self.decoder(tile)
^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/diffusers/models/autoencoders/autoencoder_kl_hunyuan_video.py", line 618, in forward
hidden_states = up_block(hidden_states)
^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/diffusers/models/autoencoders/autoencoder_kl_hunyuan_video.py", line 408, in forward
hidden_states = upsampler(hidden_states)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/diffusers/models/autoencoders/autoencoder_kl_hunyuan_video.py", line 120, in forward
hidden_states = self.conv(hidden_states)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/diffusers/models/autoencoders/autoencoder_kl_hunyuan_video.py", line 79, in forward
return self.conv(hidden_states)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 717, in forward
return self._conv_forward(input, self.weight, self.bias)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ADATA 1TB/ComfyUI Clone/FramePack/.FramePack/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 712, in _conv_forward
return F.conv3d(
^^^^^^^^^
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 14.34 GiB. GPU 0 has a total capacity of 23.51 GiB of which 2.99 GiB is free. Process 9854 has 14.10 MiB memory in use. Including non-PyTorch memory, this process has 13.91 GiB memory in use. Of the allocated memory 11.28 GiB is allocated by PyTorch, and 2.13 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
Unloaded AutoencoderKLHunyuanVideo as complete.
Unloaded DynamicSwap_LlamaModel as complete.
Unloaded CLIPTextModel as complete.
Unloaded SiglipVisionModel as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Unloaded DynamicSwap_HunyuanVideoTransformer3DModelPacked as complete.
I also tried different versions of python. And no go..
ComfyUI seems to work fine.. openwebui in the other hand.. I had to run it as a docker.
any ideas folks?