RTX 5070 (Blackwell) Compatibility: Severe Slowdown/Freeze due to DynamicSwap I/O Bottleneck

[cubeforeq5-oss]

Hello,

I am experiencing a critical performance issue when attempting to generate videos using FramePack on the new NVIDIA RTX 5070 (Blackwell) GPU. The issue appears to be related to model swapping/loading overhead rather than GPU calculation speed.

1. Environment Details

• GPU: NVIDIA GeForce RTX 5070 (12 GB VRAM)
• CPU: AMD Ryzen 7 5700X
• RAM: 64 GB
• OS: Windows 11
• FramePack Setup: Custom install with Python 3.12, CUDA 12.8, PyTorch nightly build (cu128), and latest sageattention.

2. Issue Description: High Usage / Low Power Freeze

Video generation does complete, but it is extremely slow due to the constant, lengthy waiting periods during model loading.

The key observation is a significant contradiction in GPU telemetry during the wait periods:

• GPU Usage: Consistently 99%
• GPU Power Draw: Extremely low, only ~49 Watts
• GPU Temperature: Very low, ~50°C
• Clock Speed: High (~2880 MHz)

This indicates the GPU is stuck waiting for data, not performing heavy calculation, suggesting an I/O or memory access bottleneck caused by the DynamicSwap feature.

3. Specific Bottleneck Log

The process frequently freezes/stalls at the model loading and unloading points, specifically with the Autoencoder and Transformer models.

Example Log of Stall:

Unloaded DynamicSwap_HunyuanVideoTransformer3DModelPacked as complete. ... (Waiting for seconds/minutes) Loaded AutoencoderKLHunyuanVideo to cuda:0 as complete. ... (Processing starts: ~6.55 s/it) ... (Processing finishes, then stall occurs) Moving DynamicSwap_HunyuanVideoTransformer3DModelPacked to cuda:0 with preserved memory: 6 GB

4. Current Performance

After a full clean reinstall, the actual calculation speed is good (~6.55 s/it), but the overall generation time is poor due to the prolonged loading/offloading stalls.

We suspect this is a compatibility issue with the DynamicSwap mechanism and the RTX 5000 series (Blackwell) architecture, possibly related to how the custom sageattention or memory allocation handles the new hardware.

Thank you for your attention to this critical issue.

[nono7415]

I have the same problem with
GPU: NVIDIA GeForce RTX 4080 (16 GB VRAM)
CPU: INTEL I9
RAM: 64 GB
OS: Windows 11

Thank you for your attention.

[nemuri96ai]

Hi, I’m having the exact same issue on an RTX 5070 (Blackwell).
Did you manage to find any solution or workaround?

If there is even a temporary fix to reduce the long loading stalls, I would really appreciate it. Thank you!

[Careed315]

I seem to have the same issue as well with a 5080 and 128GB of ddr5 ram... I also don't see any copy usage in the GPU task manager or CPU usage or disk usage spikes.
I have sageattention installed as well but its still slow on the loading/unloading
Thanks!

[peerobo]

I have experienced the same problem but after using pytorch stable

pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128

I have not experienced that slow offloading.

[Squeedwad]

Same issue with the loading offloading, except with a 5090, don't even have sage installed because I fresh installed. Unironically, my 3060 setup I had before this fresh install was faster than this. Each second ends up taking 3+ hours due to the loading/offloading.

[nono7415]

Hi everyone,

The root cause, as previously suspected, was a bug introduced by the mandatory PyTorch 2.9.0 upgrade.
The VAE slowdown was not directly caused by Xformers but by a known bug within PyTorch 2.9.0 itself, specifically affecting 3D convolution operations on NVIDIA cards, which are crucial for video VAE processing.

This bug forced the VAE workload to move from VRAM (GPU memory) to System RAM, resulting in extremely slow generation times (e.g., 40 minutes for a short clip) and potential system crashes.

The only way to completely eliminate this bug and restore normal VAE performance was to force a downgrade of PyTorch to a known stable version that does not contain the bug (we settled on 2.5.1).

Step 1: Install Stable PyTorch Version with CUDA

We had to bypass the default installer to ensure we got the CUDA-enabled version (+cu121) of PyTorch 2.5.1, as the default install often pulls the CPU version, causing the error: Torch not compiled with CUDA enabled.
Bash

Target: PyTorch 2.5.1, TorchVision 0.20.1, TorchAudio 2.5.1
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --force-reinstall --index-url https://download.pytorch.org/whl/cu121

Step 2: Stabilize Core Scientific Libraries
The PyTorch downgrade broke the compatibility of NumPy and SciPy, which needed to be locked to specific stable versions:
Bash

Resolves NumPy/SciPy conflicts
pip install numpy==1.26.4 scipy==1.12.0 --force-reinstall
Step 3: Remove Conflicting Attention Modules (If Present)
To ensure a clean restart, any existing attention modules that caused conflicts or instability were removed:
Bash
Removes xFormers, which conflicts with PyTorch 2.9 and breaks startup
pip uninstall xformers -y

🚀 Result

After these steps, FramePack Studio started successfully, recognized the GPU, and the VAE processing times returned to normal (pre-update performance).

Hope this helps others experiencing the same critical issue!

[Squeedwad]

Hi everyone,

The root cause, as previously suspected, was a bug introduced by the mandatory PyTorch 2.9.0 upgrade. The VAE slowdown was not directly caused by Xformers but by a known bug within PyTorch 2.9.0 itself, specifically affecting 3D convolution operations on NVIDIA cards, which are crucial for video VAE processing.

This bug forced the VAE workload to move from VRAM (GPU memory) to System RAM, resulting in extremely slow generation times (e.g., 40 minutes for a short clip) and potential system crashes.

The only way to completely eliminate this bug and restore normal VAE performance was to force a downgrade of PyTorch to a known stable version that does not contain the bug (we settled on 2.5.1).

Step 1: Install Stable PyTorch Version with CUDA

We had to bypass the default installer to ensure we got the CUDA-enabled version (+cu121) of PyTorch 2.5.1, as the default install often pulls the CPU version, causing the error: Torch not compiled with CUDA enabled. Bash

Target: PyTorch 2.5.1, TorchVision 0.20.1, TorchAudio 2.5.1 pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --force-reinstall --index-url https://download.pytorch.org/whl/cu121

Step 2: Stabilize Core Scientific Libraries The PyTorch downgrade broke the compatibility of NumPy and SciPy, which needed to be locked to specific stable versions: Bash

Resolves NumPy/SciPy conflicts pip install numpy==1.26.4 scipy==1.12.0 --force-reinstall Step 3: Remove Conflicting Attention Modules (If Present) To ensure a clean restart, any existing attention modules that caused conflicts or instability were removed: Bash Removes xFormers, which conflicts with PyTorch 2.9 and breaks startup pip uninstall xformers -y

🚀 Result

After these steps, FramePack Studio started successfully, recognized the GPU, and the VAE processing times returned to normal (pre-update performance).

Hope this helps others experiencing the same critical issue!

Didn't work for me. Get this error instead now without generating anything. Tried it on a fresh install too with this error popping up instead.

[peerobo]

@Squeedwad you can try my command on fresh install.

pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128

[Squeedwad]

@Squeedwad you can try my command on fresh install.

pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128

Still erroring out on fresh install

[peerobo]

@Squeedwad did you activate venv before update with pip?

[Squeedwad]

@Squeedwad did you activate venv before update with pip?
@peerobo

For the fresh install, no, did the commands immediately upon extracting the files

[peerobo]

Did you run the command on python embed?

install the zip file then in the framepack_cu126_torch26\system\python folder run those commands:
.\python.exe -m pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128

[Squeedwad]

Did you run the command on python embed?

install the zip file then in the framepack_cu126_torch26\system\python folder run those commands:
.\python.exe -m pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128

Ran it through the command terminal from the folder

[himwithnoname-boop]

I've edited the methods in the PY file and enable acceleration which appears to have helped as well as the pytorch upgrades:

Try this UI with the extra options I put together... hopefully it will work...works for me on a 5090 - obviously you need to modify run.bat to call this py file and have the file in the same location as the original

test-vid-ui.py