Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
109 commits
Select commit Hold shift + click to select a range
fc4a76e
[1/N] add helios
Feb 24, 2026
58f4603
fix test
Feb 24, 2026
280e3d8
make fix-copies
Feb 24, 2026
6e0f23b
change script path
Feb 24, 2026
3a04f67
fix cus script
Feb 24, 2026
d693c04
update docs
Feb 25, 2026
44b04dd
fix documented check
Feb 25, 2026
50ab6a5
update links for docs and examples
Feb 25, 2026
75809c6
change default config
Feb 25, 2026
608f22c
small refactor
Feb 25, 2026
31fb0cf
add test
Feb 25, 2026
f276d2f
Update src/diffusers/models/transformers/transformer_helios.py
SHYuanBest Feb 26, 2026
061a442
remove register_buffer for _scale_cache
Feb 26, 2026
ba05daf
fix non-cuda devices error
Feb 26, 2026
bc3c528
remove "handle the case when timestep is 2D"
SHYuanBest Feb 26, 2026
2508058
refactor HeliosMultiTermMemoryPatch and process_input_hidden_states
SHYuanBest Feb 26, 2026
96747b7
Update src/diffusers/pipelines/helios/pipeline_helios.py
SHYuanBest Feb 26, 2026
37eb8f0
Update src/diffusers/models/transformers/transformer_helios.py
SHYuanBest Feb 26, 2026
439b76d
Update src/diffusers/pipelines/helios/pipeline_helios.py
SHYuanBest Feb 26, 2026
bb2482a
fix calculate_shift
SHYuanBest Feb 26, 2026
3680998
Update src/diffusers/pipelines/helios/pipeline_helios.py
SHYuanBest Feb 26, 2026
9b4fbe7
rewritten `einops` in pure `torch`
SHYuanBest Feb 26, 2026
cee81f8
fix: pass patch_size to apply_schedule_shift instead of hardcoding
SHYuanBest Feb 26, 2026
a7960fe
remove the logics of 'vae_decode_type'
SHYuanBest Feb 26, 2026
2683ca0
move some validation into check_inputs()
SHYuanBest Feb 26, 2026
2f72a8c
rename helios scheduler & merge all into one step()
SHYuanBest Feb 26, 2026
9413826
add some details to doc
SHYuanBest Feb 26, 2026
3bd8b86
move dmd step() logics from pipeline to scheduler
SHYuanBest Feb 26, 2026
f5fe040
change to Python 3.9+ style type
SHYuanBest Feb 26, 2026
1572cf0
fix NoneType error
SHYuanBest Feb 26, 2026
3b78e9b
refactor DMD scheduler's set_timestep
SHYuanBest Feb 26, 2026
ba4d203
change rope related vars name
SHYuanBest Feb 26, 2026
4bdcc0a
fix stage2 sample
SHYuanBest Feb 26, 2026
2f7bfc5
fix dmd sample
SHYuanBest Feb 26, 2026
90c355d
Update src/diffusers/models/transformers/transformer_helios.py
SHYuanBest Feb 27, 2026
8256941
Update src/diffusers/models/transformers/transformer_helios.py
SHYuanBest Feb 27, 2026
389c830
remove redundant & refactor norm_out
SHYuanBest Feb 27, 2026
cbe52d5
Update src/diffusers/pipelines/helios/pipeline_helios.py
SHYuanBest Feb 27, 2026
20eeed6
change "is_keep_x0" to "keep_first_frame"
SHYuanBest Feb 27, 2026
38d50d2
use a more intuitive name
SHYuanBest Feb 27, 2026
378d8a9
refactor dynamic_time_shifting
SHYuanBest Feb 27, 2026
eff865c
remove use_dynamic_shifting args
SHYuanBest Feb 27, 2026
866c55c
remove usage of UniPCMultistepScheduler
SHYuanBest Feb 27, 2026
54abd1c
separate stage2 sample to HeliosPyramidPipeline
SHYuanBest Feb 27, 2026
5a87c0c
Update src/diffusers/models/transformers/transformer_helios.py
SHYuanBest Feb 28, 2026
a573460
Update src/diffusers/models/transformers/transformer_helios.py
SHYuanBest Feb 28, 2026
94988e8
Update src/diffusers/models/transformers/transformer_helios.py
SHYuanBest Feb 28, 2026
0171b3f
Update src/diffusers/models/transformers/transformer_helios.py
SHYuanBest Feb 28, 2026
e4efddf
fix transformer
SHYuanBest Feb 28, 2026
e1bb744
use a more intuitive name
SHYuanBest Feb 28, 2026
7a1785e
update example script
SHYuanBest Feb 28, 2026
f811ce2
fix requirements
Feb 28, 2026
926b0d6
remove redudant attention mask
Feb 28, 2026
244d994
fix
Feb 28, 2026
c8f571d
optimize pipelines
SHYuanBest Feb 28, 2026
50b565a
make style .
SHYuanBest Feb 28, 2026
ca8cc7e
update TYPE_CHECKING
SHYuanBest Mar 1, 2026
7174e44
change to use torch.split
SHYuanBest Mar 1, 2026
dee65a5
derive memory patch sizes from patch_size multiples
SHYuanBest Mar 1, 2026
d431f54
remove some hardcoding
SHYuanBest Mar 1, 2026
c618877
move some checks into check_inputs
SHYuanBest Mar 1, 2026
f25d927
refactor sample_block_noise
SHYuanBest Mar 1, 2026
42d30d0
optimize encoding chunks logits for v2v
SHYuanBest Mar 1, 2026
04a0342
use num_history_latent_frames = sum(history_sizes)
SHYuanBest Mar 1, 2026
971f08c
Update src/diffusers/pipelines/helios/pipeline_helios.py
SHYuanBest Mar 1, 2026
f0f99c1
remove redudant optimized_scale
SHYuanBest Mar 1, 2026
4e418a4
Update src/diffusers/pipelines/helios/pipeline_helios_pyramid.py
SHYuanBest Mar 1, 2026
70dbf19
use more descriptive name
SHYuanBest Mar 1, 2026
ae7817f
optimize history_latents
SHYuanBest Mar 1, 2026
dba5aac
remove not used "num_inference_steps"
SHYuanBest Mar 1, 2026
6e4d099
removed redudant "pyramid_num_stages"
SHYuanBest Mar 1, 2026
63983a7
add "is_cfg_zero_star" and "is_distilled" to HeliosPyramidPipeline
SHYuanBest Mar 1, 2026
72b8811
remove redudant
SHYuanBest Mar 1, 2026
5a64c6d
change example scripts name
SHYuanBest Mar 1, 2026
0b6a41e
change example scripts name
SHYuanBest Mar 1, 2026
11e1a9a
correct docs
SHYuanBest Mar 1, 2026
a88f60f
update example
SHYuanBest Mar 1, 2026
c4fc87b
update docs
SHYuanBest Mar 1, 2026
7236203
Update tests/models/transformers/test_models_transformer_helios.py
SHYuanBest Mar 2, 2026
f460a89
Update tests/models/transformers/test_models_transformer_helios.py
SHYuanBest Mar 2, 2026
6cccfa7
separate HeliosDMDScheduler
Mar 2, 2026
3bec7cf
fix numerical stability issue:
Mar 2, 2026
2800ac5
Update src/diffusers/schedulers/scheduling_helios_dmd.py
SHYuanBest Mar 2, 2026
e2c5ed3
Update src/diffusers/schedulers/scheduling_helios_dmd.py
SHYuanBest Mar 2, 2026
0f2c828
Update src/diffusers/schedulers/scheduling_helios_dmd.py
SHYuanBest Mar 2, 2026
9601ee1
Update src/diffusers/schedulers/scheduling_helios_dmd.py
SHYuanBest Mar 2, 2026
ae2a300
Update src/diffusers/schedulers/scheduling_helios_dmd.py
SHYuanBest Mar 2, 2026
d9515b9
remove redudant
Mar 2, 2026
6c167c0
small refactor
Mar 2, 2026
256544b
remove use_interpolate_prompt logits
Mar 2, 2026
724d6b7
simplified model test
Mar 2, 2026
604247c
fallbackt to BaseModelTesterConfig
Mar 3, 2026
8573843
remove _maybe_expand_t2v_lora_for_i2v
Mar 3, 2026
7754a66
fix HeliosLoraLoaderMixin
Mar 3, 2026
c3c7c97
update docs
Mar 3, 2026
5d9ed98
use randn_tensor for test
Mar 3, 2026
8a63b0b
fix doc typo
Mar 3, 2026
6098ebd
optimize code
Mar 3, 2026
fa927e3
mark torch.compile xfail
Mar 4, 2026
d2118ef
change paper name
SHYuanBest Mar 4, 2026
95a0408
Make get_dummy_inputs deterministic using self.generator
dg845 Mar 4, 2026
339afe5
Set less strict threshold for test_save_load_float16 test for Helios …
dg845 Mar 4, 2026
6ead7ad
make style and make quality
dg845 Mar 4, 2026
9ae6306
Merge pull request #3 from huggingface/fix-helios-tests
SHYuanBest Mar 4, 2026
e53374d
Preparation for merging
SHYuanBest Mar 4, 2026
dcbb182
add torch.Generator
SHYuanBest Mar 4, 2026
b9efc15
Fix HeliosPipelineOutput doc path
dg845 Mar 4, 2026
ab8127d
Fix Helios related (optimize docs & remove redudant) (#13210)
SHYuanBest Mar 4, 2026
b16635e
Merge branch 'main' into helios
sayakpaul Mar 4, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion docs/source/en/_toctree.yml
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,8 @@
title: Model accelerators and hardware
- isExpanded: false
sections:
- local: using-diffusers/helios
title: Helios
- local: using-diffusers/consisid
title: ConsisID
- local: using-diffusers/sdxl
Expand Down Expand Up @@ -350,6 +352,8 @@
title: FluxTransformer2DModel
- local: api/models/glm_image_transformer2d
title: GlmImageTransformer2DModel
- local: api/models/helios_transformer3d
title: HeliosTransformer3DModel
- local: api/models/hidream_image_transformer
title: HiDreamImageTransformer2DModel
- local: api/models/hunyuan_transformer2d
Expand Down Expand Up @@ -625,7 +629,6 @@
title: Image-to-image
- local: api/pipelines/stable_diffusion/inpaint
title: Inpainting

- local: api/pipelines/stable_diffusion/latent_upscale
title: Latent upscaler
- local: api/pipelines/stable_diffusion/ldm3d_diffusion
Expand Down Expand Up @@ -674,6 +677,8 @@
title: ConsisID
- local: api/pipelines/framepack
title: Framepack
- local: api/pipelines/helios
title: Helios
- local: api/pipelines/hunyuan_video
title: HunyuanVideo
- local: api/pipelines/hunyuan_video15
Expand Down Expand Up @@ -745,6 +750,10 @@
title: FlowMatchEulerDiscreteScheduler
- local: api/schedulers/flow_match_heun_discrete
title: FlowMatchHeunDiscreteScheduler
- local: api/schedulers/helios_dmd
title: HeliosDMDScheduler
- local: api/schedulers/helios
title: HeliosScheduler
- local: api/schedulers/heun
title: HeunDiscreteScheduler
- local: api/schedulers/ipndm
Expand Down
5 changes: 5 additions & 0 deletions docs/source/en/api/loaders/lora.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ LoRA is a fast and lightweight training method that inserts and trains a signifi
- [`AuraFlowLoraLoaderMixin`] provides similar functions for [AuraFlow](https://huggingface.co/fal/AuraFlow).
- [`LTXVideoLoraLoaderMixin`] provides similar functions for [LTX-Video](https://huggingface.co/docs/diffusers/main/en/api/pipelines/ltx_video).
- [`SanaLoraLoaderMixin`] provides similar functions for [Sana](https://huggingface.co/docs/diffusers/main/en/api/pipelines/sana).
- [`HeliosLoraLoaderMixin`] provides similar functions for [HunyuanVideo](https://huggingface.co/docs/diffusers/main/en/api/pipelines/helios).
- [`HunyuanVideoLoraLoaderMixin`] provides similar functions for [HunyuanVideo](https://huggingface.co/docs/diffusers/main/en/api/pipelines/hunyuan_video).
- [`Lumina2LoraLoaderMixin`] provides similar functions for [Lumina2](https://huggingface.co/docs/diffusers/main/en/api/pipelines/lumina2).
- [`WanLoraLoaderMixin`] provides similar functions for [Wan](https://huggingface.co/docs/diffusers/main/en/api/pipelines/wan).
Expand Down Expand Up @@ -86,6 +87,10 @@ LoRA is a fast and lightweight training method that inserts and trains a signifi

[[autodoc]] loaders.lora_pipeline.SanaLoraLoaderMixin

## HeliosLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.HeliosLoraLoaderMixin

## HunyuanVideoLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.HunyuanVideoLoraLoaderMixin
Expand Down
35 changes: 35 additions & 0 deletions docs/source/en/api/models/helios_transformer3d.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
<!-- Copyright 2025 The HuggingFace Team. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License. -->

# HeliosTransformer3DModel

A 14B Real-Time Autogressive Diffusion Transformer model (support T2V, I2V and V2V) for 3D video-like data from [Helios](https://github.com/PKU-YuanGroup/Helios) was introduced in [Helios: Real Real-Time Long Video Generation Model](https://huggingface.co/papers/) by Peking University & ByteDance & etc.

The model can be loaded with the following code snippet.

```python
from diffusers import HeliosTransformer3DModel

# Best Quality
transformer = HeliosTransformer3DModel.from_pretrained("BestWishYsh/Helios-Base", subfolder="transformer", torch_dtype=torch.bfloat16)
# Intermediate Weight
transformer = HeliosTransformer3DModel.from_pretrained("BestWishYsh/Helios-Mid", subfolder="transformer", torch_dtype=torch.bfloat16)
# Best Efficiency
transformer = HeliosTransformer3DModel.from_pretrained("BestWishYsh/Helios-Distilled", subfolder="transformer", torch_dtype=torch.bfloat16)
```

## HeliosTransformer3DModel

[[autodoc]] HeliosTransformer3DModel

## Transformer2DModelOutput

[[autodoc]] models.modeling_outputs.Transformer2DModelOutput
Loading
Loading