Conversation
Added DeepFloyd/IF-I-XL + IF-II-L model configuration, pipeline_stages configuration for multi-stage models
…multistage pipeline i/o
…ng for video models
…o model_utils, cleaned up configs
dylanuys
reviewed
Feb 3, 2025
| raise NotImplementedError(f"Unsupported prompt type: {self.prompt_type}") | ||
| return prompt | ||
|
|
||
| def create_pipeline_generator(self, model_config: Dict[str, Any], model: Any) -> callable: |
Contributor
There was a problem hiding this comment.
Suggestion: This looks like it might belong in a model_utils.py or a similar file
Contributor
Author
There was a problem hiding this comment.
Agree! Moved to model utils
dylanuys
reviewed
Feb 3, 2025
| self.model[stage_name].set_progress_bar_config(disable=True) | ||
|
|
||
| # Apply CPU offloading if enabled | ||
| if model_config.get('enable_model_cpu_offload', False): |
Contributor
There was a problem hiding this comment.
Suggestion: Pull out enabling cpu offload, vae tiling, etc. into its own utility function like:
def enable_optimizations(model, cpu_offload: bool, vae_tiling: bool, etc...)
and use this for both single and multistage pipeline setup
Contributor
Author
There was a problem hiding this comment.
Agree here as well, moved to model utils to keep synthetic_data_generator lean
dylanuys
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR; Added support for two new text-to-image (t2i) models in validator challenges
This expands our total t2i generator pool from 6 to 8 unique synthetic sources (excluding image-to-image). We predict this will raise the challenge difficulty for miners, as models must generalize to a wider variety of generation methods to stay performant.
Action Required for Validators: Both components of the DeepFloyd pipeline are gated on HuggingFace and require that validators visit the model repos to gain access. A new install requirement is also needed to run Janus image generation without errors.
Supporting groundwork includes:
pipeline_stagesstructure inbitmind/validator/config.pywith stage-specificpipeline_clsandfrom_pretrained_argsmodel_utils.py, consistent with previously added custom load functions for Hunyuan Video and Animate DiffMisc.
run_data_generator.pyandrun_cache_updater.py