Release 3.1.3 by dylanuys · Pull Request #237 · BitMind-AI/bitmind-subnet

dylanuys · 2025-07-08T22:22:14Z

This release includes a fix for logging predicted masks from segmentation miners to w&b

Previously was logging these as raw arrays, but w&b converts these to histograms to optimize storage
Now breaking out the results.predictions field into individual results.predictions_<uid> fields, and storing predicted masks as wandb.Image. Not able to store these in a list, hence the necessity to break the list out into one field per miner prediction.
Also cleaned up logs, printing mask statistics (shape, min, max) for segmentation challenge results rather than the mask itself.

This release also includes a new testnet feature that allows miners to compute approximate IoU scores by sending the ground truth mask as a b64 image in the request headers. To avoid size limitations, the mask is downsized to 128x128. The example code in neurons/miner/segmentation_miner.py demonstrates how to decode and resize the mask to compute IoU.

* V3 * removing v2 ci pipeline * removing outdated .gitmodules * keeping the noise that sample size of 50 and slight decay of .5 in EMA provides to avoid having any one model completely dominate the subnet. * release 2.2.6 datasets, models, and lora support (#188) * deprecate stable-diffusion-inpainting * .env templates * V3/RGB (#191) * bgr images --> rgb images * proper BGR -> RGB conversion * eradicate all usage of bgr in image challenge flow * extract frames as rgb * skip extraneous rgb conversion * fix deeperforesnics consistency * v2 frame sampling parity + eidon mp4 fix * missing import * handling improper reporting of fps in wembs * correct content-type on miner side * max_fpx setting * improved video metadata extraction * cleaning up ffprobe options * fixing first frame rotation edge case * i2i fix --------- Co-authored-by: Dylan Uys <dylan@bitmind.ai> * V3 frame extraction (#192) * bgr images --> rgb images * proper BGR -> RGB conversion * eradicate all usage of bgr in image challenge flow * extract frames as rgb * skip extraneous rgb conversion * fix deeperforesnics consistency * v2 frame sampling parity + eidon mp4 fix * missing import * handling improper reporting of fps in wembs * correct content-type on miner side * max_fpx setting * improved video metadata extraction * cleaning up ffprobe options * fixing first frame rotation edge case * i2i fix * frame extraction --------- Co-authored-by: Dylan Uys <dylan@bitmind.ai> * setup.sh * removing wandb log call from generator * V3/2.2.9 (#189) * mugshot dataset * black * i2v support and fixed prompt motion enhancement * gen pipeline updates for i2v * fixing prompt indexing * properly handling new prompt dictionary key (task type) * V3/2.2.11 (#190) * mugshot dataset * black * i2v support and fixed prompt motion enhancement * gen pipeline updates for i2v * prompt sanitation + i2v model * more retries for prompt sanitation * fixing truthy tuple assertion * Update min_compute.yml * fixing setup script name in docs * correct script name * updated requirements.txt with bittensor-cli * removing wandb.off * import cleanup * miner substrate thread restart + vali autoupdate test * temporary v3 branch set to test autoudpate * autoupdate update * lower frequency of audoupdate check * autoudpate test * check autoupdate at setp 0 * typo * autoupdate test * dont set weights immediately at startup in case of many restarts * Pyproject toml (#193) * pyproject setup * executable setup.sh * autoupdate test * resetting version after autoupdate tests * Add Hugging Face model access instructions to validator docs; improve logging and fix LLM device mapping for multi-GPU - Added section to Validating.md with instructions for gaining access to required Hugging Face models (FLUX.1-dev, DeepFloyd IF). - Added logging of generation arguments in generation_pipeline.py. - Fix LLM loading for multi-GPU in prompt_generator.py: use device_map and remove .to(self.device) for quantized models. Quantized LLMs must use device_map for correct device placement; calling .to(self.device) causes device mismatch errors. Parse GPU ID from device string for device_map assignment. * fixing image_samples check for i2i * hf_xet requirement * wandb autorestart * Fix: raise error if image is None for i2i/i2v tasks and ensure image is converted from array * fixing wandb autorestart * error log * Update setup.sh to install Node.js 20.x LTS from NodeSource for pm2 compatibility; add doc note for existing validators' Hugging Face access * external port for proxy cuz tensordock rugged us (#196) * incentive doc * Typo * proxy updates * v2 parity encoding (#197) * final autoupdate test * reset version --------- Co-authored-by: Benjamin S Liang <caliangben@gmail.com> Co-authored-by: Dylan Uys <dylan@bitmind.ai>

* fixing wandb cache clean paths (#202) * max frames configuration * fn header update * slight increase to timeout

* adding extra metadata to testnet requests for miners * adding label and mediatype to testnet metadata

* log augmentation params * braindead typo

* fix hotkey check in sync_metagraph * bump version

* healthcheck wip * remove old miner health task vars, change miner healtchechk endpoint name * health count logging * fixing query based on health logic * updating healthcheck interval to 10 * removing unecessary lock * use DEFAULT_TIMEOUT * adding blacklisting for bad responses * fixing detect_image fallback * a couple comments * update functionality for generator and proxy

* adding basic health dict to miner tracker * move all request processing logic to epistula module * reflect request processing updates in eval engine * healthy/unhealthy miner uid functions * proxy using simpler miner health from tracker state * new fn signature for score_challenge

* scraper wip * fixing queries with max date set in tbs, also adding placeholder for reverse image search which i cant get to work rn due to captchas * taking first sentence of prompt as initial version of search queries * - Adding specific media scraping interval config - Adding retry logic and error handling to scraping callback * Fixing enum value for output path in scraper * add selenium to requirements * Fixing str treated as enum * cleaning up * increasing media update intervals * [testnet] Safer Miner Prediction History (#214) * centralizing logic for safely getting valid predictions and associated labels * cleaning up

* adding bm-aura-imagegen dataset * log mask as npy artifact * format

* binary image endpoint * adding video endpoint without preprocessing (#221) * fix frames list (should be np array)

* segmentation v1 * remove debug logging * legacy detector backward compatibility * initial reward split for segmentation release * typo fix and increasing window size --------- Co-authored-by: Benjamin S Liang <caliangben@gmail.com> Co-authored-by: Dylan Uys <dylan@bitmind.ai>

* LLM-based query generation + moving scraping to generator class * Release 3.0.11 (#223) * V3 (#187) * V3 * removing v2 ci pipeline * removing outdated .gitmodules * keeping the noise that sample size of 50 and slight decay of .5 in EMA provides to avoid having any one model completely dominate the subnet. * release 2.2.6 datasets, models, and lora support (#188) * deprecate stable-diffusion-inpainting * .env templates * V3/RGB (#191) * bgr images --> rgb images * proper BGR -> RGB conversion * eradicate all usage of bgr in image challenge flow * extract frames as rgb * skip extraneous rgb conversion * fix deeperforesnics consistency * v2 frame sampling parity + eidon mp4 fix * missing import * handling improper reporting of fps in wembs * correct content-type on miner side * max_fpx setting * improved video metadata extraction * cleaning up ffprobe options * fixing first frame rotation edge case * i2i fix --------- Co-authored-by: Dylan Uys <dylan@bitmind.ai> * V3 frame extraction (#192) * bgr images --> rgb images * proper BGR -> RGB conversion * eradicate all usage of bgr in image challenge flow * extract frames as rgb * skip extraneous rgb conversion * fix deeperforesnics consistency * v2 frame sampling parity + eidon mp4 fix * missing import * handling improper reporting of fps in wembs * correct content-type on miner side * max_fpx setting * improved video metadata extraction * cleaning up ffprobe options * fixing first frame rotation edge case * i2i fix * frame extraction --------- Co-authored-by: Dylan Uys <dylan@bitmind.ai> * setup.sh * removing wandb log call from generator * V3/2.2.9 (#189) * mugshot dataset * black * i2v support and fixed prompt motion enhancement * gen pipeline updates for i2v * fixing prompt indexing * properly handling new prompt dictionary key (task type) * V3/2.2.11 (#190) * mugshot dataset * black * i2v support and fixed prompt motion enhancement * gen pipeline updates for i2v * prompt sanitation + i2v model * more retries for prompt sanitation * fixing truthy tuple assertion * Update min_compute.yml * fixing setup script name in docs * correct script name * updated requirements.txt with bittensor-cli * removing wandb.off * import cleanup * miner substrate thread restart + vali autoupdate test * temporary v3 branch set to test autoudpate * autoupdate update * lower frequency of audoupdate check * autoudpate test * check autoupdate at setp 0 * typo * autoupdate test * dont set weights immediately at startup in case of many restarts * Pyproject toml (#193) * pyproject setup * executable setup.sh * autoupdate test * resetting version after autoupdate tests * Add Hugging Face model access instructions to validator docs; improve logging and fix LLM device mapping for multi-GPU - Added section to Validating.md with instructions for gaining access to required Hugging Face models (FLUX.1-dev, DeepFloyd IF). - Added logging of generation arguments in generation_pipeline.py. - Fix LLM loading for multi-GPU in prompt_generator.py: use device_map and remove .to(self.device) for quantized models. Quantized LLMs must use device_map for correct device placement; calling .to(self.device) causes device mismatch errors. Parse GPU ID from device string for device_map assignment. * fixing image_samples check for i2i * hf_xet requirement * wandb autorestart * Fix: raise error if image is None for i2i/i2v tasks and ensure image is converted from array * fixing wandb autorestart * error log * Update setup.sh to install Node.js 20.x LTS from NodeSource for pm2 compatibility; add doc note for existing validators' Hugging Face access * external port for proxy cuz tensordock rugged us (#196) * incentive doc * Typo * proxy updates * v2 parity encoding (#197) * final autoupdate test * reset version --------- Co-authored-by: Benjamin S Liang <caliangben@gmail.com> Co-authored-by: Dylan Uys <dylan@bitmind.ai> * autoupdate set to main * testing autoupdate on testnet * autoupdate enabled by default * autoudpate testnet * pointing autoupdate at main by default * removing extra state load command * setting back to 360 epoch length * burn for initial v3 release rampup * debug log typo * fixed merge to testnet * Max Frames and Timeout (#203) * fixing wandb cache clean paths (#202) * max frames configuration * fn header update * slight increase to timeout * adding extra metadata to testnet requests for miners (#201) * remove max size arg * Testnet Metadata (#204) * adding extra metadata to testnet requests for miners * adding label and mediatype to testnet metadata * Log Augmentation Parameters (#205) * log augmentation params * braindead typo * bump verison * [testnet] Release 3.0.5 (#207) * fix hotkey check in sync_metagraph * bump version * [testnet] Miner healthchecks (#209) * healthcheck wip * remove old miner health task vars, change miner healtchechk endpoint name * health count logging * fixing query based on health logic * updating healthcheck interval to 10 * removing unecessary lock * use DEFAULT_TIMEOUT * adding blacklisting for bad responses * fixing detect_image fallback * a couple comments * update functionality for generator and proxy * [testnet] Revised Miner Healthcheck (#210) * adding basic health dict to miner tracker * move all request processing logic to epistula module * reflect request processing updates in eval engine * healthy/unhealthy miner uid functions * proxy using simpler miner health from tracker state * new fn signature for score_challenge * [testnet] Image Scraping (#213) * scraper wip * fixing queries with max date set in tbs, also adding placeholder for reverse image search which i cant get to work rn due to captchas * taking first sentence of prompt as initial version of search queries * - Adding specific media scraping interval config - Adding retry logic and error handling to scraping callback * Fixing enum value for output path in scraper * add selenium to requirements * Fixing str treated as enum * cleaning up * increasing media update intervals * [testnet] Safer Miner Prediction History (#214) * centralizing logic for safely getting valid predictions and associated labels * cleaning up * Aura Dataset and Mask W&B Logging (#217) * adding bm-aura-imagegen dataset * log mask as npy artifact * format * increasing media cache refresh default * disabling scrape_new_media_on_interval * warnings for missing keys in miner history state loading * lowering media scraping default (not currenlty used) * remvoning unused healthcheck endpoint in miner * [testnet] Binary Image Proxy Endpoint (#220) * binary image endpoint * adding video endpoint without preprocessing (#221) * fix frames list (should be np array) * adding wan2.1-t2v-1.3B (#222) * removing scraping from this release * version bump * whitespace * putting peft requirement back * ftfy requirement * refactor(models): make Wan2.1 VAE loading lazy - Convert VAE loading to use lazy tuple pattern (fn, args) - Update load_vae to use kwargs for consistency * Remove duplicate function * fix torch_dtype typo for wan * extending window to 200 * imports --------- Co-authored-by: Benjamin S Liang <caliangben@gmail.com> Co-authored-by: Dylan Uys <dylan@bitmind.ai> * Release/3.0.11 (#224) * Release 3.0.11 (#225) * updating config to allow toggling generation/scraping on and off * updated scraper to get native resolution images * override aug level * native resolution wip * Release 3.1.0 (#227) * V3 (#187) * V3 * removing v2 ci pipeline * removing outdated .gitmodules * keeping the noise that sample size of 50 and slight decay of .5 in EMA provides to avoid having any one model completely dominate the subnet. * release 2.2.6 datasets, models, and lora support (#188) * deprecate stable-diffusion-inpainting * .env templates * V3/RGB (#191) * bgr images --> rgb images * proper BGR -> RGB conversion * eradicate all usage of bgr in image challenge flow * extract frames as rgb * skip extraneous rgb conversion * fix deeperforesnics consistency * v2 frame sampling parity + eidon mp4 fix * missing import * handling improper reporting of fps in wembs * correct content-type on miner side * max_fpx setting * improved video metadata extraction * cleaning up ffprobe options * fixing first frame rotation edge case * i2i fix --------- Co-authored-by: Dylan Uys <dylan@bitmind.ai> * V3 frame extraction (#192) * bgr images --> rgb images * proper BGR -> RGB conversion * eradicate all usage of bgr in image challenge flow * extract frames as rgb * skip extraneous rgb conversion * fix deeperforesnics consistency * v2 frame sampling parity + eidon mp4 fix * missing import * handling improper reporting of fps in wembs * correct content-type on miner side * max_fpx setting * improved video metadata extraction * cleaning up ffprobe options * fixing first frame rotation edge case * i2i fix * frame extraction --------- Co-authored-by: Dylan Uys <dylan@bitmind.ai> * setup.sh * removing wandb log call from generator * V3/2.2.9 (#189) * mugshot dataset * black * i2v support and fixed prompt motion enhancement * gen pipeline updates for i2v * fixing prompt indexing * properly handling new prompt dictionary key (task type) * V3/2.2.11 (#190) * mugshot dataset * black * i2v support and fixed prompt motion enhancement * gen pipeline updates for i2v * prompt sanitation + i2v model * more retries for prompt sanitation * fixing truthy tuple assertion * Update min_compute.yml * fixing setup script name in docs * correct script name * updated requirements.txt with bittensor-cli * removing wandb.off * import cleanup * miner substrate thread restart + vali autoupdate test * temporary v3 branch set to test autoudpate * autoupdate update * lower frequency of audoupdate check * autoudpate test * check autoupdate at setp 0 * typo * autoupdate test * dont set weights immediately at startup in case of many restarts * Pyproject toml (#193) * pyproject setup * executable setup.sh * autoupdate test * resetting version after autoupdate tests * Add Hugging Face model access instructions to validator docs; improve logging and fix LLM device mapping for multi-GPU - Added section to Validating.md with instructions for gaining access to required Hugging Face models (FLUX.1-dev, DeepFloyd IF). - Added logging of generation arguments in generation_pipeline.py. - Fix LLM loading for multi-GPU in prompt_generator.py: use device_map and remove .to(self.device) for quantized models. Quantized LLMs must use device_map for correct device placement; calling .to(self.device) causes device mismatch errors. Parse GPU ID from device string for device_map assignment. * fixing image_samples check for i2i * hf_xet requirement * wandb autorestart * Fix: raise error if image is None for i2i/i2v tasks and ensure image is converted from array * fixing wandb autorestart * error log * Update setup.sh to install Node.js 20.x LTS from NodeSource for pm2 compatibility; add doc note for existing validators' Hugging Face access * external port for proxy cuz tensordock rugged us (#196) * incentive doc * Typo * proxy updates * v2 parity encoding (#197) * final autoupdate test * reset version --------- Co-authored-by: Benjamin S Liang <caliangben@gmail.com> Co-authored-by: Dylan Uys <dylan@bitmind.ai> * autoupdate set to main * testing autoupdate on testnet * autoupdate enabled by default * autoudpate testnet * pointing autoupdate at main by default * removing extra state load command * setting back to 360 epoch length * burn for initial v3 release rampup * debug log typo * fixed merge to testnet * Max Frames and Timeout (#203) * fixing wandb cache clean paths (#202) * max frames configuration * fn header update * slight increase to timeout * adding extra metadata to testnet requests for miners (#201) * remove max size arg * Testnet Metadata (#204) * adding extra metadata to testnet requests for miners * adding label and mediatype to testnet metadata * Log Augmentation Parameters (#205) * log augmentation params * braindead typo * bump verison * [testnet] Release 3.0.5 (#207) * fix hotkey check in sync_metagraph * bump version * [testnet] Miner healthchecks (#209) * healthcheck wip * remove old miner health task vars, change miner healtchechk endpoint name * health count logging * fixing query based on health logic * updating healthcheck interval to 10 * removing unecessary lock * use DEFAULT_TIMEOUT * adding blacklisting for bad responses * fixing detect_image fallback * a couple comments * update functionality for generator and proxy * [testnet] Revised Miner Healthcheck (#210) * adding basic health dict to miner tracker * move all request processing logic to epistula module * reflect request processing updates in eval engine * healthy/unhealthy miner uid functions * proxy using simpler miner health from tracker state * new fn signature for score_challenge * [testnet] Image Scraping (#213) * scraper wip * fixing queries with max date set in tbs, also adding placeholder for reverse image search which i cant get to work rn due to captchas * taking first sentence of prompt as initial version of search queries * - Adding specific media scraping interval config - Adding retry logic and error handling to scraping callback * Fixing enum value for output path in scraper * add selenium to requirements * Fixing str treated as enum * cleaning up * increasing media update intervals * [testnet] Safer Miner Prediction History (#214) * centralizing logic for safely getting valid predictions and associated labels * cleaning up * Aura Dataset and Mask W&B Logging (#217) * adding bm-aura-imagegen dataset * log mask as npy artifact * format * increasing media cache refresh default * disabling scrape_new_media_on_interval * warnings for missing keys in miner history state loading * lowering media scraping default (not currenlty used) * remvoning unused healthcheck endpoint in miner * [testnet] Binary Image Proxy Endpoint (#220) * binary image endpoint * adding video endpoint without preprocessing (#221) * fix frames list (should be np array) * adding wan2.1-t2v-1.3B (#222) * [testnet] Segmentation Challenges (#226) * segmentation v1 * remove debug logging * legacy detector backward compatibility * initial reward split for segmentation release * typo fix and increasing window size --------- Co-authored-by: Benjamin S Liang <caliangben@gmail.com> Co-authored-by: Dylan Uys <dylan@bitmind.ai> * removing scraper for current release * whitespace * fix bug in miner history resetting when a detector changes to a detector without dereg * limit uids * cleanup * proxy updates --------- Co-authored-by: Benjamin S Liang <caliangben@gmail.com> Co-authored-by: Dylan Uys <dylan@bitmind.ai> * Patch/autoupdate minor version (#229) * stamina for scraping * cleanup --------- Co-authored-by: Benjamin S Liang <caliangben@gmail.com> Co-authored-by: Dylan Uys <dylan@bitmind.ai>

* wandb logging fix for masks * exclude mask in detector challenge headers * return empty lists when no predictions received

benliang99

LGTM! Feedback given on Slack

…into release/3.1.3

dylanuys and others added 30 commits May 14, 2025 12:19

autoupdate set to main

99e3768

testing autoupdate on testnet

20af125

autoupdate enabled by default

7e4a5db

autoudpate testnet

1af284c

pointing autoupdate at main by default

08f816e

removing extra state load command

4c88d41

setting back to 360 epoch length

413e2f3

burn for initial v3 release rampup

b5ebae5

debug log typo

302b5f1

merging 3.0.2 into testnet

8df5eaa

fixed merge to testnet

5e9a480

Max Frames and Timeout (#203)

7bacb6e

* fixing wandb cache clean paths (#202) * max frames configuration * fn header update * slight increase to timeout

adding extra metadata to testnet requests for miners (#201)

eef4cfe

remove max size arg

0f369bb

Merge branch 'main' into testnet

f9be715

Testnet Metadata (#204)

35c81c2

* adding extra metadata to testnet requests for miners * adding label and mediatype to testnet metadata

Log Augmentation Parameters (#205)

d64eadb

* log augmentation params * braindead typo

bump verison

ce4e69d

[testnet] Release 3.0.5 (#207)

abfd4b2

* fix hotkey check in sync_metagraph * bump version

merging main

35020e5

merging main

20a60db

Aura Dataset and Mask W&B Logging (#217)

13cec30

* adding bm-aura-imagegen dataset * log mask as npy artifact * format

increasing media cache refresh default

9ff4c02

disabling scrape_new_media_on_interval

37c9981

warnings for missing keys in miner history state loading

0502723

lowering media scraping default (not currenlty used)

bc4021d

dylanuys and others added 20 commits June 6, 2025 15:41

merging main

c652972

[testnet] Binary Image Proxy Endpoint (#220)

c261712

* binary image endpoint * adding video endpoint without preprocessing (#221) * fix frames list (should be np array)

adding wan2.1-t2v-1.3B (#222)

50061f2

import and timeout

edaa87f

add mask in headers on testnet (#231)

c1a57c2

merging main

47f9a96

fixing mask getting filtered out of headers

48f1b8d

Merge branch 'main' into testnet

24bd01d

send mask as 128x128 in header

e1c9ccc

fixing check to include mask for seg challenges

acc7ced

[testnet] wandb mask logging fix (#236)

dbbf0dc

* wandb logging fix for masks * exclude mask in detector challenge headers * return empty lists when no predictions received

bump version

f02399f

removing scraping prior to release 3.2.0

e331073

version

9b59616

rolling back prompt gen and transforms

9c779a5

compute iou on testnet

f6e8a5b

fixing example segmentation miner testnet mask handling

c80edd9

removing 3.2.0 reqs

b54dc81

benliang99 self-assigned this Jul 8, 2025

benliang99 approved these changes Jul 8, 2025

View reviewed changes

dylanuys added 6 commits July 8, 2025 23:38

reload -> restart

ea7183c

neurotic autoupdate test

b8a5666

trigger autoupdate

91077c3

Merge branch 'release/3.1.3' of github.com:BitMind-AI/bitmind-subnet …

4a32599

…into release/3.1.3

resetting after autoupdate test

34bfdd9

point autoupdate back at main

dee3b97

dylanuys merged commit 8df4395 into main Jul 9, 2025

dylanuys deleted the release/3.1.3 branch January 31, 2026 01:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release 3.1.3#237

Release 3.1.3#237
dylanuys merged 57 commits intomainfrom
release/3.1.3

dylanuys commented Jul 8, 2025 •

edited

Loading

Uh oh!

benliang99 left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

dylanuys commented Jul 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

benliang99 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dylanuys commented Jul 8, 2025 •

edited

Loading