Skip to content

[v10] Development framework#3079

Merged
basfroman merged 58 commits intoSDKv10from
feat/roman/e2e-tests-framework
Oct 6, 2025
Merged

[v10] Development framework#3079
basfroman merged 58 commits intoSDKv10from
feat/roman/e2e-tests-framework

Conversation

@basfroman
Copy link
Collaborator

@basfroman basfroman commented Oct 4, 2025

Summary

This PR introduces a new Development framework for Bittensor SDK.

The framework provides a clean orchestration layer for testing and automating subnet lifecycle operations.
It abstracts low-level Subtensor extrinsics into clear, reproducible steps and now serves as the foundation for all existing E2E tests.

As a result, tests are easier to write, understand, and maintain with clearer structure, better logs, and more consistent behavior during execution.


Motivation

Until now, E2E tests relied on direct Subtensor API calls, which led to repeated code, inconsistent patterns, and limited debugging insight.
This framework consolidates those patterns into a single, reusable system built around the TestSubnet class.

TestSubnet standardizes subnet operations such as registration, activation, neuron addition, and hyperparameter updates under one API.
It introduces synchronous and asynchronous execution modes, unified extrinsic responses, and automatic NETUID handling across test runs.

The result is a much more reliable, maintainable, and developer-friendly way to test the Bittensor protocol, both for SDK devs and the broader community.


Key Advantages

  • Unified, declarative syntax for subnet orchestration.
  • Supports both single-step and multi-step execution flows.
  • Works in synchronous (execute_steps) and asynchronous (async_execute_steps) modes.
  • Automatically injects the correct NETUID for all operations within the same test instance.
  • Uses the standardized ExtrinsicResponse type for richer debugging including status, message, fees, receipts, and etc.
  • Produces structured, colorized logs that make on-chain activity and errors easy to trace.
  • Reduces code duplication and improves readability across all test suites.
  • Community-ready: can also be used by subnet operators and researchers to deploy, configure, and test subnet behavior in real or simulated environments.

Result

All E2E test suites are now built on this framework.
The codebase is simpler, easier to maintain, and much more transparent during debugging and analysis.

For full technical documentation, see bittensor/extras/dev_framework/README.md.

@basfroman basfroman self-assigned this Oct 4, 2025
@basfroman basfroman added enhancement New feature or request feature new feature added labels Oct 4, 2025
@basfroman basfroman mentioned this pull request Oct 4, 2025
@basfroman basfroman changed the title [WIP][v10]Framework for e2e testing [v10]Framework for e2e testing Oct 4, 2025
@basfroman basfroman requested a review from a team October 4, 2025 04:25
@basfroman basfroman changed the title [v10]Framework for e2e testing [v10]Development framework Oct 6, 2025
@basfroman basfroman changed the title [v10]Development framework [v10] Development framework Oct 6, 2025
basfroman and others added 9 commits October 6, 2025 12:42
Co-authored-by: BD Himes <37844818+thewhaleking@users.noreply.github.com>
Co-authored-by: BD Himes <37844818+thewhaleking@users.noreply.github.com>
Co-authored-by: BD Himes <37844818+thewhaleking@users.noreply.github.com>
Co-authored-by: BD Himes <37844818+thewhaleking@users.noreply.github.com>
Copy link
Contributor

@thewhaleking thewhaleking left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like this, but I think the structure could use some work. Namely having the generated code be gitignored. If you're a user generating methods, you are going to run into merge conflicts when you just want to git pull.

@basfroman basfroman merged commit 278a209 into SDKv10 Oct 6, 2025
692 of 693 checks passed
@basfroman basfroman deleted the feat/roman/e2e-tests-framework branch October 6, 2025 22:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request feature new feature added

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants