[RFC] Deprecate TorchTune service in favor of Unsloth/Megatron #514
Description
Summary
This RFC proposes removing the TorchTune training backend (src/art/torchtune/) and focusing development efforts on Unsloth and Megatron backends.
Background
TorchTune (from Meta/PyTorch) was integrated as a distributed full-finetuning backend. However, the library has shown signs of stagnation:
- Last PyPI release: v0.6.1 (April 2025)
- Last significant updates: May 2025
- No releases in ~9 months as of January 2026
In the fast-moving LLM fine-tuning space, this lack of updates is concerning. Meanwhile, alternatives like Unsloth continue to receive frequent updates with support for new models and optimizations.
Current State
ART currently supports three training backends:
- UnslothService - Default backend, actively maintained upstream
- TinkerService - Alternative backend
- TorchtuneService - Distributed full-finetuning (proposed for removal)
The TorchTune service code is located in:
src/art/torchtune/service.pysrc/art/torchtune/recipe.pysrc/art/torchtune/config.pysrc/art/torchtune/batch.pysrc/art/dev/torchtune.py
Proposal
Remove the TorchTune service and focus on Unsloth for single-node training and Megatron for distributed training.
Rationale
- Reduce maintenance burden of a stagnating dependency
- Focus engineering effort on actively maintained alternatives
- Unsloth provides excellent single-GPU/node performance
- Megatron-LM is battle-tested for large-scale distributed training
Future Direction
We are currently working on an experimental MegatronService which may be an appropriate path for full fine-tuning (FFT) in the future.
If you are currently using the TorchTune backend, please comment on this issue and let us know your use case. Otherwise, we will remove it in a near-future release.
References
- TorchTune GitHub: https://github.com/pytorch/torchtune
- TorchTune PyPI: https://pypi.org/project/torchtune/