Add in-tree multi-repo support and torchtitan profile#9
Merged
xmfan merged 8 commits intodrisspg:mainfrom Apr 13, 2026
Merged
Conversation
xmfan
commented
Apr 8, 2026
prompts/adhoc_torchtitan.md
Outdated
| ## Debugging Tools | ||
|
|
||
| **Distributed training debugging**: | ||
| - Run with single process first: `CUDA_VISIBLE_DEVICES=0 {workspace}/jobs/{job_id}/.venv/bin/python <script.py>` |
Collaborator
Author
There was a problem hiding this comment.
this only works if the script uses fake process group. let's remove this instruction and always use torchrun
xmfan
commented
Apr 8, 2026
xmfan
commented
Apr 8, 2026
xmfan
commented
Apr 8, 2026
ptq/application/pr_service.py
Outdated
| job_dir = f"{backend.workspace}/jobs/{job_id}" | ||
| worktree = f"{job_dir}/pytorch" | ||
|
|
||
| from ptq.repo_profiles import get_profile |
Collaborator
Author
There was a problem hiding this comment.
imports at top of file
xmfan
commented
Apr 8, 2026
ptq/application/rebase_service.py
Outdated
| job_dir = f"{workspace}/jobs/{job_id}" | ||
| worktree = f"{job_dir}/pytorch" | ||
|
|
||
| from ptq.repo_profiles import get_profile |
xmfan
commented
Apr 8, 2026
ptq/cli.py
Outdated
| ] = "pytorch", | ||
| ) -> None: | ||
| """Launch an AI agent to investigate a PyTorch issue or run an adhoc task. | ||
| """Launch an AI agent to investigate a PyTorch/TorchTitan issue or run an adhoc task. |
Collaborator
Author
There was a problem hiding this comment.
let's just remove repo names from prompts
xmfan
force-pushed
the
xmfan/torchtitan
branch
3 times, most recently
from
50a894b to
643ebe2
Compare
xmfan
changed the title
Move hardcoded pytorch profile into a config-driven RepoProfile registry loaded from [repos.*] sections in config.toml. Prompt templates are discovered by naming convention. Built-in defaults used as fallback when config has no [repos] section.
- Add torchtitan profile to config.toml and _DEFAULT_PROFILES - Add investigate/adhoc prompt templates for torchtitan - Add repo field to JobRecord and RunRequest - Include repo name in job IDs to avoid cross-repo collisions - Filter find_by_issue by repo for correct re-run matching - Update agent.py and issue.py to use repo profiles
- run_service / worktree_service: repo-aware worktree and venv setup; move _setup_lightweight_venv to worktree_service - job_service / pr_service / rebase_service: top-level profile imports - cli.py: generic --repo flag, auto-reload via create_debug_app factory - workspace.py: generic _clone_repo driven by repo profiles - app.py: add create_debug_app() factory for uvicorn auto-reload
- routes.py: pass profile objects to template for dynamic repo dropdown, repo column in job list, merge-base diff, dynamic issue links - templates: iterate repos from config, repo column, dynamic issue links
xmfan
commented
Apr 9, 2026
| from ptq.application.worktree_service import provision_worktree, validate_workspace | ||
| from ptq.domain.policies import make_job_id | ||
| from ptq.infrastructure.backends import create_backend | ||
| from ptq.repo_profiles import get_profile |
Collaborator
Author
There was a problem hiding this comment.
apparently there's some circular imports
drisspg
approved these changes
Apr 9, 2026
xmfan
changed the title
xmfan
changed the title
xmfan
force-pushed
the
xmfan/torchtitan
branch
2 times, most recently
from
80dbd15 to
09538ea
Compare
config._parse() was importing repo_profiles.load_profiles_from_config, while repo_profiles._loaded_profiles() imported config.load_config. Fix: config.py now stores the raw [repos.*] TOML dict as repos_raw. repo_profiles parses it when loading profiles. One-way dependency.
ptq setup now only clones pytorch by default. Add-on repos like
torchtitan are cloned only when explicitly requested:
ptq setup <machine> --extras torchtitan
xmfan
changed the title
xmfan
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refactor hardcoded pytorch profile into a config-driven repo registry ([repos.*] in config.toml)
Add torchtitan as an add-on repo with dedicated prompts, lightweight venv setup (inherits base torch build), and per-job worktrees
Wire --repo through CLI, web UI, services, job IDs, and issue lookup
Both repos are always cloned in every workspace; --repo selects which repo an issue is filed in
Test plan