fix: [DYN-2883] Triton backend directory default

[nealvaidya]

Summary

• Set BACKEND_DIR=/opt/tritonserver/backends in the Triton worker container image.
• Let launch/identity.sh honor BACKEND_DIR from the environment while preserving the local source-build fallback.
• Document the container/local backend directory behavior in the Triton example README.

Fixes DYN-2883

Root Cause

The container image copies prebuilt Triton backends into /opt/tritonserver/backends, but the launch script always defaulted to examples/backends/tritonserver/backends, which is only populated by the local make all workflow. The container quick-start therefore exited before the worker could start.

Validation

• bash -n examples/backends/tritonserver/launch/identity.sh
• BACKEND_DIR=/opt/tritonserver/backends bash examples/backends/tritonserver/launch/identity.sh --help
• BACKEND_DIR=/tmp/dynamo-missing-backends bash examples/backends/tritonserver/launch/identity.sh confirmed the override path is used for validation
• git show --check --stat --oneline HEAD

Did not run the full Docker/GPU launch locally.

Summary by CodeRabbit

• Documentation

  • Added configuration documentation for the BACKEND_DIR environment variable to specify Tritonserver backends directory location.

• Chores

  • Updated configuration files to support the BACKEND_DIR environment variable with default path /opt/tritonserver/backends.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[nealvaidya]

/ok to test aaec7c9

[github-actions]

🌿 Fern Docs Preview: https://nvidia-preview-9af9065d-87c1-4bec-bcd4-001a4d22e3b0.docs.buildwithfern.com/dynamo/dev

[coderabbitai]

Walkthrough

Introduces a BACKEND_DIR environment variable across the Tritonserver backend examples. The variable is declared in the Dockerfile, documented in README, and utilized in launch scripts to allow explicit backends path referencing with appropriate defaults for containerized and local deployments.

Changes

Cohort / File(s)	Summary
BACKEND_DIR Environment Variable examples/backends/tritonserver/Dockerfile, examples/backends/tritonserver/README.md, examples/backends/tritonserver/launch/identity.sh	Adds BACKEND_DIR environment variable declaration in Dockerfile, documents default values in README, and updates identity.sh script to honor the variable with parameter expansion fallback and help text.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely summarizes the main change: fixing the Triton backend directory default configuration.
Description check	✅ Passed	The description includes all required template sections (Overview/Summary, Details, Related Issues) with comprehensive information about changes, root cause, and validation.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

examples/backends/tritonserver/launch/identity.sh (1)

1-130: Consider adopting shared launch-script patterns for consistency.

The script does not follow the launch-script conventions outlined in the coding guidelines. Consider refactoring to:

1. Source ../../../common/launch_utils.sh and ../../../common/gpu_utils.sh for shared utilities
2. Background the worker process (lines 123-129) and use wait_any_exit instead of foreground execution
3. Replace the custom banner (lines 98-104) with print_launch_banner
4. Replace sleep-based readiness (line 119) with a proper readiness check

These changes would align the script with the repository's standard launch patterns used by other backends (vLLM, TRT-LLM, SGLang).

As per coding guidelines: "For any script under examples/backends/*/launch (including identity.sh), ensure it follows the shared launch-script conventions: source ../../../common/gpu_utils.sh and ../../../common/launch_utils.sh instead of reimplementing process-management; background all launched processes and use wait_any_exit (not bare wait or foreground execution); emit a consistent startup banner via print_launch_banner; avoid relying on sleep-based readiness."

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@examples/backends/tritonserver/launch/identity.sh` around lines 1 - 130,
Refactor the script to follow shared launch-script conventions by sourcing
../../../common/launch_utils.sh and ../../../common/gpu_utils.sh at the top,
replace the custom banner block with a call to print_launch_banner, background
the Triton worker (start tritonworker.py with & and capture its PID instead of
running it in the foreground), use wait_any_exit to wait for any background
process to exit (instead of relying on trap + foreground wait), and replace the
fixed sleep readiness delay with the repository standard readiness check helper
(from launch_utils.sh) to detect when the frontend is ready before starting the
worker; update references to FRONTEND_PID/TRITON_PID accordingly.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@examples/backends/tritonserver/launch/identity.sh`:
- Around line 1-130: Refactor the script to follow shared launch-script
conventions by sourcing ../../../common/launch_utils.sh and
../../../common/gpu_utils.sh at the top, replace the custom banner block with a
call to print_launch_banner, background the Triton worker (start tritonworker.py
with & and capture its PID instead of running it in the foreground), use
wait_any_exit to wait for any background process to exit (instead of relying on
trap + foreground wait), and replace the fixed sleep readiness delay with the
repository standard readiness check helper (from launch_utils.sh) to detect when
the frontend is ready before starting the worker; update references to
FRONTEND_PID/TRITON_PID accordingly.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 675b9bb8-57bb-49b9-9388-5b60ab1ec810

📥 Commits

Reviewing files that changed from the base of the PR and between 3b8a384 and aaec7c9.

📒 Files selected for processing (3)

• examples/backends/tritonserver/Dockerfile
• examples/backends/tritonserver/README.md
• examples/backends/tritonserver/launch/identity.sh