Skip to content

Replace dm-tree with optree#6225

Merged
rostan-t merged 3 commits intoNVIDIA:mainfrom
rostan-t:dmtree-to-optree
Feb 25, 2026
Merged

Replace dm-tree with optree#6225
rostan-t merged 3 commits intoNVIDIA:mainfrom
rostan-t:dmtree-to-optree

Conversation

@rostan-t
Copy link
Collaborator

Category:

Other (e.g. Documentation, Tests, Configuration)

Description:

dm-tree is not actively maintained and has known performance issues. optree is an alternative that provides a superset of the dm-tree features.

The dependency on dm-tree makes it harder to install DALI on Python 3.14 because it doesn't provide wheels and the source distribution doesn't compile. This is fixed by optree. Additionally, it supports free-threaded Python.

Additional information:

Affected modules and functionalities:

Conditionals, scalar promotion.

Key points relevant for the review:

Tests:

  • Existing tests apply
  • New tests added
    • Python tests
    • GTests
    • Benchmark
    • Other
  • N/A

Checklist

Documentation

  • Existing documentation applies
  • Documentation updated
    • Docstring
    • Doxygen
    • RST
    • Jupyter
    • Other
  • N/A

DALI team only

Requirements

  • Implements new requirements
  • Affects existing requirements
  • N/A

REQ IDs: N/A

JIRA TASK: N/A

Signed-off-by: Rostan Tabet <rtabet@nvidia.com>
@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 24, 2026

Greptile Summary

This PR migrates from dm-tree to optree for tree structure manipulation. The migration is motivated by dm-tree being unmaintained and lacking Python 3.14 support, while optree provides a superset of features plus free-threaded Python support.

Key changes:

  • Replaced tree.map_structure() with optree.tree_map() across conditionals and operator preprocessing
  • Replaced tree.assert_same_structure() with manual structure comparison using optree.tree_structure() equality checks
  • Updated all dependency files (setup.py, conda recipe, Dockerfile, test scripts)
  • Added explicit numpy dependency in Dockerfile (was previously transitive through dm-tree)
  • Updated test assertions to match optree's error message format

The changes are straightforward API replacements with equivalent functionality. All affected modules (conditionals, scalar promotion) maintain their original behavior.

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk
  • The migration from dm-tree to optree is a clean, well-executed dependency replacement. All API mappings are correct (tree.map_structureoptree.tree_map, structure comparison logic properly reimplemented), dependency files comprehensively updated, and test expectations adjusted appropriately. The addition of explicit numpy dependency prevents transitive dependency issues. No logical changes to functionality.
  • No files require special attention

Important Files Changed

Filename Overview
dali/python/nvidia/dali/_conditionals.py Replaced tree.map_structure with optree.tree_map and tree.assert_same_structure with manual structure comparison using optree.tree_structure
dali/python/nvidia/dali/ops/init.py Updated import and replaced tree.map_structure with optree.tree_map for scalar constant promotion
dali/python/setup.py.in Replaced dm-tree dependency with optree in Python package dependencies
docker/Dockerfile Replaced dm-tree with optree and explicitly added numpy (previously a transitive dependency of dm-tree)

Last reviewed commit: 8aaeb81

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

@JanuszL JanuszL self-assigned this Feb 24, 2026
@rostan-t
Copy link
Collaborator Author

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [44703456]: BUILD FAILED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [44708749]: BUILD STARTED

It was previously transitively included by dm-tree.

Signed-off-by: Rostan Tabet <rtabet@nvidia.com>
@rostan-t
Copy link
Collaborator Author

!build

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [44710400]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [44708749]: BUILD FAILED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [44710400]: BUILD FAILED

Signed-off-by: Rostan Tabet <rtabet@nvidia.com>
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

8 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

@rostan-t
Copy link
Collaborator Author

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [44788092]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [44788092]: BUILD PASSED

@rostan-t rostan-t merged commit eb55610 into NVIDIA:main Feb 25, 2026
7 checks passed
@rostan-t rostan-t deleted the dmtree-to-optree branch February 25, 2026 12:38
jantonguirao added a commit to jantonguirao/DALI that referenced this pull request Mar 2, 2026
- black/black-jupyter: 25.12.0 -> 26.1.0
- protobuf/libprotobuf/libprotobuf-static: 6.31.1 -> 6.33.5
  dm-tree blocker removed (replaced by optree in NVIDIA#6225)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
jantonguirao added a commit to jantonguirao/DALI that referenced this pull request Mar 2, 2026
- black/black-jupyter: 25.12.0 -> 26.1.0
- protobuf/libprotobuf/libprotobuf-static: 6.31.1 -> 6.33.5
  dm-tree blocker removed (replaced by optree in NVIDIA#6225)

Signed-off-by: Joaquin Anton Guirao <janton@nvidia.com>
jantonguirao added a commit to jantonguirao/DALI that referenced this pull request Mar 3, 2026
- black/black-jupyter: 25.12.0 -> 26.1.0
- protobuf/libprotobuf/libprotobuf-static: 6.31.1 -> 6.33.5
  dm-tree blocker removed (replaced by optree in NVIDIA#6225)

Signed-off-by: Joaquin Anton Guirao <janton@nvidia.com>
jantonguirao added a commit to jantonguirao/DALI that referenced this pull request Mar 6, 2026
- black/black-jupyter: 25.12.0 -> 26.1.0
- protobuf/libprotobuf/libprotobuf-static: 6.31.1 -> 6.33.5
  dm-tree blocker removed (replaced by optree in NVIDIA#6225)

Signed-off-by: Joaquin Anton Guirao <janton@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants