feat(units): add an expression parser by HaoZeke · Pull Request #173 · metatensor/metatomic

HaoZeke · 2026-03-04T11:09:57Z

Closes #154.

Replaced the per-quantity lookup tables with a Shunting-Yard expression parser
that works on arbitrary compound unit strings in the spirit of lumol.

"kJ/mol/A^2"  -->  tokenize  -->  shunting-yard  -->  AST  -->  eval
                   [kJ,/,mol,     [kJ,mol,/,         tree     {factor, dim}
                    /,A,^,2]       A,2,^,/]

Each token resolves to an SI conversion factor and a 5-element dimension vector
[L, T, M, Q, Theta]. The parser composes these through multiplication, division,
and exponentiation. Conversion factor between two expressions = ratio of their
SI factors after verifying dimension equality.

API changes

| Before (3-arg)                                  | After (2-arg)                                    |
|-------------------------------------------------+--------------------------------------------------|
| ~unit_conversion_factor("energy", "eV", "meV")~   | ~unit_conversion_factor("eV", "meV")~              |
| ~unit_conversion_factor("force", "eV/A", "eV/A")~ | ~unit_conversion_factor("eV/A", "eV/A")~           |
| Not possible                                    | ~unit_conversion_factor("(eV*u)^(1/2)", "u*A/fs")~ |

Expression syntax

Operators: * (multiply), / (divide), ^ (power), () (grouping).
Whitespace ignored. Case-insensitive. Numeric literals allowed in exponents.
Fractional exponents via parenthesized division: ^(1/2).

Token table

Single flat unordered_map with 30+ entries covering length (angstrom, bohr, nm,
m, cm, mm, um), energy (eV, meV, hartree, ry, joule, kcal, kJ), time (fs, ps),
mass (u, kg, g, electron_mass), charge (e, coulomb), dimensionless (mol), and
derived (hbar).

Notes

kelvin is NOT in the token table because temperature conversions between
offset-based scales (Celsius, Fahrenheit) are non-multiplicative.
DIM_TEMPERATURE exists as dimension [0,0,0,0,1] for potential future use but
no tokens currently carry it. (maybe once we do an API break, can revisit during mini-metatomic)

Contributor (creator of pull-request) checklist

Tests updated (for new features and bugfixes)?
Documentation updated (for new features)?
Issue referenced (for PRs that solve an issue)?

Reviewer checklist

CHANGELOG updated with public API or any other important changes?

GardevoirX · 2026-03-04T12:46:24Z

Thanks a lot! I think it would be better if we can use the new functionality to check if the quantity and unit match, when initializing ModelOutput here
https://github.com/HaoZeke/metatomic/blob/0fe0ef73e82b089811278472acba56267578b80c/metatomic-torch/include/metatomic/torch/model.hpp#L48-L61

Address PR metatensor#173 review feedback from GardevoirX: - Add s, second, ms, us, ns, ps with full-word aliases to time tokens - Add tests verifying ModelOutput rejects mismatched quantity/unit dims - Add tests for standalone micro sign (U+00B5) -> Dalton resolution - Update docs token table and doxygen with new time unit coverage - Fix stray dash in RST list-table Dimensionless row

GardevoirX

LGTM, love it!

Replace the old string-matching unit conversion with a Shunting-Yard expression parser that supports *, /, ^, and parentheses. Unit code extracted to units.hpp/cpp per review feedback. Micro sign handling uses explicit base_units entries instead of global normalization.

Remove standalone micro sign test (no longer normalizes globally), add micro sign microsecond tests instead. Update error message assertions ("unknown unit" not "unknown unit token"). Add comment explaining why models.cpp test uses valid quantity/unit strings.

Move full unit expression documentation to the Doxygen comment on unit_conversion_factor in units.hpp (renders via autofunction). Remove redundant standalone sections from misc.rst, keeping only the known-quantities table and deprecation note.

Remove validate_unit from CHANGELOG (not public API). Move unit_conversion_factor docstring from documentation.py to the Python function in __init__.py (documentation.py is only for C++ ops).

to_lower() now skips non-ASCII bytes, preventing macOS locale from mangling UTF-8 micro sign (0xB5 -> 'u'). Restore unit_conversion_factor in documentation.py since __init__.py imports it for Sphinx builds.

Luthaf · 2026-03-10T13:31:09Z



-def unit_conversion_factor(quantity: str, from_unit: str, to_unit: str):
+def unit_conversion_factor(from_unit: str, to_unit: str) -> float:


This does not need to be in this file if we re-define a Python function anyway

Still needed for the Sphinx/Type checking thing, probably could be handled better by refactoring the sphinx imports but maybe not here?

- Add cross-ref link in misc.rst docs to C++ API reference - Add [[deprecated]] attribute to 3-arg unit_conversion_factor overload - Remove units.hpp include from model.hpp, add direct includes in .cpp - Clean up AST evaluator comment (remove lumol reference) - Simplify test comment in models.cpp - Suppress deprecated warnings in register.cpp and unit tests

Use C++ [[deprecated]] attribute as requested by review. Pragma guards are isolated in wrapper functions (register.cpp, tests/units.cpp) to keep call sites clean.

Expose the 2-arg C++ op directly as metatomic.torch.unit_conversion_factor instead of a *args/**kwargs Python wrapper. This allows calling it from TorchScript models (e.g. inside AtomisticModel). The deprecated 3-arg form remains available via the C++ op torch.ops.metatomic.unit_conversion_factor for backward compat. Add TorchScript compatibility test. Update Python tests to call the 3-arg C++ op directly. Drop unnecessary comment in models.cpp.

…model.py - Add units.hpp to torch.hpp so C++ tests find the symbol - Replace torch.ops.metatomic.unit_conversion_factor_v2 calls in model.py with the module-level unit_conversion_factor (per review request)

Wrap long line in test_conversion_length_3arg to satisfy ruff. style: remove unused warnings import in units test Fixes ruff F401 (unused import) and I001 (unsorted imports). style: fix ruff import sorting in units test Ruff expects a blank line after imports. Update metatomic-torch/src/units.cpp Co-authored-by: Guillaume Fraux <luthaf@luthaf.fr> Update metatomic-torch/src/units.cpp Co-authored-by: Guillaume Fraux <luthaf@luthaf.fr>

Empty unit strings are now rejected with a clear error message instead of silently returning 1.0, which could lead to incorrect scientific results from undetected typos or misconfigurations.

Add checks for infinity and NaN results after multiplication, division, and exponentiation operations in the unit expression evaluator. This prevents silent propagation of invalid conversion factors from extreme exponents like 'angstrom^-100'.

Remove outdated 'Known quantities and units' table that implied users must pick from predefined quantities. The new expression parser accepts any valid unit expression without specifying a quantity. Added documentation covering: - Supported base tokens (length, energy, time, mass, charge, etc.) - Expression syntax (*, /, ^, parentheses, whitespace) - Examples of compound expressions (kJ/mol, eV/A^3, (eV*u)^(1/2)) The parser automatically verifies dimensional compatibility between source and target units.

- Add _known-quantities-units label for cross-references from documentation.py - Fix 'Supported base tokens' title underline (was too short)

GardevoirX

Looks much better, thanks a lot!

Replace user-facing "token" terminology with "unit"/"base unit" across docs, docstrings, C++ header comments, and error messages. Internal parser names (TokenType, tokenize) kept as implementation details. Fix kwargs backward compatibility in register.cpp by using the old 3-arg parameter names (quantity, from_unit, to_unit?) instead of generic _0, _1, _2. Rename RST label from _known-quantities-units to _known-base-units.

Add tests for version_compatible branches (major/minor mismatch), lazy __getattr__ import paths, mass/charge/derived unit conversions, and 3-arg kwargs backward compatibility. Brings project coverage above 75% threshold.

- check for full exception messages (some tests where passing by detecting a different error than the intended one) - check for float equality with ULPs as much as possible

HaoZeke requested review from GardevoirX and Luthaf March 4, 2026 11:10

HaoZeke force-pushed the feat/unit-expression-parser branch 3 times, most recently from 92bf24c to 0fe0ef7 Compare March 4, 2026 12:05

GardevoirX reviewed Mar 4, 2026

View reviewed changes

Comment thread metatomic-torch/src/model.cpp Outdated

Comment thread metatomic-torch/src/model.cpp Outdated

Comment thread python/metatomic_torch/tests/units.py

HaoZeke requested a review from GardevoirX March 4, 2026 15:26

GardevoirX reviewed Mar 4, 2026

View reviewed changes

Luthaf reviewed Mar 9, 2026

View reviewed changes

HaoZeke added 3 commits March 9, 2026 12:47

HaoZeke force-pushed the feat/unit-expression-parser branch from 01198d7 to eb8e28a Compare March 9, 2026 11:48

HaoZeke added 4 commits March 9, 2026 12:50

fix: address remaining review items

1aa7e0e

Remove validate_unit from CHANGELOG (not public API). Move unit_conversion_factor docstring from documentation.py to the Python function in __init__.py (documentation.py is only for C++ ops).

style: use literal µ instead of \u00b5 escape in Python tests

006284d

style: use literal µ instead of hex escapes in C++ sources

7eb3d41

fix: ASCII-only tolower and restore documentation.py stub

4c8ab76

to_lower() now skips non-ASCII bytes, preventing macOS locale from mangling UTF-8 micro sign (0xB5 -> 'u'). Restore unit_conversion_factor in documentation.py since __init__.py imports it for Sphinx builds.

Luthaf reviewed Mar 10, 2026

View reviewed changes

HaoZeke added 4 commits March 11, 2026 10:55

fix: add [[deprecated]] attribute to 3-arg unit_conversion_factor

71331c1

Use C++ [[deprecated]] attribute as requested by review. Pragma guards are isolated in wrapper functions (register.cpp, tests/units.cpp) to keep call sites clean.

fix: add units.hpp to umbrella header, use unit_conversion_factor in …

7ba0e25

…model.py - Add units.hpp to torch.hpp so C++ tests find the symbol - Replace torch.ops.metatomic.unit_conversion_factor_v2 calls in model.py with the module-level unit_conversion_factor (per review request)

HaoZeke force-pushed the feat/unit-expression-parser branch from 5fd5187 to 6588fa1 Compare March 12, 2026 01:00

HaoZeke force-pushed the feat/unit-expression-parser branch from 1ca61a6 to e9217ad Compare March 12, 2026 13:15

HaoZeke requested a review from Luthaf March 12, 2026 13:16

HaoZeke added 2 commits March 12, 2026 14:27

fix(units): throw error on empty unit strings instead of returning 1.0

3762bf1

Empty unit strings are now rejected with a clear error message instead of silently returning 1.0, which could lead to incorrect scientific results from undetected typos or misconfigurations.

GardevoirX reviewed Mar 12, 2026

View reviewed changes

Comment thread docs/src/torch/reference/misc.rst Outdated

HaoZeke requested a review from GardevoirX March 13, 2026 02:48

fix(docs): add missing label and fix title underline in misc.rst

37899dd

- Add _known-quantities-units label for cross-references from documentation.py - Fix 'Supported base tokens' title underline (was too short)

GardevoirX reviewed Mar 13, 2026

View reviewed changes

Luthaf reviewed Mar 13, 2026

View reviewed changes

Comment thread docs/src/torch/reference/misc.rst Outdated

Comment thread docs/src/torch/reference/misc.rst

Comment thread metatomic-torch/src/register.cpp Outdated

Comment thread metatomic-torch/tests/models.cpp

HaoZeke added 2 commits March 15, 2026 01:40

fix(docs): update all cross-refs to renamed _known-base-units label

88c0c33

HaoZeke force-pushed the feat/unit-expression-parser branch from 9e8c172 to 250b3cd Compare March 16, 2026 15:12

HaoZeke force-pushed the feat/unit-expression-parser branch from 250b3cd to ef3a1df Compare March 16, 2026 15:16

HaoZeke requested a review from Luthaf March 16, 2026 15:18

chore(pragma): swallow deprecations

c7a839b

HaoZeke force-pushed the feat/unit-expression-parser branch from 4ff62f8 to c7a839b Compare March 16, 2026 16:14

Luthaf reviewed Mar 16, 2026

View reviewed changes

Comment thread metatomic-torch/src/register.cpp Outdated

Luthaf reviewed Mar 17, 2026

View reviewed changes

Comment thread metatomic-torch/src/register.cpp Outdated

HaoZeke force-pushed the feat/unit-expression-parser branch 4 times, most recently from bc1091d to 738b6a4 Compare March 17, 2026 14:23

feat(units): more flexible argument management

10a5e2c

HaoZeke force-pushed the feat/unit-expression-parser branch from 738b6a4 to 10a5e2c Compare March 17, 2026 14:37

style: apply ruff formatting to tests/units.py

3a98e26

HaoZeke requested a review from Luthaf March 17, 2026 15:17

HaoZeke and others added 3 commits March 17, 2026 19:52

refactor(units): remove thread-local cache (YAGNI per review)

d987f48

Cleanup

39bb015

Use tigher tests

b0fcc4a

- check for full exception messages (some tests where passing by detecting a different error than the intended one) - check for float equality with ULPs as much as possible

Luthaf merged commit a59f4cc into metatensor:main Mar 18, 2026
49 checks passed

HaoZeke deleted the feat/unit-expression-parser branch March 18, 2026 15:28

GardevoirX mentioned this pull request Apr 18, 2026

Refactor unit conversion and allow additional inputs metatensor/lammps#49

Draft



		def unit_conversion_factor(quantity: str, from_unit: str, to_unit: str):
		def unit_conversion_factor(from_unit: str, to_unit: str) -> float:

Conversation

HaoZeke commented Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

API changes

Expression syntax

Token table

Notes

Contributor (creator of pull-request) checklist

Reviewer checklist

Uh oh!

GardevoirX commented Mar 4, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

GardevoirX left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Luthaf Mar 10, 2026

Choose a reason for hiding this comment

Uh oh!

HaoZeke Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

GardevoirX left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

HaoZeke commented Mar 4, 2026 •

edited

Loading