Low dimensional benchmark by JonathanSchmidt1 · Pull Request #355 · ddmms/ml-peg

JonathanSchmidt1 · 2026-02-09T08:07:02Z

Pre-review checklist for PR author

PR author must check the checkboxes below when creating the PR.

I've confirmed the contribution guidelines.

Summary

Adds relaxation of two-dimensional and one-dimensional crystal structures benchmarking energies and areas/lengths.

Linked issue

Resolves #354

Progress

Calculations
Analysis
Application
Documentation

Testing

mace-mpa-0

New decorators/callbacks

No

joehart2001 · 2026-04-15T20:19:26Z

Thanks for the PR @JonathanSchmidt1 ! some comments:

i think a good place for the data download would be in ~/cache/ml_peg/low_dimensional_relaxation, as this is where the s3 download stores the other data
i think we should add structure visualisaitont to the density scatters
can you mark the test with either slow or very slow using e.g. @pytest.mark.slow
rebase

Convergence:

i think i remember discussing this in a drop in on how to best visualise this?
one option could be a bar chart, which is being added in PR Water slab dipoles #349
but it is probably interesting to see which are the structures which fail, so maybe a scatter of energy vs area where the colour is converged or not converged? i did something like this ish for the phonon benchmakr wehre we colour the points based on which quater of the confusion matrix the point is in
or maybe we save the forces and do a violin plot with the final max force, and we have a dashed line indicated the fmax thrreshold. maybe i like this idea the most. we can plot the points so we can still visualise the structures. heres the example from the phonon benchmark BZ metric:

Initial implementation of benchmark for 2D crystal structures from Alexandria database. Uses cell mask to constrain relaxation to in-plane directions only. Metrics: - Area MAE (Å²/atom): In-plane area per atom vs PBE reference - Energy MAE (eV/atom): Energy per atom vs PBE reference - Convergence (%): Percentage of structures that converged Data source: alexandria_2d_001.json.bz2 from Alexandria database Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Add generic download_data() and load_structures() functions for both 2D/1D - Add get_length_per_atom() for 1D chain length calculation - Add relax_low_dimensional() with dimensionality parameter for cell masks - Add 1D metrics: Length MAE, Energy MAE, Convergence - Add 1D parity plots for length and energy - Parametrize test function for both 2D and 1D dimensionalities - Update app with 1D plot configurations Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Document the new low-dimensional relaxation benchmark including: - Summary of 2D and 1D structure relaxation - Metrics: Area/Length MAE, Energy MAE, and Convergence for both 2D and 1D - Description of cell masks for constrained relaxation - Data availability from Alexandria database Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add set_vacuum_padding() function that sets lattice constants to 100 Å in non-periodic directions while preserving vector directions: - 2D: sets c vector to 100 Å - 1D: sets b and c vectors to 100 Å Apply vacuum padding before relaxation in test function. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Add symmetrize_structure() using spglib.standardize_cell() - Remove FixSymmetry constraint (doesn't work for low-d relaxations) - Apply symmetrization before vacuum padding and relaxation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Update DEFAULT_DATA_FILES to use lists (2 files for 2D, 1 for 1D) - Add download_all_data() to download multiple files - Update load_structures() to pool entries from all files before sampling Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Replace 10 near-identical per-dimensionality fixtures with: - DIM_CONFIGS dict mapping dimensionality to column/label/unit - _compute_mae() and _compute_convergence() helpers - _generate_all_plots() looping over DIM_CONFIGS - Single parity_plots fixture and loop-based metrics fixture Reduces 465 lines to 353 lines with no duplication. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace custom symmetrize_structure implementation with ase.spacegroup.symmetrize.refine_symmetry. Drop spglib import. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Remove symmetrize_structure(), call refine_symmetry() directly - Add test_get_area_per_atom and test_get_length_per_atom unit tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Parallels the high-pressure PR#340 review fixes. Drops calc_kwargs={"default_dtype": "float64"} in favour of setting model.default_dtype = "float64" before model.get_calculator(), matching the pattern in calcs/surfaces/SBH17/calc_SBH17.py. Adds @pytest.mark.very_slow so the benchmark is skipped by default and only runs with --run-very-slow.

Mirrors the high-pressure benchmark's density-plot → structure viewer flow in the low-dimensional benchmark. calc: writes per-structure xyz files into outputs/{model}/{dim}/{mat_id}.xyz with atoms.info populated for ref/pred energy and geometric metric, instead of the concatenated relaxed_{dim}.xyz. Matches the pattern used by high-pressure. analyse: rebuilds get_converged_data to read per-structure xyz and copy them into app data; adds a _build_density_plot_for_condition helper that writes the plot JSON alongside sampled-structure trajectories via write_density_trajectories. Outlier handling for the convergence rate mirrors high-pressure (xyz-based subtraction of energy outliers from the converged count). app: adds collect_traj_assets + struct_from_scatter registrations per plot, and a struct-placeholder Div so clicking a point on any of the four density plots opens the underlying structure.

joehart2001 · 2026-04-30T13:51:03Z

@JonathanSchmidt1 opinions on convergence visualisation? we could also create an issue to add it in later as well. Otherwise i think we are close to merging @ElliottKasoar

…chmark # Conflicts: # docs/source/user_guide/benchmarks/bulk_crystal.rst

- Fix max-force convergence check: max(..., key=abs) returned the signed value, so a force of -5.0 satisfies < fmax. Use np.abs().max() instead. - Move downloaded Alexandria data to ~/.cache/ml_peg/low_dimensional_relaxation to match the project-wide BENCHMARK_DATA_DIR convention used by the S3 download helpers.

Clicking the "Convergence (2D)" or "Convergence (1D)" cell in the benchmark table now displays a violin plot of the final max |F| component per structure, with a dashed line at the fmax threshold. Clicking a violin sample renders the corresponding relaxed structure (converged or not), so failure modes are visible directly. Calc: relax_low_dimensional now returns the final max force and the relaxed atoms even when the relaxation does not converge; the test loop stores both in atoms.info and writes xyz files for converged and unconverged frames alike. Analysis: a new fixture pair (force_violin_2d/_1d) builds one violin figure per (model, dim) into figure_force_violin_*.json and mirrors the xyz files into per-model dirs in violin-point order. Density-scatter fixtures now skip atoms.info["converged"] == False so unconverged frames are not pulled into the parity scatters. App: VIOLIN_CONFIGS adds the violin layer to the cell-to-plot map and wires struct_from_scatter (mode="struct") for click-to-structure.

…dimensional_relaxation.py

Co-authored-by: Elliott Kasoar <45317199+ElliottKasoar@users.noreply.github.com>

ElliottKasoar added the new benchmark Proposals and suggestions for new benchmarks label Feb 9, 2026

joehart2001 reviewed Apr 15, 2026

View reviewed changes

Comment thread ml_peg/calcs/bulk_crystal/low_dimensional_relaxation/calc_low_dimensional_relaxation.py Outdated

joehart2001 reviewed Apr 15, 2026

View reviewed changes

Comment thread ml_peg/calcs/bulk_crystal/low_dimensional_relaxation/calc_low_dimensional_relaxation.py Outdated

JonathanSchmidt1 and others added 14 commits April 24, 2026 08:44

add two path

cd6e022

Use ASE refine_symmetry instead of custom spglib function

ecab2e9

Replace custom symmetrize_structure implementation with ase.spacegroup.symmetrize.refine_symmetry. Drop spglib import. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove symmetrize_structure wrapper, add geometry unit tests

63cd3a0

- Remove symmetrize_structure(), call refine_symmetry() directly - Add test_get_area_per_atom and test_get_length_per_atom unit tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Move geometry unit tests to tests/ directory

30bca66

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

add xyz storage of relaxed structures and density plots

643c630

JonathanSchmidt1 force-pushed the low_dimensional_benchmark branch from 7af3ea5 to 70d2de6 Compare April 24, 2026 10:00

joehart2001 requested a review from ElliottKasoar April 30, 2026 13:51

JonathanSchmidt1 added 3 commits April 30, 2026 14:47

Merge remote-tracking branch 'upstream/main' into low_dimensional_ben…

f3c572d

…chmark # Conflicts: # docs/source/user_guide/benchmarks/bulk_crystal.rst

ElliottKasoar reviewed Apr 30, 2026

View reviewed changes

Comment thread ml_peg/calcs/bulk_crystal/low_dimensional_relaxation/calc_low_dimensional_relaxation.py Outdated

Update ml_peg/calcs/bulk_crystal/low_dimensional_relaxation/calc_low_…

4d524ea

…dimensional_relaxation.py

ElliottKasoar previously approved these changes May 1, 2026

View reviewed changes

joehart2001 previously approved these changes May 1, 2026

View reviewed changes

ElliottKasoar reviewed May 1, 2026

View reviewed changes

Comment thread ml_peg/analysis/bulk_crystal/low_dimensional_relaxation/analyse_low_dimensional_relaxation.py Outdated

ElliottKasoar reviewed May 1, 2026

View reviewed changes

Comment thread ml_peg/app/bulk_crystal/low_dimensional_relaxation/app_low_dimensional_relaxation.py Outdated

ElliottKasoar reviewed May 1, 2026

View reviewed changes

Comment thread ml_peg/calcs/bulk_crystal/low_dimensional_relaxation/calc_low_dimensional_relaxation.py Outdated

Apply suggestions from code review

1b85778

Co-authored-by: Elliott Kasoar <45317199+ElliottKasoar@users.noreply.github.com>

ElliottKasoar dismissed stale reviews from joehart2001 and themself via 1b85778 May 1, 2026 15:19

ElliottKasoar requested a review from joehart2001 May 1, 2026 15:20

joehart2001 approved these changes May 1, 2026

View reviewed changes

ElliottKasoar merged commit 02b417e into ddmms:main May 1, 2026
7 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Low dimensional benchmark#355

Low dimensional benchmark#355
ElliottKasoar merged 19 commits intoddmms:mainfrom
JonathanSchmidt1:low_dimensional_benchmark

JonathanSchmidt1 commented Feb 9, 2026

Uh oh!

Uh oh!

Uh oh!

joehart2001 commented Apr 15, 2026

Uh oh!

joehart2001 commented Apr 30, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

JonathanSchmidt1 commented Feb 9, 2026

Pre-review checklist for PR author

Summary

Linked issue

Progress

Testing

New decorators/callbacks

Uh oh!

Uh oh!

Uh oh!

joehart2001 commented Apr 15, 2026

Uh oh!

joehart2001 commented Apr 30, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants