Skip to content

Merge COMPASS contents from MPAS-Dev/MPAS-Model/ocean/develop#2

Merged
xylar merged 538 commits intoMPAS-Dev:masterfrom
xylar:ocean/develop
Nov 9, 2020
Merged

Merge COMPASS contents from MPAS-Dev/MPAS-Model/ocean/develop#2
xylar merged 538 commits intoMPAS-Dev:masterfrom
xylar:ocean/develop

Conversation

@xylar
Copy link
Collaborator

@xylar xylar commented Nov 6, 2020

xylar and others added 30 commits April 7, 2020 10:37
Cells below maxLevelCell are no longer included.
This merge moves soma/4km/32to4km and soma/8km/32to8km test cases
into a new subdirectory called "broken" since these test cases are
not working and won't be fixed anytime soon.  With this change,
`./list_testcases.py` and `./setup_testcases.py` won't pick up
these tests because their driver config files aren't at the
expected directory level.
The Maine, QU60 and SOQU60to15 test cases now have the links to
the python script for defining their vertical grids that they need
to be set up successfully.
We no longer define a path to metis in the config file, so the
version from the conda environment needs to be used instead.
Add support for a "conda_mpi" attribute to "step" tags. If this
attribute is set to "true" and MPI is present in the conda
environment, that command will be called with `mpirun` from
the conda envrionment.

This is needed to support compass conda enviornments with mpich.
Python scripts and modules that use the netcdf4 package with
mpich support don't work properly on many compute nodes (e.g.
Grizzly at LANL and Anvil at ANL) unless they are prefixed with
`mpirun -np 1`
The paraview extractor can now be called as a function rather
than a script, and this is done during base-mesh generation
and culling.

SCRIP files can now also be created with a function, so a script
call is replaced with a function here as well.

With these changes, calls to python scripts that use NetCDF in
the parallel conda enviornment will now work as long as they
are called with `mpirun -np 1`
Also rename the load script for convenience.
This will make sure the compatible version of MPI gets used.
Since we can't detect automatically that this is a python script,
(and that it needs to support compass mpi) we need to say so
explicitly
This change is needed because of a recent modification (MPAS-Dev#495) to
`build_mesh.py`
set:
config_AM_mixedLayerDepths_Tgradient = .false.
config_AM_mixedLayerDepths_Dgradient = .false.

write MLD AM every time step for restart test
This is specified either in the config file or at the command
line.
…n/develop

Switch ARM creation to signed distance function MPAS-Dev#515

As a warm-up to revising the EC60to30 and CUSP mesh, I switched ARM from
the hard-wired function `AtlanticPacificGrid` to signed distance based
on a geojson file. The new steps are:
- Create cell width vs latitude for Atlantic and Pacific basins
- Expand from 1D to 2D
- Signed distance of Atlantic region
- Merge Atlantic and Pacific distrubutions smoothly
- Merge Atlantic and Pacific distrubutions with step function
- Create a land mask that is 1 over land
- Merge: step transition over land, smooth transition over water
… ocean/develop

Optionally add links to load_compass_env.sh in test cases MPAS-Dev#492

This is specified either in the config file or at the command line.

Like MPAS-Dev#480, this involves changes to common COMPASS infrastructure and we
should consider making a separate PR to develop instead of merging those
changes to ocean/develop.

closes MPAS-Dev#490
Each test case with an E3SM coupling step now needs a default and
and custom config file for that test case.  The custom config file
will be used to point to the correct initial condition, give the
full mesh name and other metadata, etc.

For now, only the 2 EC60to30 test cases have these custom config
files, but ther remaining test cases with E3SM coupling steps will
have these added later on.
This metadata is formatted according to the specs of @proteanplanet
Switch from institution to email address, which can be parsed from
git config (along with the name).

Switch metadata from having spaces to underscores.

Switch mesh short names from dot separation to underscores.
For example, `ECwISC60to30kmL64E3SMv2r01`
vanroekel and others added 13 commits September 4, 2020 21:04
Fix validation for QU240 test step MPAS-Dev#660

Folder is "test", not "forward".
…elop

Update to version 0.1.11 of the compass environment MPAS-Dev#688

This brings in several changes from MPAS-Tools:

* Changes to make sure the Earth radius is consistent throughout COMPASS
  and `mpas_tools.ocean` by using the CIME value
  (MPAS-Dev/MPAS-Tools#347 and
  MPAS-Dev/MPAS-Tools#341)

* Improved performance for interpolation of meshDensity and bathymetry
  (MPAS-Dev/MPAS-Tools#344)

* Reorganization of `mpas_tools.mesh.creation`
  (MPAS-Dev/MPAS-Tools#314)
This version has a larger region of enhanced resolution around
Greenland than EC15to60E2r01.
@xylar xylar requested a review from mark-petersen November 6, 2020 09:49
@xylar xylar self-assigned this Nov 6, 2020
@xylar xylar added the ocean label Nov 6, 2020
@xylar
Copy link
Collaborator Author

xylar commented Nov 6, 2020

@mark-petersen, I'm just hoping of something of a sanity check from you. For example, that the first commits and last commits make sense and aren't missing anything. And that we're not changing files we don't want to be. Wouldn't hurt to run a test, but maybe not critical either.

@mark-petersen
Copy link
Collaborator

@xylar this is great!!! I ran from the head of this branch. I only needed to add a tag:

git tag -m "test" testTag

and then everything worked. I tested on grizzly with compass_0.1.11:

list_testcases.py
setup_testcase.py
manage_regression_suite.py

and ran the nightly regression suite with the head of MPAS-O ocean/develop.

Only two requests.

  1. The head has four commits from New Mesh: EC15to60E2r4  MPAS-Model#740, would be nice to have those as a merge into the trunk here.

  2. All passed nightly regression suite, except these links that still had the prepended directories:

git grep -in testing_and_setup
ocean/soma/32km/default/config_analysis.xml:3:  
   <add_link source_path="mpas_model" source="testing_and_setup/compass/ocean/soma/32km/default/analysis/check_particle_sampling.py" dest="check_particle_sampling.py"/>
ocean/ziso/20km/default/config_forward.xml:8:   
   <add_link source_path="mpas_model" source="testing_and_setup/compass/ocean/scripts/LIGHTparticles/make_particle_file.py" dest="make_particles.py"/>

if you could remove the testing_and_setup/compass after the merge above.

@xylar
Copy link
Collaborator Author

xylar commented Nov 9, 2020

@mark-petersen, thanks very much for testing. I hadn't tried the ziso and soma tests, so it's great that you uncovered that issue. I'll do a separate PR to fix those, as you suggest.

I'll add the missing commits from ocean/develop right now. Not sure how those aren't included.

@xylar
Copy link
Collaborator Author

xylar commented Nov 9, 2020

@mark-petersen, I show both commits from MPAS-Dev/MPAS-Model#740 as being the last two commits in this PR. Could you tell me what you think is missing? Merge commits that don't include code changes don't get saved in the process of a git filter call as far as I can tell.

@mark-petersen
Copy link
Collaborator

This is what I see:

git log --graph --oneline -n 12 xylar/ocean/develop
* 7496357 (HEAD -> ocean/develop, tag: testTag, xylar/ocean/develop) Move high res 5 degrees further north
* f6e7b65 Add a new version of the EC15to60 mesh
* fb0ab84 Finishes removal of gm_stratification_ratio
* 8ce3ac1 Change flag names, as discussed with Luke
*   62eea73 Merge PR #688 'xylar/ocean/update_compass_env' into ocean/develop
|\
| * b07471e Update to version 0.1.11 of the compass environment
* | d5c06d5 Change block tests to partition tests
|/
*   68469eb Merge PR #660 'xylar/ocean/fix_QU240_test' into ocean/develop
|\
| * ae9a47f Fix validation for QU240 test step
* | dbceb32 Fix bathymetry links
* | aaa6f56 Fix tabbing
* | 2a17a9b Fixes formatting

I just didn't understand why the last four commits are on the trunk, rather than showing a merge into the trunk. It's not super important, just nice if it's a matter of rerunning git filter. Maybe it left out the last two merges for some reason.

@xylar
Copy link
Collaborator Author

xylar commented Nov 9, 2020

@mark-petersen, I see. There does seem to be a fairly complicated procedure here: https://stackoverflow.com/a/23901136/7728169 that would preserved the merge commits. I'm not sure I want to do that, particularly because I did not do that for master and develop. Also, it would make it hard to move other branches over from MPAS-Model in the future.

Copy link
Collaborator

@mark-petersen mark-petersen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please merge. Request above is not important enough for the trouble. Thanks!

@xylar
Copy link
Collaborator Author

xylar commented Nov 9, 2020

Thanks, @mark-petersen!

@xylar xylar merged commit fd85a8f into MPAS-Dev:master Nov 9, 2020
@xylar xylar deleted the ocean/develop branch November 9, 2020 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants