MPAS with Non-Cray Compiler

[tyatharva]

Hello,

Thank you for providing the MPAS setup in this repository. I think this is one of the only atmospheric models that can run on AMD GPUs!
I noticed that the current configuration is tailored to Cray systems and requires access to Cray compilers and runtime environments.

I’d like to ask:
Is it possible to build and run the MPAS configuration here using a different compiler—specifically one that supports OpenMP GPU offloading but is not tied to Cray environments?

More concretely, could the MPAS setup in this repo potentially work with either of the following compilers?

• AOMP
• LLVM Flang
• AMD Next-Gen Fortan
• Something else?

I am trying to run this on a PC with two AMD MI100 GPUs, and from what I've seen, these compilers provide OpenMP offloading support to AMD Instinct GPUs.

If any adjustments would be required (whether in the compilation/make options or in the model code here), could you advise on what those changes might be?

Additionally, the README says at least 2 GPUs are needed; is the code dependent or this requirement, or is it only an expectation for computational power?

All of this would help broaden the usability of the setup for those without access to Cray systems, and make it even more impactful!

Thanks,
Atharva

[davidsmith001998-alt]

You’re right — MPAS has primarily been built/tested in Cray environments, but it can be adapted for other compilers that support OpenMP GPU offloading on AMD hardware. A few notes:

AOMP (AMD’s OpenMP compiler)

This is the most straightforward option since it is designed for AMD GPUs.

You may need to adjust makefiles to point to flang/clang from AOMP and ensure -fopenmp offloading flags are passed.

Set the appropriate GPU target (e.g. -fopenmp-targets=amdgcn-amd-amdhsa -Xopenmp-target=amdgcn-amd-amdhsa -march=gfx908 for MI100).

LLVM Flang / Next-Gen Fortran (flang-new)

Still evolving, but in combination with LLVM/Clang, it can support OpenMP offloading.

Stability and performance may vary compared to AOMP.

Code adjustments

In most cases, only minor edits to the build scripts are needed (compiler name, flags).

The model code itself generally should not need large changes, unless vendor-specific pragmas are hard-coded.

GPU requirement

The “2 GPU” note in the README is more about recommended computational throughput. In principle, you could run on a single GPU, though performance scaling may not be ideal.

Suggested approach:

Start with AOMP and try a clean build.

Compare logs and compiler output with the Cray build to catch missing flags.

If errors persist, enabling verbose (make V=1) will help track missing symbols.

Also, if you’re spending long debugging/testing sessions on this, I’ve been using a small free tool to log and calculate work hours (including breaks and overtime): ecalculadoradehoras.com. It’s been handy for tracking time during GPU testing runs.