We have received several mentions of the need for efficient caching when running our slurm jobs.
From @fgvangessel-umd
In particular, I am thinking of the airfoil problem (as well as future problems which share commonalities). The specific considerations I wanted to mention are:
• Instantiation of Airfoil problem involves creation of template files (i.e. input files) in a study_{seed} directory. How can we structure the SLURM jobs to copy from a single common template directory?
• Similarly, Airfoil problem uses a Docker container (involving a container pull). How can we pull a single container and then have each job launch from this same container (The container pull process is lengthy ~15 minutes in some instances).
In summary, how do we make EngiBench (and slurm capabilities) more compatible with IO heavy simulations AND simulations which require use of a Docker container.
I know @mkeeler43 also had a few issues with this, please tell us here :).
To be clarified with @g-braeunlich as to what is possible.
We have received several mentions of the need for efficient caching when running our slurm jobs.
From @fgvangessel-umd
In particular, I am thinking of the airfoil problem (as well as future problems which share commonalities). The specific considerations I wanted to mention are:
• Instantiation of Airfoil problem involves creation of template files (i.e. input files) in a study_{seed} directory. How can we structure the SLURM jobs to copy from a single common template directory?
• Similarly, Airfoil problem uses a Docker container (involving a container pull). How can we pull a single container and then have each job launch from this same container (The container pull process is lengthy ~15 minutes in some instances).
In summary, how do we make EngiBench (and slurm capabilities) more compatible with IO heavy simulations AND simulations which require use of a Docker container.
I know @mkeeler43 also had a few issues with this, please tell us here :).
To be clarified with @g-braeunlich as to what is possible.