Expose mesh deformation (SU2_DEF) and AD gradient projection (SU2_DOT_AD) to the pysu2 Python wrapper

[aa-g]

Context
I am working in a parallel MDO framework where one or several instances of SU2 (corresponding to different operating cases) can coupled with other solvers and tools, to perform coupled aero-structural shape optimisation.

The framework initialises MPI and assigns processors to the different solvers and tools as required. The SU2 instances are initialised and controlled through Python, and given a communicator for the relevant subset of the MPI processes.

Issue
Shape optimisation with SU2 (using discrete adjoints) require four operations at each major optimisation iteration:

1. SU2_DEF: Deform the mesh according to the new design variables (or design deformations, in contrast to e.g. the structural deformations in an aeroelastic analysis), and output the deformed mesh to file,
2. SU2_CFD: Update the primal solution, which may include a mesh deformation due to structural deformations in an aeroelastic analysis, and output the converged state to file,
3. SU2_CFD_AD: Update the discrete adjoint solution corresponding to the converged primal solution, once for each objective function of interest, and output the converged adjoints to file,
4. SU2_DOT_AD: Calculate the surface sensititivities from the discrete adjoint solution (from what I can gather, this step is required in the discrete-adjoint sensitivity analysis to account for the in-direct influence surface deformations have on the objective through the volume deformation; essentially a differentiation of SU2_DEF), and output these to file.

Currently, the SU2 Python wrapper exposes driver classes which allow control of the execution of steps 2 and 3 described above, namely CSinglezoneDriver and CDiscAdjSinglezonedriver (among other variants). For steps 1 and 4 however, there is no such option: these steps must be executed by a "command-line call" in a newly spawned sub-process, as is done in the SU2 Python tools, in SU2.run.DEF and SU2.run.DOT respectively.

However, spawning new -- parallel -- processes using subprocess from within an MPI environment is "dangerous", besides being rather rude towards workload managers such as Slurm, making this an unviable solution in this kind of application.

Proposed solution
If at all reasonable, I would like to have the functionality of SU2_DEF and SU2_DOT_AD exposed to the Python wrappers pysu2 and pysu2ad, ideally using an approach which is consistent with the existing implementation for SU2_CFD and SU2_CFD_AD.

If I understand correctly, this change would require something along the lines of:

• Refactoring the main functions of SU2_DEF.cpp and SU2_DOT_AD.cpp into new driver classes, e.g. CDeformationDriver and CDiscAdjProjectionDriver, broken down into pre-processing, execution, output, and postprocessing.
• Exposing these new driver classes to Python through a SWIG .i file

Beyond offering a solution to the problem described above, allowing the complete optimisation iteration to be controlled externally (for example by an MDO framework), this might in the long term also allow providing deformations to the SU2_DEF routine directly in Python, as an alternative to the surface deformation file. Similarly, the surface sensitivities resulting from SU2_DOT_AD could be accessed directly in Python for post-processing.

Alternatives
The proposed solution is hopefully a starting point for some discussion, should there be interest in the issue. Suggestions which could help with the refinement and implementation of the proposed solution, or any alternative, would be very welcome.

Disclaimer
Please excuse the long read. My understanding of the inner workings of SU2 is limited, therefore I have tried to give a complete description of the assumptions that have led to the issue and the proposed solution. Please let me know if there are mistakes, or if further detail can be useful.

[pcarruscag]

There was a lot of talk about refactoring the shape optimization framework to be able to work completely in memory.
I think the idea was not to have more drivers but simply to have more granularity such that you could manipulate the geometry from python (without writing it to files).
We also wanted introspection, i.e. you could probe SU2 via python for what variables and functions are available.
We also talked about having the projection step (DOT) included in CFD_AD so that you only need to worry about surface sensitivities, note that DEF is already part of CFD for forward mode AD.
(talking does not go very far)

If you want to lead this work, we can help you understand the inner workings. We meet every Wednesday at 4pm CET https://meet.jit.si/SU2_DevMeeting to discuss this type of issue.

[aa-g]

There was a lot of talk about refactoring the shape optimization framework to be able to work completely in memory.
I think the idea was not to have more drivers but simply to have more granularity such that you could manipulate the geometry from python (without writing it to files).
We also wanted introspection, i.e. you could probe SU2 via python for what variables and functions are available.
We also talked about having the projection step (DOT) included in CFD_AD so that you only need to worry about surface sensitivities, note that DEF is already part of CFD for forward mode AD.
(talking does not go very far)

If you want to lead this work, we can help you understand the inner workings. We meet every Wednesday at 4pm CET https://meet.jit.si/SU2_DevMeeting to discuss this type of issue.

Thank you for the quick reply. The discussions you mention sound very promising, I would be glad to join tomorrow afternoon and learn more, so that I might contribute.

One quick remark: I am aware that it is indeed possible to deform the mesh within SU2_CFD, I use this extensively for static aero-elastic analysis. However, I have not been able to have the deformed mesh be written to file at output (in SU2 or CGNS format).

This is fine for aero-elastic updates (movement of the surface due to structural displacements), where it is sufficient to see the deformed mesh in the post-processing files. It becomes impractical though, for design updates (movement of the surface to due to design displacements, i.e. representing changes in the design variables) where the new mesh is a useful intermediate result.

Even if this is just an issue in my configuration file, I think that a stand-alone driver or equivalent of SU2_DEF would remain useful. For each new set of updated design variables during shape optimisation, a single mesh deformation is sufficient to provide the new, updated jig geometry which all of the aerodynamic or aero-elastic analyses (one for each operating point) have in common. There would be no need to initialise the entire SU2_CFD machinery for this step, or to apply the design update separately for each of the SU2 instances.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is still a relevant issue please comment on it to restart the discussion. Thank you for your contributions.