How to detect failed simulation?

[bilderbuchi]

I have a Modelica model (can't share that sadly). Using OMEdit, using certain parameter values the simulation (rightly so) fails to finish, and the OMEdit output console gives an error Simulation process failed. Exited with code -11.

Now, I am using OMPython to automate simulations of that model, and certain parameter combinations lead to simulations (correctly) failing. However, I could not find a way to detect, using OMPython, that the simulation fails!
Using the same parameters that correctly and visibly fail with OMEdit above, when doing model.simulate() followed by model.sendExpression('getErrorString()'), I only get an empty string back. In fact, in the (powershell) console I only see warnings and info-level messages, but no errors. However, a last info message says model terminate | Integrator failed. | Simulations terminated at time 167.512, which indicates a failed simulation to me.
Why is this info-level, though, and not error-level?
How can I detect that a simulation did not finish succesfully?

[adeas31]

Could you please test this with the latest OMPython? We have changed the way errors and assertions are handled in OMPython.

You might not get the same output as in OMEdit since OMEdit runs the simulation executable itself and reports the executable errors directly like Exited with code -11.
In OMPython you are calling [simulate](https://build.openmodelica.org/Documentation/OpenModelica.Scripting.simulate.html) API so the errors are shown in SimulationResult record.

[bilderbuchi]

Sorry, I don't have access to the model anymore (and don't even remember which one that was)

so the errors are shown in SimulationResult record.

AFAICT from the source code, simulate does not return the SimulationResult record, so how would I check. It even swallows the returncode (only emits a logging warning).

Even if the record were returned, the SimulationResult you linked to above does not contain a simulation result code/success boolean/etc, so the way to check if the simulation was successful would then be to parse the messages and see if they contain "error"? Is that the intended way? 🤔

[adeas31]

AFAICT from the source code, simulate does not return the SimulationResult record, so how would I check. It even swallows the returncode (only emits a logging warning).

Sorry my bad. I thought that it is using simulate API but it doesn't. It calls the simulation executable directly. So the standard output is logged and in case of error an exception is thrown.

OMPython/OMPython/OMCSession.py

Line 813 in 299b3aa

raise OMCSessionException(f"Error running model executable {repr(cmdl)}: {stderr}")

Even if the record were returned, the SimulationResult you linked to above does not contain a simulation result code/success boolean/etc, so the way to check if the simulation was successful would then be to parse the messages and see if they contain "error"? Is that the intended way? 🤔

I guess you can simply use the code with try catch.
@syntron any suggestions here?

[bilderbuchi]

ah, did not dig deep enough -- an Exception on stderr sounds like a good enough heuristic.

[bilderbuchi]

I guess we can close this and reopen/create a new issue if needed.

[syntron]

The challenge is that using the executable we do not have so much options to provide meaningful error messages:

• if there is some output to stdout it is printed as logger.debug()
• if there is stderr an exception is raised (OMCSessionException)
• a timeout or a CalledProcessError (= non-zero exit code) gives also an exception (OMCSessionException)

For all see OMCProcess.run_model_executable(). From my understanding, the executable was selected as it allows to run simulations independent of OM / no need to allways recompile.

However, all of this does not help, if the executable just crashs - see #261 - in that case no output at all is provided. This is handled via a check if the result file is empty. However, as discussed in the corresponding OpenModelica bug (OpenModelica/OpenModelica#13829) this is something OMPython can not solve ...