Airfoil history processing requires pyoptsparse

[ffelten]

I thought we fixed the issue by having our own pyoptsparse history but it was not--I had a pyoptsparse installed in my local venv.

It seems that when reading the opt.hst file, it tries to unpickle pyoptsparse classes, which will trigger a ModuleNotFound pyoptsparse. I can provide an opt.hst file, here is come code to test:

optisteps_history = []
history = History("opt.hst")
call_counters = history.getCallCounters()
iters = list(map(int, call_counters)) if call_counters is not None else []

for i in range(len(iters)):
    vals = history.read(int(iters[i]))
    if vals is not None and "funcs" in vals and "obj" in vals["funcs"] and not vals["fail"]:
        values = history.getValues(names=["obj"], callCounters=[i], allowSens=False, major=False, scale=True)
        if values is not None and "obj" in values:
            objective = values["obj"]
            optisteps_history.append(OptiStep(obj_values=np.array(objective), step=vals["iter"]))

history.close()
print(optisteps_history)

We bypassed it by installing pyoptsparse on the cluster but it might not work everywhere (requires native deps).

[g-braeunlich]

Here is a hack which gets the above snippet to work:

import io
import os
import pickle
from typing import Any
import unittest.mock

import numpy as np

from engibench.core import OptiStep
from engibench.problems.airfoil.pyopt_history import History

DIR = os.path.dirname(__file__)


class FakePyOptSparseObject:
    def __init__(self, *args, **kwargs) -> None:
        self.args = args
        self.kwargs = kwargs

    def _mapContoOpt_Dict(self, d):
        return d

    def _mapXtoOpt_Dict(self, d):
        return d

    def _mapObjtoOpt_Dict(self, d):
        return d


class Unpickler(pickle.Unpickler):
    def find_class(self, module, name):
        if module == "pyoptsparse" or module.startswith("pyoptsparse."):
            return FakePyOptSparseObject
        return super().find_class(module, name)

    @classmethod
    def loads(cls, s: bytes) -> Any:
        """Helper function analogous to pickle.loads()."""
        return cls(io.BytesIO(s)).load()


optisteps_history = []

with unittest.mock.patch("sqlitedict.loads", Unpickler.loads):
    history = History(os.path.join(DIR, "opt.hst"))
call_counters = history.getCallCounters()
iters = list(map(int, call_counters)) if call_counters is not None else []

for i in range(len(iters)):
    vals = history.read(int(iters[i]))
    if vals is not None and "funcs" in vals and "obj" in vals["funcs"] and not vals["fail"]:
        values = history.getValues(names=["obj"], callCounters=[i], allowSens=False, major=False, scale=True)
        if values is not None and "obj" in values:
            objective = values["obj"]
            optisteps_history.append(OptiStep(obj_values=np.array(objective), step=vals["iter"]))

history.close()
print(optisteps_history)

[ffelten]

Here is a hack which gets the above snippet to work:

Should we add that into the repo, e.g., next to our own pyoptsparse history?

[g-braeunlich]

I first would like to improve my mess a bit