Switch from SingleNodeExecutor to TestClusterExecutor

[jan-janssen]

Currently still requires shutdown(wait=True, cancel_futures=False).

[github-actions]

Coverage Report

File	Stmts	Miss	Cover	Missing
__init__.py	0	0	100%
app.py	107	23	78%	51–53, 60–61, 71, 129–130, 141, 144, 147–148, 198, 200, 205–207, 231–233, 260–262
database.py	130	32	75%	69, 98–100, 121–123, 151–153, 165–167, 192–194, 205–206, 208–209, 215, 217–218, 220, 222–224, 238, 261, 265, 283–284
models.py	72	20	72%	37–39, 56, 58, 63–66, 68–72, 74–75, 124–126, 134
visualization.py	93	13	86%	39, 56–59, 148, 152, 177, 179–180, 230–232
worker.py	78	73	6%	23–30, 44, 46, 48–49, 52, 55–56, 58, 60–61, 67–69, 72–73, 75–76, 78–79, 82–85, 88–89, 92–93, 101, 105, 107, 112, 115–118, 121, 126–127, 138, 141–144, 151, 155–158, 161–165, 168–170, 175–176, 189, 191, 193–197
TOTAL	480	161	66%

[ltalirz]

@jan-janssen I just want to point out one subtle point, which is that on main the calculation runs to completion inside a worker process spawned by the API.

I.e. /check is just checking whether that process has written the results to the top-level task store, /check is not using executorlib.

In my branch I move to a mode where /submit does a fire-and-forget submission to executorlib, while /check uses executorlib to check the status of the job.

[jan-janssen]

@jan-janssen I just want to point out one subtle point, which is that on main the calculation runs to completion inside a worker process spawned by the API.

Yes, I just want to separate the pull request into multiple steps to simplify the debugging.

• This pull request implements the switch to the TestClusterExecutor - this is fixed in [Fix] Clean up shutdown routine for ClusterExecutors pyiron/executorlib#920
• The transition to the FluxClusterExecutor was implemented in Switch from TestClusterExecutor to FluxClusterExecutor  #131 - the focus here was to setup flux to work with the API

I.e. /check is just checking whether that process has written the results to the top-level task store, /check is not using executorlib.

I agree with this strategy.

In my branch I move to a mode where /submit does a fire-and-forget submission to executorlib, while /check uses executorlib to check the status of the job.

To simplify this I implemented pyiron/executorlib#915 and pyiron/executorlib#914 .

[ltalirz]

superseded by #121