feat: API with scheduler support

[ltalirz]

No description provided.

[github-actions]

Coverage Report

File	Stmts	Miss	Cover	Missing
__init__.py	0	0	100%
app.py	38	5	86%	40–42, 44–45
config.py	15	3	80%	17–19
database.py	144	32	77%	74, 104–106, 133–135, 153, 163–165, 177–179, 212–214, 225–226, 228–229, 238, 240–241, 243, 245–247, 289, 292, 313–314
jobs.py	33	20	39%	41, 43, 50–52, 54, 63–66, 69–73, 75, 98–99, 101, 107
models.py	85	14	83%	64, 66, 71–73, 78–80, 82–83, 154–156, 164
visualization.py	93	13	86%	39, 56–59, 148, 152, 177, 179–180, 230–232
routers
__init__.py	0	0	100%
meltquench.py	115	29	74%	69–70, 85, 89–91, 122–123, 222–224, 275, 308–309, 311–312, 317–318, 323, 329–330, 332–339
workflows
__init__.py	2	0	100%
meltquench.py	29	20	31%	67–68, 71–76, 79–81, 84, 96–99, 117–118, 120, 122
TOTAL	554	136	75%

[ltalirz]

@jan-janssen For some reason, I'm seeing a behavior where when I run through my API integration test once, it fails - it keeps pinging the /check endpoint until I kill the test (see also CI).

There are no errors in the API logs. When I locally restart the API and run the test again, it passes immediately (i.e. the simulation worked the first time and I can see it produces the cache files, they are just not picked up the first time).
Note: this also works, if I remove the high-level caching that I introduced (tasks.db, so the problem should be at a lower level).

I'm a bit lost at the moment - if you have any ideas/spot anything, let me know.

Relevant code should be in

---

workflow:

amorphouspy/amorphouspy_api/src/amorphouspy_api/workflows/meltquench.py

Line 31 in 38e3043

def run_meltquench_workflow(

how it is submitted by the API:

amorphouspy/amorphouspy_api/src/amorphouspy_api/app.py

Line 91 in 38e3043

def submit_to_executor(request_data: dict) -> dict:

amorphouspy/amorphouspy_api/src/amorphouspy_api/app.py

Line 364 in 38e3043

async def check(task_id: str) -> TaskResponse:

The code is likely overly cautious/complex in some parts (was tries to get it to work that didn't work and can be removed again later)

[jan-janssen]

@ltalirz You were a bit too fast. From my perspective there are currently two conflicting tests the test_check_running_then_complete() test which wants to wait for the future to complete and the test_submit_meltquench_and_check() test which requires the future to return the result directly.

[ltalirz]

There's a few things to be cleaned up - I will likely not have time today to finish it, but should be able to have another look tomorrow

@jan-janssen One question: in 669a33c we introduced workers with separate subprocesses in order to work around the signal handling of pyiron

We no longer need this here, correct?
My plan was to delete this now and just make the job submission async (it should anyhow be fast).

[jan-janssen]

We no longer need this here, correct? My plan was to delete this now and just make the job submission async (it should anyhow be fast).

Yes, that works fine.

As suggested in #124 it seems to be an issue with orphan processes being killed from the testing framework, so the transition to flux as backend should solve this issue.

[ltalirz]

@jan-janssen I fixed the basic logic for submission and /check; I also switched to the TestClusterExecutor

With that I now see

INFO:     127.0.0.1:42466 - "GET /check/83c4627d-411a-443a-ba70-78364bc9e323 HTTP/1.1" 200 OK
Exception in thread Thread-61 (execute_tasks_h5):
Traceback (most recent call last):
  File "/home/runner/miniconda3/envs/test/lib/python3.13/threading.py", line 1044, in _bootstrap_inner
    self.run()
    ~~~~~~~~^^
  File "/home/runner/miniconda3/envs/test/lib/python3.13/threading.py", line 995, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/miniconda3/envs/test/lib/python3.13/site-packages/executorlib/task_scheduler/file/shared.py", line 151, in execute_tasks_h5
    process_dict[k] for k in future_wait_key_lst
    ~~~~~~~~~~~~^^^
KeyError: 'get_structure_dictf69a7790ce10f1dde00df1103f771a09'

I can reproduce this locally as well.

Maybe some of the logic in /check is still not correct...

[ltalirz]

relevant code:
https://github.com/glasagent/amorphouspy/pull/121/changes#diff-fb310cd2ccc4a6f9566da2791956c1f5e5ddcb3c9e85b85e5ef96cddb0210449R140

and
https://github.com/glasagent/amorphouspy/pull/121/changes#diff-fb310cd2ccc4a6f9566da2791956c1f5e5ddcb3c9e85b85e5ef96cddb0210449R281

[jan-janssen]

@jan-janssen I fixed the basic logic for submission and /check; I also switched to the TestClusterExecutor

This is fixed in pyiron/executorlib#913