We need an architecture for running async jobs that are managed by the create-a-container API server. At a high-level, this will be a new set of DB models to handle job status information and a new system service (job-runner) that runs with apiserver-level privileges to handle jobs and report the statuses. More detail on the anticipated implementation:
- A new Sequelize model/migration(s)
Jobs (id PK AUTOINCREMENT, command STRING, status STRING [or ENUM or INTEGER]). This is where other routes, such as POST /containers, can add new jobs with the default "pending" status to wait to be picked up by the job runner.
- A new Sequelize model/migration(s)
JobStatus[or Progress or Output] (id PK AUTOINCREMENT, jobId FK references Jobs.id, timestamp TIME, output STRING). This is where the job runner will log job output to be viewed from the frontend, such as on GET /jobs/:id/status (which will replace GET /api/stream/:jobId)
- A new system service
job-runner which is ran with the same configuration as the create-a-container service. It has a main loop which (1) checks the Jobs model for jobs in the pending state, (2) spawns Jobs.command into a subprocess, (3) watches the stdout/err of the command and updates JobStatus with log lines, and (4) reports Jobs.status to be success or failure based on exit-code once the command exits.
It is expected that the job-runner will be ran on the same container as create-a-container at least for now. Jobs should expect to work within the create-a-container environment, including access to database models. I would implement this as job-runner.js existing alongside server.js in the create-a-container/ directory and using that as the CWD of all sub processes as well.
We need an architecture for running async jobs that are managed by the
create-a-containerAPI server. At a high-level, this will be a new set of DB models to handle job status information and a new system service (job-runner) that runs with apiserver-level privileges to handle jobs and report the statuses. More detail on the anticipated implementation:Jobs (id PK AUTOINCREMENT, command STRING, status STRING [or ENUM or INTEGER]). This is where other routes, such asPOST /containers, can add new jobs with the default "pending" status to wait to be picked up by the job runner.JobStatus[or Progress or Output] (id PK AUTOINCREMENT, jobId FK references Jobs.id, timestamp TIME, output STRING). This is where the job runner will log job output to be viewed from the frontend, such as onGET /jobs/:id/status(which will replaceGET /api/stream/:jobId)job-runnerwhich is ran with the same configuration as thecreate-a-containerservice. It has a main loop which (1) checks theJobsmodel for jobs in the pending state, (2) spawnsJobs.commandinto a subprocess, (3) watches the stdout/err of the command and updatesJobStatuswith log lines, and (4) reportsJobs.statusto be success or failure based on exit-code once the command exits.It is expected that the
job-runnerwill be ran on the same container ascreate-a-containerat least for now. Jobs should expect to work within thecreate-a-containerenvironment, including access to database models. I would implement this asjob-runner.jsexisting alongsideserver.jsin the create-a-container/ directory and using that as the CWD of all sub processes as well.