Skip to content

Replace notebooks with pipelines that use SQL Runner #9

@inglesp

Description

@inglesp

This is more general issue around notebooks that run on the server, that encompasses some others, including:
opensafely-core/cohort-extractor#830
opensafely-core/cohort-extractor#831

There are several notebooks that are intended to be run on the server, including the database notebooks, and the ones in this repo. Applications and status of notebooks is described in this doc.

The current process for running the database notebooks is described in the team manual, and in the raw-data-plausibility-checks repo README for other notebooks.

We need to

  1. Define a common process for running notebooks on the server and how/where to publish outputs
  2. Find a way to automate the running of certain notebooks - typically ones that should rerun after a database build (database notebooks, vaccine names notebook, maybe others?). This could possibly be triggered by a database build, if we can identify that - perhaps by job-runner going into maintenance mode? Or if the notebooks are not too heavy-duty, a weekly (or whatever) cron job.

Will has started a discussion in #issue-db-notebooks-release and also in this thread.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions