diff --git a/documentation/DS-documentation/_toc.yml b/documentation/DS-documentation/_toc.yml index 5035b2f..7ba972c 100644 --- a/documentation/DS-documentation/_toc.yml +++ b/documentation/DS-documentation/_toc.yml @@ -2,14 +2,17 @@ # Learn more at https://jupyterbook.org/customize/toc.html format: jb-book -root: overview +root: introduction parts: -- caption: Creating DS +- caption: FAQ + chapters: + - file: overview +- caption: Adapting Distributed-Something to a new application chapters: - file: customizing_DS - file: implementing_DS - file: troubleshooting_implementation -- caption: Running DS +- caption: Running an application made by Distributed-Something as an end user chapters: - file: step_0_prep - file: step_1_configuration @@ -18,7 +21,7 @@ parts: - file: step_2_submit_jobs - file: step_3_start_cluster - file: step_4_monitor -- caption: +- caption: Technical guides chapters: - file: troubleshooting_runs - file: versions diff --git a/documentation/DS-documentation/customizing_DS.md b/documentation/DS-documentation/customizing_DS.md index 111ac27..1256cbe 100644 --- a/documentation/DS-documentation/customizing_DS.md +++ b/documentation/DS-documentation/customizing_DS.md @@ -1,3 +1,4 @@ +(customization)= # Customizing DS Distributed-Something is a template. diff --git a/documentation/DS-documentation/introduction.md b/documentation/DS-documentation/introduction.md new file mode 100644 index 0000000..317c9e4 --- /dev/null +++ b/documentation/DS-documentation/introduction.md @@ -0,0 +1,29 @@ +# Introduction to Distributed-Something + +Distributed-Something is a framework for scaling parallelizable jobs on [Amazon Web Services](https://aws.amazon.com/). This framework can be extended to most Dockerized tools. Distributed-Something handles the infrastructure, so that developers and end-users alike can focus on the tool and the science. + +Distributed-Something is pure Python and lightweight, increasing its accessibility and usability with low-to-moderate-computational-comfort scientists. + +## Distributed-Something for application developers + +If the tool you want to parallelize with Distributed-Something already exists as a Docker on [Docker Hub](https://hub.docker.com/) or a repository like [BioContainers](https://biocontainers.pro/), then customization of Distributed-Something is a snap! + +Decide which aspects of the tool you wish to wrap need to be exposed to the end-user (typically: input locations, output locations, and some command line flags), and add them to the framework. Our [documentation](customization) and code comments will guide you to where things need to go. + +Add information about the customizations you added to the main [user-facing documentation](use), then share your implementation with your coworkers or the world at large! + +## Distributed-Something for application users + +Distributed-Something-wrapped applications endeavor to make it easy for non-computationally-comfortable end users to get started with. + +Tool users need only Python 3.7+ with `boto3` installed on the machine they wish to launch it from, and to edit plain-text files; no workflow languages required. + +# Is this the only framework for scaling jobs available? + +Absolutely not! We are aware of and have in many cases used the ones below. Ultimately, the right tool for your use case is the one you can get running, at a compute-cost- and human-hour-price that is optimal for you. + +* [Galaxy](https://galaxyproject.org/) +* [Terra](https://terra.bio/) +* [AWS Batch](https://docs.aws.amazon.com/batch/index.html) +* [AWS Parallel Cluster](https://docs.aws.amazon.com/parallelcluster/index.html) +* [AWS Genomics CLI](https://aws.github.io/amazon-genomics-cli/docs/overview/) diff --git a/documentation/DS-documentation/step_1_configuration.md b/documentation/DS-documentation/step_1_configuration.md index f481adf..d192fef 100644 --- a/documentation/DS-documentation/step_1_configuration.md +++ b/documentation/DS-documentation/step_1_configuration.md @@ -1,3 +1,4 @@ +(use)= # Step 1: Configuration The first step in setting up any job is editing the values in the config.py file.