From 9402f8e3c13f969915778afee4077476179c7fdc Mon Sep 17 00:00:00 2001 From: bendichter Date: Thu, 7 Sep 2023 11:25:30 +0200 Subject: [PATCH 01/11] add docs on the hub --- docs/50_hub.md | 32 ++++++++++++++++++++++++++++++++ mkdocs.yml | 1 + 2 files changed, 33 insertions(+) create mode 100644 docs/50_hub.md diff --git a/docs/50_hub.md b/docs/50_hub.md new file mode 100644 index 00000000..6ec699fb --- /dev/null +++ b/docs/50_hub.md @@ -0,0 +1,32 @@ +# Using the DANDI Hub + +The DANDI Hub is a JupyterHub instance in the cloud to interact with the data stored in DANDI, and is free to use for exploratory analysis of data on DANDI. Note that DANDI Hub is not intended for significant computation, but provides a place to introspect Dandisets and to perform some analysis and visualization of data. + +## Registration + +To use the [DANDI Hub](http://hub.dandiarchive.org), you must first register for an account using the [DANDI website](http://dandiarchive.org). + +## Choosing a server option + +When you start up the DANDI Hub, you will be asked to select across a number of server options. For basic exploration, Tiny or Base would most likely be appropriate. The DANDI Hub also currently offers Medium and Large options, which have more available memory and compute power. The "T4 GPU inference" server comes with an associated T4 GPU, and is intended to be used for applications that require GPU for inference. We request that users of this server be considerate of their usage of the DANDI Hub as a free community resource. Training large deep neural networks is not appropriate. A "Base (MATLAB)" server is also available, which provides a MATLAB cloud installation. + +## Example notebooks + +The best way to share analyses on DANDI data is through the DANDI example notebooks. These notebooks are organized by `///`. Dandiset contributors are encouraged to use these notebooks to demonstrate how to read, analyze, and visualize the data, and how to produce figures from associated scientific publications. + +### Contributing an example notebook + +Notebooks can be submitted as a Pull Request to the [DANDI example-notebooks repository](https://github.com/catalystneuro/example-notebooks). + +#### Environment specification +Best practice is to include one or more notebooks alongside an environment.yml file, which provides a conda-style specification of the environment required to run the notebooks. +1. Create a new environment: `conda create -n -python ` +2. Use `conda install ` and `pip install ` to install the necessary dependencies until the notebook runs through successfully. +3. Export the environment: `conda env export > environment.yml`. + +See detailed instructions for creating a `environment.yml` file [here](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#sharing-an-environment). + +#### File organization +When constructing the Pull Request, ensure that you have the proper directory structure: `///`. If you share more than one notebook, also include a `README.md` file at `///README.md` explaining the purpose of each notebook and providing context and links to relevant publications. + +Once this Pull Requests is accepted, your contributed notebook will be available to all DANDI Hub users. \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index f4c50e9f..4f1d5f3a 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -34,6 +34,7 @@ nav: - Developer Guide: - Project Structure: "20_project_structure.md" - Notes: "40_development.md" + - DANDI Hub: "50_hub.md" - Terms and Policies: - Terms: "about/terms.md" - Policies: "about/policies.md" From e67b2ccaabb8760cf0c4c5d03e57a5f7801f1672 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Thu, 7 Sep 2023 06:25:35 -0400 Subject: [PATCH 02/11] Update docs/50_hub.md --- docs/50_hub.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/50_hub.md b/docs/50_hub.md index 6ec699fb..d12a782a 100644 --- a/docs/50_hub.md +++ b/docs/50_hub.md @@ -1,6 +1,6 @@ # Using the DANDI Hub -The DANDI Hub is a JupyterHub instance in the cloud to interact with the data stored in DANDI, and is free to use for exploratory analysis of data on DANDI. Note that DANDI Hub is not intended for significant computation, but provides a place to introspect Dandisets and to perform some analysis and visualization of data. +[DANDI Hub](http://hub.dandiarchive.org) is a JupyterHub instance in the cloud to interact with the data stored in DANDI, and is free to use for exploratory analysis of data on DANDI. Note that DANDI Hub is not intended for significant computation, but provides a place to introspect Dandisets and to perform some analysis and visualization of data. ## Registration From f03a22f0cb6c33ccdfc2eb88a995e796bfa3b618 Mon Sep 17 00:00:00 2001 From: bendichter Date: Thu, 7 Sep 2023 16:52:57 +0200 Subject: [PATCH 03/11] remove specific submission instructions, since they are now a PR on the example-notebooks repo --- docs/50_hub.md | 15 --------------- 1 file changed, 15 deletions(-) diff --git a/docs/50_hub.md b/docs/50_hub.md index 6ec699fb..df3ea70d 100644 --- a/docs/50_hub.md +++ b/docs/50_hub.md @@ -14,19 +14,4 @@ When you start up the DANDI Hub, you will be asked to select across a number of The best way to share analyses on DANDI data is through the DANDI example notebooks. These notebooks are organized by `///`. Dandiset contributors are encouraged to use these notebooks to demonstrate how to read, analyze, and visualize the data, and how to produce figures from associated scientific publications. -### Contributing an example notebook - -Notebooks can be submitted as a Pull Request to the [DANDI example-notebooks repository](https://github.com/catalystneuro/example-notebooks). - -#### Environment specification -Best practice is to include one or more notebooks alongside an environment.yml file, which provides a conda-style specification of the environment required to run the notebooks. -1. Create a new environment: `conda create -n -python ` -2. Use `conda install ` and `pip install ` to install the necessary dependencies until the notebook runs through successfully. -3. Export the environment: `conda env export > environment.yml`. - -See detailed instructions for creating a `environment.yml` file [here](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#sharing-an-environment). - -#### File organization -When constructing the Pull Request, ensure that you have the proper directory structure: `///`. If you share more than one notebook, also include a `README.md` file at `///README.md` explaining the purpose of each notebook and providing context and links to relevant publications. - Once this Pull Requests is accepted, your contributed notebook will be available to all DANDI Hub users. \ No newline at end of file From d1db03dbb73df8b0ee9a1d38ce3161c126d22f81 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Mon, 18 Dec 2023 11:04:39 -0500 Subject: [PATCH 04/11] Update docs/50_hub.md Co-authored-by: Yaroslav Halchenko --- docs/50_hub.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/50_hub.md b/docs/50_hub.md index e7cd7751..72d12aae 100644 --- a/docs/50_hub.md +++ b/docs/50_hub.md @@ -1,6 +1,7 @@ # Using the DANDI Hub -[DANDI Hub](http://hub.dandiarchive.org) is a JupyterHub instance in the cloud to interact with the data stored in DANDI, and is free to use for exploratory analysis of data on DANDI. Note that DANDI Hub is not intended for significant computation, but provides a place to introspect Dandisets and to perform some analysis and visualization of data. +[DANDI Hub](http://hub.dandiarchive.org) is a JupyterHub instance in the cloud to interact with the data stored in DANDI, and is free to use for exploratory analysis of data on DANDI. +Note that DANDI Hub is not intended for significant computation, but provides a place to introspect Dandisets and to perform some analysis and visualization of data. ## Registration From 7daa32e53c4caa465d3dcca6b170ed74ce23ddaa Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Mon, 18 Dec 2023 11:05:34 -0500 Subject: [PATCH 05/11] Update docs/50_hub.md Co-authored-by: Yaroslav Halchenko --- docs/50_hub.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/50_hub.md b/docs/50_hub.md index 72d12aae..5b72f3e8 100644 --- a/docs/50_hub.md +++ b/docs/50_hub.md @@ -9,7 +9,13 @@ To use the [DANDI Hub](http://hub.dandiarchive.org), you must first register for ## Choosing a server option -When you start up the DANDI Hub, you will be asked to select across a number of server options. For basic exploration, Tiny or Base would most likely be appropriate. The DANDI Hub also currently offers Medium and Large options, which have more available memory and compute power. The "T4 GPU inference" server comes with an associated T4 GPU, and is intended to be used for applications that require GPU for inference. We request that users of this server be considerate of their usage of the DANDI Hub as a free community resource. Training large deep neural networks is not appropriate. A "Base (MATLAB)" server is also available, which provides a MATLAB cloud installation. +When you start up the DANDI Hub, you will be asked to select across a number of server options. +For basic exploration, Tiny or Base would most likely be appropriate. +The DANDI Hub also currently offers Medium and Large options, which have more available memory and compute power. +The "T4 GPU inference" server comes with an associated T4 GPU, and is intended to be used for applications that require GPU for inference. +We request that users of this server be considerate of their usage of the DANDI Hub as a free community resource. +Training large deep neural networks is not appropriate. +A "Base (MATLAB)" server is also available, which provides a MATLAB cloud installation but you would be required to provide your own license. ## Example notebooks From 9756ff3e2ba46c1cab8ab9429b894df678d14da1 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Mon, 18 Dec 2023 11:09:36 -0500 Subject: [PATCH 06/11] Update docs/50_hub.md Co-authored-by: Yaroslav Halchenko --- docs/50_hub.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/50_hub.md b/docs/50_hub.md index 5b72f3e8..a19aefed 100644 --- a/docs/50_hub.md +++ b/docs/50_hub.md @@ -19,6 +19,8 @@ A "Base (MATLAB)" server is also available, which provides a MATLAB cloud instal ## Example notebooks -The best way to share analyses on DANDI data is through the DANDI example notebooks. These notebooks are organized by `///`. Dandiset contributors are encouraged to use these notebooks to demonstrate how to read, analyze, and visualize the data, and how to produce figures from associated scientific publications. +The best way to share analyses on DANDI data is through the DANDI example notebooks. +These notebooks are maintained in https://github.com/dandi/example-notebooks repository which provides more information about their organization. +Dandiset contributors are encouraged to use these notebooks to demonstrate how to read, analyze, and visualize the data, and how to produce figures from associated scientific publications. Once this Pull Requests is accepted, your contributed notebook will be available to all DANDI Hub users. \ No newline at end of file From 551cbac810c1c4f2036aa3432f791c9aba5afa00 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Sat, 23 Dec 2023 15:23:01 -0500 Subject: [PATCH 07/11] Update docs/50_hub.md Co-authored-by: Yaroslav Halchenko --- docs/50_hub.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/50_hub.md b/docs/50_hub.md index a19aefed..5e584a9d 100644 --- a/docs/50_hub.md +++ b/docs/50_hub.md @@ -1,6 +1,6 @@ # Using the DANDI Hub -[DANDI Hub](http://hub.dandiarchive.org) is a JupyterHub instance in the cloud to interact with the data stored in DANDI, and is free to use for exploratory analysis of data on DANDI. +[DANDI Hub](http://hub.dandiarchive.org) is a [JupyterHub](https://jupyterhub.readthedocs.io) instance in the cloud to interact with the data stored in DANDI, and is free to use for exploratory analysis of data on DANDI. Note that DANDI Hub is not intended for significant computation, but provides a place to introspect Dandisets and to perform some analysis and visualization of data. ## Registration From e4ea76cdb290fa3e3a8c1bdce304bfb02b8f7b04 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Sat, 23 Dec 2023 15:23:35 -0500 Subject: [PATCH 08/11] Update docs/50_hub.md Co-authored-by: Kabilar Gunalan --- docs/50_hub.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/50_hub.md b/docs/50_hub.md index 5e584a9d..214cb3ba 100644 --- a/docs/50_hub.md +++ b/docs/50_hub.md @@ -20,7 +20,7 @@ A "Base (MATLAB)" server is also available, which provides a MATLAB cloud instal ## Example notebooks The best way to share analyses on DANDI data is through the DANDI example notebooks. -These notebooks are maintained in https://github.com/dandi/example-notebooks repository which provides more information about their organization. +These notebooks are maintained in the [dandi/example-notebooks](https://github.com/dandi/example-notebooks) repository which provides more information about their organization. Dandiset contributors are encouraged to use these notebooks to demonstrate how to read, analyze, and visualize the data, and how to produce figures from associated scientific publications. Once this Pull Requests is accepted, your contributed notebook will be available to all DANDI Hub users. \ No newline at end of file From d50b4800406483ab983ef81933f4b7074aa3f7a3 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Sat, 23 Dec 2023 15:24:03 -0500 Subject: [PATCH 09/11] Update docs/50_hub.md Co-authored-by: Kabilar Gunalan --- docs/50_hub.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/50_hub.md b/docs/50_hub.md index 214cb3ba..4530abd0 100644 --- a/docs/50_hub.md +++ b/docs/50_hub.md @@ -23,4 +23,4 @@ The best way to share analyses on DANDI data is through the DANDI example notebo These notebooks are maintained in the [dandi/example-notebooks](https://github.com/dandi/example-notebooks) repository which provides more information about their organization. Dandiset contributors are encouraged to use these notebooks to demonstrate how to read, analyze, and visualize the data, and how to produce figures from associated scientific publications. -Once this Pull Requests is accepted, your contributed notebook will be available to all DANDI Hub users. \ No newline at end of file +Notebooks can be added and updated through a pull request to the [dandi/example-notebooks](https://github.com/dandi/example-notebooks) repository. Once the pull request is merged, your contributed notebook will be available to all DANDI Hub users. \ No newline at end of file From ca4f0640495326e71b3549fb4d1f1a6b15ee7a79 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Tue, 9 Jan 2024 09:00:27 -0500 Subject: [PATCH 10/11] Update docs/50_hub.md Co-authored-by: Yaroslav Halchenko --- docs/50_hub.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/50_hub.md b/docs/50_hub.md index 4530abd0..304fd87e 100644 --- a/docs/50_hub.md +++ b/docs/50_hub.md @@ -23,4 +23,5 @@ The best way to share analyses on DANDI data is through the DANDI example notebo These notebooks are maintained in the [dandi/example-notebooks](https://github.com/dandi/example-notebooks) repository which provides more information about their organization. Dandiset contributors are encouraged to use these notebooks to demonstrate how to read, analyze, and visualize the data, and how to produce figures from associated scientific publications. -Notebooks can be added and updated through a pull request to the [dandi/example-notebooks](https://github.com/dandi/example-notebooks) repository. Once the pull request is merged, your contributed notebook will be available to all DANDI Hub users. \ No newline at end of file +Notebooks can be added and updated through a pull request to the [dandi/example-notebooks](https://github.com/dandi/example-notebooks) repository. +Once the pull request is merged, your contributed notebook will be available to all DANDI Hub users. \ No newline at end of file From 35917e1111d7256a4969f29e1b8a49479034656b Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Tue, 9 Jan 2024 09:14:53 -0500 Subject: [PATCH 11/11] Update docs/50_hub.md Co-authored-by: Kabilar Gunalan --- docs/50_hub.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/50_hub.md b/docs/50_hub.md index 304fd87e..4a523f5f 100644 --- a/docs/50_hub.md +++ b/docs/50_hub.md @@ -1,6 +1,7 @@ # Using the DANDI Hub [DANDI Hub](http://hub.dandiarchive.org) is a [JupyterHub](https://jupyterhub.readthedocs.io) instance in the cloud to interact with the data stored in DANDI, and is free to use for exploratory analysis of data on DANDI. +For instructions on how to navigate JupyterHub see this [YouTube tutorial](https://www.youtube.com/watch?v=5pf0_bpNbkw&t=09m20s). Note that DANDI Hub is not intended for significant computation, but provides a place to introspect Dandisets and to perform some analysis and visualization of data. ## Registration