From ecdbf220f22c93a86d178170676f9404df129b90 Mon Sep 17 00:00:00 2001 From: Shaden Smith Date: Wed, 18 Mar 2020 08:58:57 -0700 Subject: [PATCH 1/3] drop /docs/ extension --- docs/_config.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_config.yml b/docs/_config.yml index c0b6eb108af8..f92dc041a802 100644 --- a/docs/_config.yml +++ b/docs/_config.yml @@ -48,7 +48,7 @@ defaults: - scope: path: "_pages" values: - permalink: /docs/:basename + permalink: /docs/:basename/ - scope: path: "" type: posts From a03f37c5d4e61613904b9b3aa8e219f73b428766 Mon Sep 17 00:00:00 2001 From: Shaden Smith Date: Wed, 18 Mar 2020 09:26:12 -0700 Subject: [PATCH 2/3] Better config filename --- README.md | 2 +- docs/_data/navigation.yml | 2 +- docs/_pages/{config_json.md => config-json.md} | 0 docs/_tutorials/getting-started.md | 2 +- 4 files changed, 3 insertions(+), 3 deletions(-) rename docs/_pages/{config_json.md => config-json.md} (100%) diff --git a/README.md b/README.md index c9accefcd09a..56687c347d17 100755 --- a/README.md +++ b/README.md @@ -413,7 +413,7 @@ as the hostname. | Article | Description | | ---------------------------------------------------------------------------------------------- | -------------------------------------------- | | [DeepSpeed Features](https://www.deepspeed.ai/features/) | DeepSpeed features | -| [DeepSpeed JSON Configuration](https://www.deepspeed.ai/docs/config_json/) | Configuring DeepSpeed | +| [DeepSpeed JSON Configuration](https://www.deepspeed.ai/docs/config-json/) | Configuring DeepSpeed | | [API Documentation](https://deepspeed.readthedocs.io/en/latest/) | Generated DeepSpeed API documentation | | [CIFAR-10 Tutorial](https://www.deepspeed.ai/tutorials/cifar-10) | Getting started with CIFAR-10 and DeepSpeed | | [Megatron-LM Tutorial](https://www.deepspeed.ai/tutorials/megatron/) | Train GPT2 with DeepSpeed and Megatron-LM | diff --git a/docs/_data/navigation.yml b/docs/_data/navigation.yml index bd896797f76d..18fc4194e6da 100644 --- a/docs/_data/navigation.yml +++ b/docs/_data/navigation.yml @@ -25,7 +25,7 @@ lnav: - title: "Launching" url: /getting-started/#launching-deepspeed-training - title: "Configuration" - url: /docs/config_json/ + url: /docs/config-json/ - title: "Tutorials" url: /tutorials/ children: diff --git a/docs/_pages/config_json.md b/docs/_pages/config-json.md similarity index 100% rename from docs/_pages/config_json.md rename to docs/_pages/config-json.md diff --git a/docs/_tutorials/getting-started.md b/docs/_tutorials/getting-started.md index 69ffab990eb0..e0fa2871f5ee 100644 --- a/docs/_tutorials/getting-started.md +++ b/docs/_tutorials/getting-started.md @@ -116,7 +116,7 @@ the `step` value is stored as part of the `client_sd`. DeepSpeed features can be enabled, disabled, or configured using a config JSON file that should be specified as `args.deepspeed_config`. A sample config file is shown below. For a full set of features see [ API -doc](/docs/config_json/). +doc](/docs/config-json/). ```json { From 508a4ccce98cd1d8d7d0d1928613911db7a00431 Mon Sep 17 00:00:00 2001 From: Shaden Smith Date: Wed, 18 Mar 2020 10:07:16 -0700 Subject: [PATCH 3/3] Clean up configuration ToC --- docs/_data/navigation.yml | 13 +++++++++++++ docs/_pages/config-json.md | 9 +++++---- 2 files changed, 18 insertions(+), 4 deletions(-) diff --git a/docs/_data/navigation.yml b/docs/_data/navigation.yml index 18fc4194e6da..ab730c27f083 100644 --- a/docs/_data/navigation.yml +++ b/docs/_data/navigation.yml @@ -26,6 +26,19 @@ lnav: url: /getting-started/#launching-deepspeed-training - title: "Configuration" url: /docs/config-json/ + children: + - title: "Batch size" + url: /docs/config-json/#batch-size-related-parameters + - title: "Optimizer" + url: /docs/config-json/#optimizer-parameters + - title: "Scheduler" + url: /docs/config-json/#scheduler-parameters + - title: "Communication" + url: /docs/config-json/#communication-options + - title: "FP16" + url: /docs/config-json/#fp16-training-options + - title: "Logging" + url: /docs/config-json/#logging - title: "Tutorials" url: /tutorials/ children: diff --git a/docs/_pages/config-json.md b/docs/_pages/config-json.md index 35804f18e726..66c0820a7558 100644 --- a/docs/_pages/config-json.md +++ b/docs/_pages/config-json.md @@ -1,7 +1,11 @@ --- title: "DeepSpeed Configuration JSON" --- -## REQUIRED DeepSpeed Config JSON Parameters + +### Batch Size Related Parameters + +**Note:** configuring ***train\_batch\_size*** is required. +{: .notice--warning} ***train\_batch\_size***: [integer] @@ -9,9 +13,6 @@ title: "DeepSpeed Configuration JSON" | ------------------------------------------------------------ | ------- | | The effective training batch size. This is the amount of data samples that leads to one step of model update. ***train\_batch\_size*** is aggregated by the batch size that a single GPU processes in one forward/backward pass (a.k.a., ***train\_step\_batch\_size***), the gradient accumulation steps (a.k.a., ***gradient\_accumulation\_steps***), and the number of GPUs. | `32` | -## OPTIONAL DeepSpeed Config JSON Parameters - -### Batch Size Related Parameters ***train\_micro\_batch\_size\_per\_gpu***: [integer]