Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -413,7 +413,7 @@ as the hostname.
| Article | Description |
| ---------------------------------------------------------------------------------------------- | -------------------------------------------- |
| [DeepSpeed Features](https://www.deepspeed.ai/features/) | DeepSpeed features |
| [DeepSpeed JSON Configuration](https://www.deepspeed.ai/docs/config_json/) | Configuring DeepSpeed |
| [DeepSpeed JSON Configuration](https://www.deepspeed.ai/docs/config-json/) | Configuring DeepSpeed |
| [API Documentation](https://deepspeed.readthedocs.io/en/latest/) | Generated DeepSpeed API documentation |
| [CIFAR-10 Tutorial](https://www.deepspeed.ai/tutorials/cifar-10) | Getting started with CIFAR-10 and DeepSpeed |
| [Megatron-LM Tutorial](https://www.deepspeed.ai/tutorials/megatron/) | Train GPT2 with DeepSpeed and Megatron-LM |
Expand Down
2 changes: 1 addition & 1 deletion docs/_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ defaults:
- scope:
path: "_pages"
values:
permalink: /docs/:basename
permalink: /docs/:basename/
- scope:
path: ""
type: posts
Expand Down
15 changes: 14 additions & 1 deletion docs/_data/navigation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,20 @@ lnav:
- title: "Launching"
url: /getting-started/#launching-deepspeed-training
- title: "Configuration"
url: /docs/config_json/
url: /docs/config-json/
children:
- title: "Batch size"
url: /docs/config-json/#batch-size-related-parameters
- title: "Optimizer"
url: /docs/config-json/#optimizer-parameters
- title: "Scheduler"
url: /docs/config-json/#scheduler-parameters
- title: "Communication"
url: /docs/config-json/#communication-options
- title: "FP16"
url: /docs/config-json/#fp16-training-options
- title: "Logging"
url: /docs/config-json/#logging
- title: "Tutorials"
url: /tutorials/
children:
Expand Down
9 changes: 5 additions & 4 deletions docs/_pages/config_json.md → docs/_pages/config-json.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,18 @@
---
title: "DeepSpeed Configuration JSON"
---
## REQUIRED DeepSpeed Config JSON Parameters

### Batch Size Related Parameters

**Note:** configuring ***train\_batch\_size*** is required.
{: .notice--warning}

***train\_batch\_size***: [integer]

| Value | Example |
| ------------------------------------------------------------ | ------- |
| The effective training batch size. This is the amount of data samples that leads to one step of model update. ***train\_batch\_size*** is aggregated by the batch size that a single GPU processes in one forward/backward pass (a.k.a., ***train\_step\_batch\_size***), the gradient accumulation steps (a.k.a., ***gradient\_accumulation\_steps***), and the number of GPUs. | `32` |

## OPTIONAL DeepSpeed Config JSON Parameters

### Batch Size Related Parameters

***train\_micro\_batch\_size\_per\_gpu***: [integer]

Expand Down
2 changes: 1 addition & 1 deletion docs/_tutorials/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ the `step` value is stored as part of the `client_sd`.
DeepSpeed features can be enabled, disabled, or configured using a config JSON
file that should be specified as `args.deepspeed_config`. A sample config file
is shown below. For a full set of features see [ API
doc](/docs/config_json/).
doc](/docs/config-json/).

```json
{
Expand Down