From 834eee1a4268fa23d973ca92b7207a97dc98b414 Mon Sep 17 00:00:00 2001 From: jxxiaoshaoye <1700017801@pku.edu.cn> Date: Thu, 13 May 2021 15:26:16 +0800 Subject: [PATCH 1/6] add doc of type embedding --- doc/train-se-e2-a-tebd.md | 40 +++++++++++++++++++++++++++++++++++++++ doc/train-se-e2-a.md | 2 ++ 2 files changed, 42 insertions(+) create mode 100644 doc/train-se-e2-a-tebd.md diff --git a/doc/train-se-e2-a-tebd.md b/doc/train-se-e2-a-tebd.md new file mode 100644 index 0000000000..cb5272070d --- /dev/null +++ b/doc/train-se-e2-a-tebd.md @@ -0,0 +1,40 @@ +# Train a Deep Potential model using descriptor `"se_e2_a_tebd"` + +The notation of `se_e2_a_tebd` is short for the Deep Potential Smooth Edition (DeepPot-SE) constructed from all information (both angular and radial) of atomic configurations. The `e2` stands for the embedding with two-atoms information, and `tebd` stands for applying type embedding for each atom type. + +A complete training input script of this example can be find in the directory. +```bash +$deepmd_source_dir/examples/water/se_e2_a_tebd/input.json +``` +With atom type embedding, we can share one descriptor embedding net and one fitting net in total, which decline training complexity largely. + +The training input script is similar to that of [`se_e2_a`](train-se-e2-a.md#the-training-input-script). By adding the `type_embedding` section, model will automatically apply `se_e2_a_tebd` approach and generate type embedding vectors. If type embedding is detected, descriptor and fitting net would take it as a part of input. + +### Type embedding net +The `model` defines how the model is constructed, adding a section of type embedding net: +```json= + "model": { + "type_map": ["O", "H"], + "type_embedding":{ + ... + }, + "descriptor" :{ + ... + }, + "fitting_net" : { + ... + } + } +``` + +The construction of type embedding net is given by `type_embedding`. An example of type_embedding is provided as follows +```json= + "type_embedding":{ + "neuron": [2, 4, 8], + "resnet_dt": false, + "seed": 1 + } +``` +* The `neuron` specifies the size of the type embedding net. From left to right the members denote the sizes of each hidden layer from input end to the output end, respectively. It takes one-hot vector as input and output dimension equals to the last dimension of the `neuron` list. If the outer layer is of twice size as the inner layer, then the inner layer is copied and concatenated, then a [ResNet architecture](https://arxiv.org/abs/1512.03385) is built between them. +* If the option `resnet_dt` is set `true`, then a timestep is used in the ResNet. +* `seed` gives the random seed that is used to generate random numbers when initializing the model parameters. diff --git a/doc/train-se-e2-a.md b/doc/train-se-e2-a.md index 03bf33295d..730e0ac32a 100644 --- a/doc/train-se-e2-a.md +++ b/doc/train-se-e2-a.md @@ -47,6 +47,7 @@ The construction of the descriptor is given by section `descriptor`. An example "rcut": 6.00, "sel": [46, 92], "neuron": [25, 50, 100], + "type_one_side": true, "axis_neuron": 16, "resnet_dt": false, "seed": 1 @@ -56,6 +57,7 @@ The construction of the descriptor is given by section `descriptor`. An example * `rcut` is the cut-off radius for neighbor searching, and the `rcut_smth` gives where the smoothing starts. * `sel` gives the maximum possible number of neighbors in the cut-off radius. It is a list, the length of which is the same as the number of atom types in the system, and `sel[i]` denote the maximum possible number of neighbors with type `i`. * The `neuron` specifies the size of the embedding net. From left to right the members denote the sizes of each hidden layer from input end to the output end, respectively. If the outer layer is of twice size as the inner layer, then the inner layer is copied and concatenated, then a [ResNet architecture](https://arxiv.org/abs/1512.03385) is built between them. +* If the option `type_one_side` is set to `true`, then descriptor will only use information of neighbor atom when generating description. While `false` means using both centric atom and neighbor atom information. * The `axis_neuron` specifies the size of submatrix of the embedding matrix, the axis matrix as explained in the [DeepPot-SE paper](https://arxiv.org/abs/1805.09003) * If the option `resnet_dt` is set `true`, then a timestep is used in the ResNet. * `seed` gives the random seed that is used to generate random numbers when initializing the model parameters. From 5d5d3ae7627fdedff94d9be2f34ef97728da4129 Mon Sep 17 00:00:00 2001 From: jxxiaoshaoye <1700017801@pku.edu.cn> Date: Thu, 13 May 2021 15:31:37 +0800 Subject: [PATCH 2/6] add doc of se_e2_a_tebd --- doc/train-se-e2-a-tebd.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/doc/train-se-e2-a-tebd.md b/doc/train-se-e2-a-tebd.md index cb5272070d..d3bce65918 100644 --- a/doc/train-se-e2-a-tebd.md +++ b/doc/train-se-e2-a-tebd.md @@ -8,7 +8,7 @@ $deepmd_source_dir/examples/water/se_e2_a_tebd/input.json ``` With atom type embedding, we can share one descriptor embedding net and one fitting net in total, which decline training complexity largely. -The training input script is similar to that of [`se_e2_a`](train-se-e2-a.md#the-training-input-script). By adding the `type_embedding` section, model will automatically apply `se_e2_a_tebd` approach and generate type embedding vectors. If type embedding is detected, descriptor and fitting net would take it as a part of input. +The training input script is similar to that of [`se_e2_a`](train-se-e2-a.md#the-training-input-script), but different by adding the `type_embedding` section. ### Type embedding net The `model` defines how the model is constructed, adding a section of type embedding net: @@ -26,6 +26,7 @@ The `model` defines how the model is constructed, adding a section of type embed } } ``` +Model will automatically apply `se_e2_a_tebd` approach and generate type embedding vectors. If type embedding is detected, descriptor and fitting net would take it as a part of input. The construction of type embedding net is given by `type_embedding`. An example of type_embedding is provided as follows ```json= From 17e8d3fa168ab7858c66c716e266a1806983b54a Mon Sep 17 00:00:00 2001 From: jxxiaoshaoye <1700017801@pku.edu.cn> Date: Thu, 13 May 2021 16:00:57 +0800 Subject: [PATCH 3/6] add document of se_e2_a_tebd --- doc/train-se-e2-a-tebd.md | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/doc/train-se-e2-a-tebd.md b/doc/train-se-e2-a-tebd.md index d3bce65918..e895adc858 100644 --- a/doc/train-se-e2-a-tebd.md +++ b/doc/train-se-e2-a-tebd.md @@ -1,12 +1,6 @@ -# Train a Deep Potential model using descriptor `"se_e2_a_tebd"` - -The notation of `se_e2_a_tebd` is short for the Deep Potential Smooth Edition (DeepPot-SE) constructed from all information (both angular and radial) of atomic configurations. The `e2` stands for the embedding with two-atoms information, and `tebd` stands for applying type embedding for each atom type. - -A complete training input script of this example can be find in the directory. -```bash -$deepmd_source_dir/examples/water/se_e2_a_tebd/input.json -``` -With atom type embedding, we can share one descriptor embedding net and one fitting net in total, which decline training complexity largely. +# Train a Deep Potential model using `type embedding` approach + +We generate specific type embedding vector for each atom type, so that we can share one descriptor embedding net and one fitting net in total, which decline training complexity largely. The training input script is similar to that of [`se_e2_a`](train-se-e2-a.md#the-training-input-script), but different by adding the `type_embedding` section. @@ -26,9 +20,9 @@ The `model` defines how the model is constructed, adding a section of type embed } } ``` -Model will automatically apply `se_e2_a_tebd` approach and generate type embedding vectors. If type embedding is detected, descriptor and fitting net would take it as a part of input. +Model will automatically apply type embedding approach and generate type embedding vectors. If type embedding vector is detected, descriptor and fitting net would take it as a part of input. -The construction of type embedding net is given by `type_embedding`. An example of type_embedding is provided as follows +The construction of type embedding net is given by `type_embedding`. An example of `type_embedding` is provided as follows ```json= "type_embedding":{ "neuron": [2, 4, 8], @@ -39,3 +33,10 @@ The construction of type embedding net is given by `type_embedding`. An example * The `neuron` specifies the size of the type embedding net. From left to right the members denote the sizes of each hidden layer from input end to the output end, respectively. It takes one-hot vector as input and output dimension equals to the last dimension of the `neuron` list. If the outer layer is of twice size as the inner layer, then the inner layer is copied and concatenated, then a [ResNet architecture](https://arxiv.org/abs/1512.03385) is built between them. * If the option `resnet_dt` is set `true`, then a timestep is used in the ResNet. * `seed` gives the random seed that is used to generate random numbers when initializing the model parameters. + + + +A complete training input script of this example can be find in the directory. +```bash +$deepmd_source_dir/examples/water/se_e2_a_tebd/input.json +``` \ No newline at end of file From 129c1bc94c53afda0729347e009769de88533fc2 Mon Sep 17 00:00:00 2001 From: jxxiaoshaoye <1700017801@pku.edu.cn> Date: Thu, 13 May 2021 16:39:02 +0800 Subject: [PATCH 4/6] add doc of type embedding --- doc/train-se-e2-a.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/train-se-e2-a.md b/doc/train-se-e2-a.md index 730e0ac32a..b9b9db31e6 100644 --- a/doc/train-se-e2-a.md +++ b/doc/train-se-e2-a.md @@ -57,7 +57,7 @@ The construction of the descriptor is given by section `descriptor`. An example * `rcut` is the cut-off radius for neighbor searching, and the `rcut_smth` gives where the smoothing starts. * `sel` gives the maximum possible number of neighbors in the cut-off radius. It is a list, the length of which is the same as the number of atom types in the system, and `sel[i]` denote the maximum possible number of neighbors with type `i`. * The `neuron` specifies the size of the embedding net. From left to right the members denote the sizes of each hidden layer from input end to the output end, respectively. If the outer layer is of twice size as the inner layer, then the inner layer is copied and concatenated, then a [ResNet architecture](https://arxiv.org/abs/1512.03385) is built between them. -* If the option `type_one_side` is set to `true`, then descriptor will only use information of neighbor atom when generating description. While `false` means using both centric atom and neighbor atom information. +* If the option `type_one_side` is set to `true`, then descriptor will consider the types of neighbor atoms. Otherwise, both the types of centric and  neighbor atoms are considered. * The `axis_neuron` specifies the size of submatrix of the embedding matrix, the axis matrix as explained in the [DeepPot-SE paper](https://arxiv.org/abs/1805.09003) * If the option `resnet_dt` is set `true`, then a timestep is used in the ResNet. * `seed` gives the random seed that is used to generate random numbers when initializing the model parameters. From 6499df3124e2302e93ffa49deda642c68cbeff76 Mon Sep 17 00:00:00 2001 From: jxxiaoshaoye <1700017801@pku.edu.cn> Date: Thu, 13 May 2021 18:47:49 +0800 Subject: [PATCH 5/6] add doc of type embed --- doc/train-se-e2-a.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/train-se-e2-a.md b/doc/train-se-e2-a.md index b9b9db31e6..c69b17a574 100644 --- a/doc/train-se-e2-a.md +++ b/doc/train-se-e2-a.md @@ -47,7 +47,7 @@ The construction of the descriptor is given by section `descriptor`. An example "rcut": 6.00, "sel": [46, 92], "neuron": [25, 50, 100], - "type_one_side": true, + "type_one_side": true, "axis_neuron": 16, "resnet_dt": false, "seed": 1 From 5fa1537fc1aa3d71a90376967fe9d33b425c439b Mon Sep 17 00:00:00 2001 From: jxxiaoshaoye <1700017801@pku.edu.cn> Date: Thu, 13 May 2021 20:40:00 +0800 Subject: [PATCH 6/6] add doc for type embedding --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 162603099a..bac29b4ddd 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ For more information, check the [documentation](https://deepmd.readthedocs.io/). * [Model compression](doc/getting-started.md#compress-a-model). Accelerate the efficiency of model inference for 4-15 times. * [New descriptors](doc/getting-started.md#write-the-input-script). Including [`se_e2_r`](doc/train-se-e2-r.md) and [`se_e3`](doc/train-se-e3.md). * [Hybridization of descriptors](doc/train-hybrid.md). Hybrid descriptor constructed from concatenation of several descriptors. -* Atom type embedding. +* [Atom type embedding](doc/train-se-e2-a-tebd.md). Enable atom type embedding to decline training complexity and refine performance. * Training and inference the dipole (vector) and polarizability (matrix). * Split of training and validation dataset. * Optimized training on GPUs.