[BUG] Loss of energy and force failed to decline with type-embedding

[Ericwang6]

Summary

When I use type-embedding feature in deepmd-kit-2.0.0b0, the loss of energy and force failed to decline during the training process. (Similar to issue #678)

Deepmd-kit version, installation way, input file, running commands, error log, etc.

• deepmd-kit version: v2.0.0b0
• Installation: from source code
• Running environment: PBS system
• input.json:

{
    "model": {
        "type_map": [
            "C",
            "H",
            "N",
            "O"
        ],
       "type_embedding": {
            "neuron": [
                4,
                4,
                4
            ],
            "resnet_dt": false,
            "seed": 1
        },
        "descriptor": {
            "type": "se_e2_a",
            "sel": [
                48,
                40,
                48,
                48
            ],
            "rcut_smth": 0.5,
            "rcut": 6.0,
            "neuron": [
                20,
                40,
                80
            ],
            "resnet_dt": false,
            "axis_neuron": 8,
            "type_one_side": true,
            "seed": 1,
            "activation_function": "gelu"
        },
        "fitting_net": {
            "neuron": [
                240,
                240,
                240
            ],
            "resnet_dt": true,
            "seed": 1,
            "activation_function": "gelu"
        }
    },
    "learning_rate": {
        "type": "exp",
        "start_lr": 0.0001,
        "stop_lr": 5e-8,
        "decay_steps": 500
    },
    "loss": {
        "type": "ener",
        "start_pref_e": 0.02,
        "limit_pref_e": 10,
        "start_pref_f": 1000,
        "limit_pref_f": 1,
        "start_pref_v": 0,
        "limit_pref_v": 0
    },
    "training": {
        "numb_steps": 100000,
        "disp_file": "lcurve.out",
        "disp_freq": 1000,
        "numb_test": 1,
        "save_freq": 1000,
        "save_ckpt": "model.ckpt",
        "disp_training": true,
        "time_training": true,
        "training_data": {
            "batch_size": "auto",
            "systems": [
                "C4H3N1O4",
                "C6H11N7O1",
                "C8H19N1O0",
                "C6H12N0O1",
                "C12H12N2O1"
            ]
        }
    }
}

Steps to Reproduce

See the attachment file.
issue-type-embedding.zip

[Ericwang6]

Besides, with type-embedding feature, I didn't met the error that occurred when using se_e2_a only. (as reported in issue #675 )

[njzjz]

I have the same issue. In the very beginning, the loss of both energy and force is much larger then that without type embedding.

[njzjz]

I think the reason is that the atom energy bias is set to 0..

deepmd-kit/deepmd/fit/ener.py

Lines 487 to 491 in c49533e

	final_layer = self._build_lower(
	0, natoms[0],
	inputs, fparam, aparam,
	bias_atom_e=0.0, suffix=suffix, reuse=reuse
	)

[Ericwang6]

What is atomic energy bias for? Is it designed to handle cases where the absolute value of target energy is too large？

[njzjz]

What is atomic energy bias for? Is it designed to handle cases where the absolute value of target energy is too large？

Yes. In theory, it should be near the atomic energy in vacuum.

[njzjz]

@Ericwang6 #1592 should fix this issue. You can try it on your system.