store energy bias with interface precision

[njzjz]

This PR moves energy biases out of NN for all situations and stores them with interface precision.

When using the FP64 precision interface and FP32 precision NN, this patch can improve the accuracy of the atomic energy when it has a large absolute value. For example, when the atomic energy is 11451.41234567 eV, the FP32 value is 11451.412 eV (places=3); but if we have an FP64 bias of 11450.000000 eV, the NN only needs to fit 1.41234567 eV, and the FP32 value is 1.4123456 eV (places=7).

[njzjz]

@wanghan-iapcm @denghuilu @iProzd Discussion is welcome.

[wanghan-iapcm]

As far as I see, apart from the precision of the bias_atom_e, the implement by this PR should be equivalent to the old one. Am I correct?

[njzjz]

As far as I see, apart from the precision of the bias_atom_e, the implement by this PR should be equivalent to the old one. Am I correct?

This PR brings the implementation for the type embedding in #1592 and #1866 to other cases.

Before, the bias is stored in the bias of the last layer in the fitting network. This PR creates a new constant variable (not trainable) for it, as we did for the type embedding in #1592 and #1866.

[wanghan-iapcm]

As far as I see, apart from the precision of the bias_atom_e, the implement by this PR should be equivalent to the old one. Am I correct?

This PR brings the implementation for the type embedding in #1592 and #1866 to other cases.

Before, the bias is stored in the bias of the last layer in the fitting network. This PR creates a new constant variable (not trainable) for it, as we did for the type embedding in #1592 and #1866.

I want to confirm for the standard cases (like se_a) the PR gives equivalent result as the old implementation.

[njzjz]

I want to confirm for the standard cases (like se_a) the PR gives equivalent result as the old implementation.

cd examples/water/se_e2_a
dp train input.json

devel:

#  step      rmse_val    rmse_trn    rmse_e_val  rmse_e_trn    rmse_f_val  rmse_f_trn         lr
      0      2.65e+01    2.76e+01      6.77e-01    6.79e-01      8.38e-01    8.71e-01    1.0e-03
    100      1.00e+01    9.86e+00      2.62e-01    2.53e-01      3.17e-01    3.11e-01    1.0e-03
    200      6.61e+00    6.83e+00      9.79e-02    9.06e-02      2.09e-01    2.16e-01    1.0e-03
    300      5.93e+00    5.74e+00      2.10e-02    1.91e-02      1.88e-01    1.81e-01    1.0e-03
    400      6.82e+00    6.73e+00      1.78e-01    1.80e-01      2.15e-01    2.13e-01    1.0e-03
    500      5.44e+00    5.26e+00      4.11e-02    4.33e-02      1.72e-01    1.66e-01    1.0e-03
    600      5.06e+00    5.58e+00      5.22e-02    5.26e-02      1.60e-01    1.76e-01    1.0e-03
    700      5.33e+00    5.23e+00      1.96e-02    1.76e-02      1.69e-01    1.65e-01    1.0e-03
    800      4.64e+00    4.29e+00      5.06e-02    5.19e-02      1.47e-01    1.36e-01    1.0e-03
    900      4.85e+00    4.44e+00      1.38e-02    1.12e-02      1.53e-01    1.41e-01    1.0e-03
   1000      4.66e+00    4.02e+00      1.19e-03    1.78e-03      1.48e-01    1.27e-01    1.0e-03

This PR:

#  step      rmse_val    rmse_trn    rmse_e_val  rmse_e_trn    rmse_f_val  rmse_f_trn         lr
      0      2.65e+01    2.76e+01      6.77e-01    6.79e-01      8.38e-01    8.71e-01    1.0e-03
    100      1.00e+01    9.86e+00      2.62e-01    2.53e-01      3.17e-01    3.11e-01    1.0e-03
    200      6.61e+00    6.83e+00      9.79e-02    9.06e-02      2.09e-01    2.16e-01    1.0e-03
    300      5.93e+00    5.74e+00      2.10e-02    1.91e-02      1.88e-01    1.81e-01    1.0e-03
    400      6.82e+00    6.73e+00      1.78e-01    1.80e-01      2.15e-01    2.13e-01    1.0e-03
    500      5.44e+00    5.26e+00      4.11e-02    4.33e-02      1.72e-01    1.66e-01    1.0e-03
    600      5.06e+00    5.58e+00      5.22e-02    5.26e-02      1.60e-01    1.76e-01    1.0e-03
    700      5.33e+00    5.23e+00      1.96e-02    1.76e-02      1.69e-01    1.65e-01    1.0e-03
    800      4.64e+00    4.29e+00      5.06e-02    5.19e-02      1.47e-01    1.36e-01    1.0e-03
    900      4.85e+00    4.44e+00      1.38e-02    1.12e-02      1.53e-01    1.41e-01    1.0e-03
   1000      4.66e+00    4.02e+00      1.19e-03    1.78e-03      1.48e-01    1.27e-01    1.0e-03

The outputs are the same.