DRHO converges much slower than EDIFF

[Shen-Zhen-Xiong]

Describe the bug

I remember in old versions of ABAUS， for scf calculation, DRHO converges almost faster than EDIFF, but after some 3.2 version of ABACUS , DRHO converges much slower than EDIFF. For example While DRHO converges to 1e-6, EDIFF only converges to 2e-05. Especially, I find a case(scan_nook_case) which DRHO has converged to 1e-05, but EDIFF only converges to -2, it is terrible。
issue_drho.tar.gz

Expected behavior

No response

To Reproduce

No response

Environment

No response

Additional Context

No response

Task list for Issue attackers (only for developers)

• Verify the issue is not a duplicate.
• Describe the bug.
• Steps to reproduce.
• Expected behavior.
• Error message.
• Environment details.
• Additional context.
• Assign a priority level (low, medium, high, urgent).
• Assign the issue to a team member.
• Label the issue with relevant tags.
• Identify possible related issues.
• Create a unit test or automated test to reproduce the bug (if applicable).
• Fix the bug.
• Test the fix.
• Update documentation (if necessary).
• Close the issue and inform the reporter (if applicable).

[sunliang98]

I have runned 3.4.0-sg15-34step, 3.4.0-sg15-no-mix-36step, and scan_notok_case under issue_drho, and found no problems in the first two cases, while the last case is surely strange.
I have created an example containing two Fe atoms according to the issue_drho/scan_notok_case, and found the same phenomenon as mentioned by this issue. Then I run this case with PW basis, the scf cannot get convergence, and after setting mixing_type broyden, mixing_gg0 0.1, mixing_beta 0.4，the PW case got convergence with drho~1e-10 and dE~1e-5 Ry, dE is still larger than drho, but it is small enough.
I think this strange phenomenon can be attributed to three things:

1. The stability of SCAN XC, especially in this SPIN 2 case, and LCAO basis may magnify this issue;
2. The stability of Pulay mixing, we note that Broyden mixing is usually faster and more stable than Pulay mixing;
3. The chosen of parameters, especially mixing_gg0 and mixing_beta.

As a result, I suggest to run this case in PW basis with above parameters, or to select a more stable XC than SCAN XC.

Two things more may help:

1. In the old version of ABACUS, the "Pulay mixing" is actually Broyden mixing with different definition of inner product, and we have implemented the true Pulay mixing few weeks ago (pr2998), so that when comparing to the old version ABACUS with mixing_type pulay, one should set mixing_type broyden, mixing_ndim 7 in the latest version ABACUS;
2. With different basis, ABACUS will use different definition of inner product and corresponding scf threshold, which is explained in scf_thr_type.

[WHUweiqingzhou]

@Shen-Zhen-Xiong @sunliang98
metaGGA calculation with nspin=2 can be updated after PR #3328, to check whether the problem is still there.

[Shen-Zhen-Xiong]

Thanks, I will test

[Shen-Zhen-Xiong]

issue_drho_v2.tar.gz
I have test the meta-GGA case(scan_notok_case) for nspin=2 with abacus-3.5.1 version. Indeed, there has been some improvement. While DRHO converges to 9.09e-7, EDIFF converges to -1.47e-4, though DRHO also converges slower than EDIFF. I noticed that “Total Magnetism on atom: Fe” are 2.331 and 2.197 for SCAN, but they are all 2.286 for PBE. I am not sure it is ok?

[WHUweiqingzhou]

@Shen-Zhen-Xiong,
as I discussed in Issues #3562, current get_drho is incomplete for meta-GGA. It is the reason why EDIFF is much larger than DRHO. It has also been reported in #3390.
And as I discussed in Issue #3438, DRHO is acutally the difference between $\rho_{i,out}$ and $\rho_{i+1,in}$ after charge mixing. So I believe you can make more balance between DRHO and EDIFF by setting mixing_tau=true in INPUT file. But it is just a tricky solution, not a physical one.
I plan to fix this problem in next two steps:

1. fix Feature: Unify the definition of get_drho and get_dene for meta-GGA calculations #3562
2. implement a new mixing method for meta_GGA Feature: change mixing for TAU of metaGGA by defining residuals with rho, magnetism and tau together. #3335.

[Shen-Zhen-Xiong]

scan_ok.tar.gz
@WHUweiqingzhou ,
after a setting "mixing_tau=true" in INPUT file, I find EDIFF converges to 1.554114e-06 while DRHO converging to 6.864e-06, and EDIFF converges to 2.968845e-07 while DRHO converging to 8.458e-07, that is ok. I hope your plan to fix the problem in a physical one.
But "Total Magnetism on atom: Fe" in mulliken.txt file also has two unequivalent values. Is this correct？