Fix: Corrected the external archive error in the jSO algorithm

[Tomato-in]

Problem Description: In the jSO algorithm, the external archive was storing successful trials, but according to the original paper, it should store eliminated individuals. The external archive is meant to maintain diversity, and storing successful trials fails to achieve this effect.

Solution: Modified the jSO algorithm's external archive update to store eliminated individuals.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.91%. Comparing base (9f03f29) to head (9c329c0).
Report is 1 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #1263   +/-   ##
========================================
  Coverage    93.91%   93.91%           
========================================
  Files          661      661           
  Lines        54866    54866           
========================================
  Hits         51526    51526           
  Misses        3340     3340           

Files with missing lines	Coverage Δ
include/boost/math/optimization/jso.hpp	81.21% <100.00%> (ø)

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9f03f29...9c329c0. Read the comment docs.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[NAThompson]

@Tomato-in : You are correct-this was a mistake on my part. Run is approved.

Do you have any ideas how to test this particular diff?

[Tomato-in]

@NAThompson Yes, we can design a set of experiments to verify whether 'storing eliminated individuals to increase diversity can prevent premature convergence'. The experimental code can be found here.

1. Experiment Design:

   • Objective Function

     double rosenbrock(std::vector<double> const & x) {
        double result = 0;
        for (size_t i = 0; i < x.size() - 1; ++i) {
            double tmp = x[i+1] - x[i]*x[i];
            result += 100*tmp*tmp + (1-x[i])*(1-x[i]); 
        }
        return result;
     }

   • Solution Space: [0, 100]^dimension
   • Dimensions: 10, 20, 30, 40, 50, 60, 70, 80, 90, 100
   • Repetitions: 20 experiments per dimension

2. Experimental Environment:

   • Operating System: Ubuntu 22.04.5 LTS (Jammy Jellyfish)
   • C++ Compiler: GCC 13.1.0 (g++ (Ubuntu 13.1.0-8ubuntu1~22.04))

3. Experimental Results:
   In the graphs, red represents the convergence of jSO_FIX, where the lower bound of the red fill shows the best results from 20 experiments, the upper bound shows the worst results, and the red solid line represents the average of all 20 experiments. Blue represents jSO's convergence in a similar manner.

   • Results for Dimensions 10-30:
     
     
     
     Observing the convergence comparison graphs for dimensions 10-30, we can draw two conclusions:
     (1). jSO_FIX outperforms jSO in terms of average results, best results, and worst results.
     (2). jSO_FIX's convergence graphs consistently show two "inflection points," demonstrating its ability to escape local optima.

   • Results for Dimension 40:
     
     Conclusion: jSO_FIX outperforms jSO in terms of average results, best results, and worst results.

   • Results for Dimensions 50-70:
     
     
     
     Conclusion: jSO_FIX underperforms compared to jSO in terms of average results, best results, and worst results.

   • Results for Dimensions 80-100:
     
     
     

     Conclusion: jSO_FIX underperforms compared to jSO in terms of average results, but achieves better best results.

   Summary of Findings

   Metric\Dimension	10	20	30	40	50	60	70	80	90	100
   Average Results	✓	✓	✓	✓	✗	✗	✗	✗	✗	✗
   Best Results	✓	✓	✓	✓	✗	✗	✗	✓	✓	✓
   Worst Results	✓	✓	✓	✓	✗	✗	✗	-	-	-
   Escaping Local Optima	✓	✓	✓	-	-	-	-	-	-	-

   Note: ✓ indicates jSO_FIX performs better, ✗ indicates jSO performs better, - indicates no significant difference.

4. Additional Experiments

   From the experiments above, we can see that for dimensions 10-40, jSO_FIX can obtain better solutions than jSO, and demonstrates the ability to escape local optima in dimensions 10-30.

   However, when the dimension exceeds 40, jSO_FIX no longer shows significant advantages. I believe the main reason is that jSO_FIX fails to demonstrate the ability to escape local optima in dimensions above 30.

   Therefore, I supplemented with an additional set of experiments. Compared to the previous experimental setup, only the algorithm's max_function_evaluations was changed from the default value of 10000 * D to 10000 * D * ln(D). The experimental results are as follows:

As can be seen, after increasing the number of algorithm evaluations, jSO_FIX can consistently escape local optima, which is extremely evident in dimensions 10-80.

Therefore, we can draw the following conclusion:
When max_function_evaluations is set appropriately, storing eliminated individuals to increase diversity can prevent premature convergence.

[NAThompson]

Very detailed analysis; thanks!

@mborland : Do you know if the CI failure is "known" or spurious somehow?

[mborland]

Very detailed analysis; thanks!

@mborland : Do you know if the CI failure is "known" or spurious somehow?

Known. I told @jzmaddock I would fix it, but then I had to take some unplanned time off. I haven't forgotten.

[NAThompson]

@mborland : Merge it over the failure?

[mborland]

@mborland : Merge it over the failure?

Yes

[NAThompson]

@Tomato-in : Thanks! This was a great first commit!