New heuristic improvements by akifcorduk · Pull Request #178 · NVIDIA/cuopt

akifcorduk · 2025-07-02T12:57:14Z

Description

This PR improves the heuristic quality and performance by following improvements:

Multi Armed Bandit algorithm for learning and selecting the more promising crossovers for a given problem.
Caching the integer-fixed problem and filling that problem structure rather than recreating the problem each time.
Improved bounds prop rounding by doing double probing on a bulk. Which means we propagate two sets of variable values which are promising. This is possible with almost no additional performance hit because most of the data are already in memory. It is only a few more vectors and few more access per propagation.
Update multi probe bounds presolve such that it can take
Diversity level adjustments by time progress.
Weight management such that the best solution is unlikely to be infeasible. We already normalize the weights but this is an added measure to prevent infeasible solutions flooding the population.
Improvement on bounds propagation recombiner, such that we do balanced rounding by trying to represent both parents. This is achieved by passing the probing candidates and counting the used candidates of each parent.
Improvement in all recombiner such that we use at least some objective variables in the recombiners. Before there was no guarantee that there was an objective component in the recombiners.
Dynamic selection of number of different variables used in the recombiners depending on if the recombiner brings the offspring back to one of the parents.
Improvement on line segment recombiner by doing a delta vector only on the subspace of different vars between parents.
Initialize the distance variables in FP to the distance of previous projection.
Improve line segment search such that it does a middle-first, binary search rather than a sequential search of the points.
Improve line segment search solution saving mechanism.
Fix a bug in get_two_random function.
Small improvements in feasibility jump.

Added functionalities:

Accepting multiple initial solutions instead of one.
Recombiner and diversity configurations files for better control.
FJ rounding of fractional solutions by adding weights on non-integrality. If within the time limit, all are not rounded, second stage only does rounding moves.

The improvements resulted in an average gap to optimality after 10mins on H100:
Regular run (B&B and heuristics): from 30% to ~23%
Heuristics only: from 33% to ~25%
On presolved instances: from 26% to ~19%

Issue

partially closes #140

Checklist

[ X] I am familiar with the Contributing Guidelines.
Testing
- [ X] New or existing tests cover these changes
- Added tests
- Created an issue to follow-up
- NA
Documentation
- [ X] The documentation is up to date with these changes
- Added new documentation
- NA

copy-pr-bot · 2025-07-02T12:57:17Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

…stic_improvements

akifcorduk · 2025-07-09T15:23:55Z

/ok to test 1fb206f

akifcorduk · 2025-07-09T15:36:39Z

/ok to test 1fb206f

akifcorduk · 2025-07-10T12:34:30Z

/ok to test b61f302

akifcorduk · 2025-07-10T15:57:00Z

/ok to test 4a3a0b5

akifcorduk · 2025-07-11T13:36:02Z

/ok to test f057b04

aliceb-nv

Thanks for the great work Akif!! Incredible improvement numbers

aliceb-nv · 2025-07-14T12:47:36Z

+    line_segment_search.settings = {};
+    bool better_cost_than_parents =
+      offspring.get_quality(weights) <
+      std::min(other_solution.get_quality(weights), guiding_solution.get_quality(weights));
+    bool better_feasibility_than_parents = offspring.get_feasible() &&
+                                           !other_solution.get_feasible() &&
+                                           !guiding_solution.get_feasible();
+    bool same_as_parents =
+      this->check_if_offspring_is_same_as_parents(offspring, guiding_solution, other_solution);
+    // adjust the max_n_of_vars_from_other
+    if (n_different_vars > (i_t)ls_recombiner_config_t::max_n_of_vars_from_other) {
+      if (same_as_parents) {
+        ls_recombiner_config_t::increase_max_n_of_vars_from_other();
+      } else {
+        ls_recombiner_config_t::decrease_max_n_of_vars_from_other();
+      }
+    }
+    if (better_cost_than_parents || better_feasibility_than_parents) {
+      CUOPT_LOG_DEBUG("Offspring is feasible or better than both parents");
+      return std::make_pair(offspring, true);
+    }


Nit: there's a fair amount of commonality in the code between the three recombiners - maybe recombine() could be turned into a member of the parent recombiner_t class? To write this common prologue and epilogue once (maybe using CRTP to call ls/fp/bp_recombiner_config_t members from the parent)

This is some larger change. Let's do it in another PR. I created an issue on that:
#208

aliceb-nv · 2025-07-14T13:05:15Z

+                    point_2.data() + solution.problem_ptr->n_variables,
+                    point_1.data(),
+                    delta_vector.begin(),
+                    [] __device__(const f_t& a, const f_t& b) { return a - b; });


Minor nit: I really doubt this has any noticeable effect in practice, but basic scalar types (like int/float/double/char/enums/etc) should be passed by value since passing by const ref/pointer just adds overhead

This must be either Cursor or some copy paste error. Fixed :)

aliceb-nv · 2025-07-14T13:16:46Z


+#include <cuda_profiler_api.h>
+


debugging leftover?

yes, fixed.

aliceb-nv · 2025-07-14T13:18:19Z

+      DEVICE_LOG_DEBUG(
+        "For cstr %d, var %d, value %f was not found in the transpose", constraint_id, col, value);


Did you run into a cusparse transpose issue?

Not related to cusparse, but in integer fixed problem. We are computing the integer fixed problem from the original problem and dynamically fill it, instead of recreating the problem every time. This was to make sure everything matches.

akifcorduk · 2025-07-15T09:29:27Z

/ok to test b7aa9da

akifcorduk · 2025-07-15T10:41:22Z

/merge

This PR improves the heuristic quality and performance by following improvements: - Multi Armed Bandit algorithm for learning and selecting the more promising crossovers for a given problem. - Caching the integer-fixed problem and filling that problem structure rather than recreating the problem each time. - Improved bounds prop rounding by doing double probing on a bulk. Which means we propagate two sets of variable values which are promising. This is possible with almost no additional performance hit because most of the data are already in memory. It is only a few more vectors and few more access per propagation. - Update multi probe bounds presolve such that it can take - Diversity level adjustments by time progress. - Weight management such that the best solution is unlikely to be infeasible. We already normalize the weights but this is an added measure to prevent infeasible solutions flooding the population. - Improvement on bounds propagation recombiner, such that we do balanced rounding by trying to represent both parents. This is achieved by passing the probing candidates and counting the used candidates of each parent. - Improvement in all recombiner such that we use at least some objective variables in the recombiners. Before there was no guarantee that there was an objective component in the recombiners. - Dynamic selection of number of different variables used in the recombiners depending on if the recombiner brings the offspring back to one of the parents. - Improvement on line segment recombiner by doing a delta vector only on the subspace of different vars between parents. - Initialize the distance variables in FP to the distance of previous projection. - Improve line segment search such that it does a middle-first, binary search rather than a sequential search of the points. - Improve line segment search solution saving mechanism. - Fix a bug in get_two_random function. - Small improvements in feasibility jump. Added functionalities: - Accepting multiple initial solutions instead of one. - Recombiner and diversity configurations files for better control. - FJ rounding of fractional solutions by adding weights on non-integrality. If within the time limit, all are not rounded, second stage only does rounding moves. The improvements resulted in an average gap to optimality after 10mins on H100: Regular run (B&B and heuristics): from 30% to ~23% Heuristics only: from 33% to ~25% On presolved instances: from 26% to ~19% ## Issue partially closes NVIDIA#140 Authors: - Akif ÇÖRDÜK (https://github.com/akifcorduk) Approvers: - Alice Boucher (https://github.com/aliceb-nv) - Ramakrishnap (https://github.com/rgsl888prabhu) URL: NVIDIA#178

akifcorduk added 7 commits July 1, 2025 04:49

first merge: merging only the mip directory

1bf2309

add more changes from include directory

d49d073

fix merge conflicts

542e4b4

fix merge conflicts

6af2454

fix more compile errors

24cbd7c

remove patch files

18bfa3c

remove patch files

804d3d5

akifcorduk added this to the 25.08 milestone Jul 2, 2025

akifcorduk added the non-breaking Introduces a non-breaking change label Jul 2, 2025

akifcorduk requested a review from a team as a code owner July 2, 2025 12:57

akifcorduk added improvement Improves an existing functionality mip labels Jul 2, 2025

akifcorduk requested review from Kh4ster and kaatish July 2, 2025 12:57

akifcorduk added 5 commits July 2, 2025 06:01

remove more unnecessary code

4040daf

log to console by default

345cc5e

add default lp state for fixed and normal problem

ff42e2c

add var_t header to host_helpers

f77e436

Merge branch 'branch-25.08' of github.com:NVIDIA/cuopt into new_heuri…

3200fb8

…stic_improvements

hlinsen reviewed Jul 8, 2025

View reviewed changes

akifcorduk added 5 commits July 9, 2025 07:38

handle reviews

eefe732

don't use max and compare internal objective

71cabf8

Merge branch 'branch-25.08' of github.com:NVIDIA/cuopt into new_heuri…

6f5fbc7

…stic_improvements

fix conflicts

9aeb920

fix some logs

1fb206f

fix cython build issues

07f0bee

akifcorduk requested a review from a team as a code owner July 10, 2025 10:55

akifcorduk requested a review from rgsl888prabhu July 10, 2025 10:55

akifcorduk added 2 commits July 10, 2025 04:12

add back base solution inheritance

8bd2a79

fix sol bound in root relaxation

b61f302

remove the epsilon in better solution check

4a3a0b5

fix adding sols from B&B, avoid timer breaking early

f057b04

aliceb-nv approved these changes Jul 14, 2025

View reviewed changes

rgsl888prabhu approved these changes Jul 14, 2025

View reviewed changes

akifcorduk added 2 commits July 15, 2025 01:29

handle some review comments

f34089e

address more review comments

b7aa9da

rapids-bot bot merged commit 80c0df8 into NVIDIA:branch-25.08 Jul 15, 2025
72 checks passed

		DEVICE_LOG_DEBUG(
		"For cstr %d, var %d, value %f was not found in the transpose", constraint_id, col, value);


		#include <cuda_profiler_api.h>

Conversation

akifcorduk commented Jul 2, 2025

Description

Issue

Checklist

Uh oh!

copy-pr-bot bot commented Jul 2, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

akifcorduk commented Jul 9, 2025

Uh oh!

akifcorduk commented Jul 9, 2025

Uh oh!

akifcorduk commented Jul 10, 2025

Uh oh!

akifcorduk commented Jul 10, 2025

Uh oh!

akifcorduk commented Jul 11, 2025

Uh oh!

aliceb-nv left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

akifcorduk commented Jul 15, 2025

Uh oh!

akifcorduk commented Jul 15, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants