Skip to content

benchmark against PATH#21

Merged
dfridovi merged 12 commits intomainfrom
benchmark/path
Dec 25, 2024
Merged

benchmark against PATH#21
dfridovi merged 12 commits intomainfrom
benchmark/path

Conversation

@dfridovi
Copy link
Copy Markdown
Member

@dfridovi dfridovi commented Dec 24, 2024

Adds support for benchmarking against the PATH solver.

  • Get solve times for random QPs
  • Compute summary statistics
  • Tune IP solver to improve metrics
  • Make QPs sparse (sparsity is only known at runtime though, not compile time)

Adding additional solver kwargs:

  • tightening_rate (default 1.0): if an inner loop succeeds, scales ε by 1 - exp(-tightening_rate * inner_iters)
  • loosening_rate (default 1.0): if an inner loop fails, scales ε by 1 + exp(-loosening_rate * inner_iters)
  • verbose (default false): only print warnings if true

Also, adding a small amount of regularization to the Newton step computation for numerical stability.

@dfridovi
Copy link
Copy Markdown
Member Author

For documentation purposes, at 5a1b9ad the stats are

julia> stats = SolverBenchmarks.summary_statistics(data)
(ip = (success_rate = 0.995, μ = 0.05398038283718592, σ = 0.00722662057721707), path = (success_rate = 0.995, μ = 0.007355797381909546, σ = 0.0016724800239494542))

So... we are about 10x slower than PATH.

@dfridovi
Copy link
Copy Markdown
Member Author

dfridovi commented Dec 25, 2024

And now only 5x slower

julia> data = SolverBenchmarks.benchmark(; data.ip_mcp, data.path_mcp, ip_kwargs = (; tightening_rate = 0.05, loosening_rate = 0.5, max_inner_iters = 5, tol = 1e-6))
julia> stats = SolverBenchmarks.summary_statistics(data)
(ip = (success_rate = 0.995, μ = 0.03280799303115578, σ = 0.005423971573040027), path = (success_rate = 0.995, μ = 0.007583776933668342, σ = 0.0022406690554635476))

@dfridovi dfridovi marked this pull request as ready for review December 25, 2024 00:23
@dfridovi dfridovi merged commit 4244684 into main Dec 25, 2024
@dfridovi dfridovi deleted the benchmark/path branch December 25, 2024 00:23
@dfridovi dfridovi mentioned this pull request Dec 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant