Skip to content

test: add benchmarks with principal#374

Merged
maksymar merged 2 commits intomainfrom
maksym/principal
Jul 15, 2025
Merged

test: add benchmarks with principal#374
maksymar merged 2 commits intomainfrom
maksym/principal

Conversation

@maksymar
Copy link
Copy Markdown
Contributor

@maksymar maksymar commented Jul 14, 2025

This PR adds benchmarks for Principal type to BTreeMap.

@maksymar maksymar requested a review from Copilot July 14, 2025 14:24
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds random data support for candid::Principal and unit type, and extends BTreeMap benchmarks to include Principal keys/values and zero-length value cases. Key changes:

  • Implemented Random for Principal and () in random.rs.
  • Introduced Empty alias and added numerous bench_tests! entries in main.rs for zero-length and Principal scenarios.
  • Updated canbench_results.yml with new benchmark results for the added tests.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
benchmarks/src/random.rs Added Random impls for Principal and unit () types.
benchmarks/btreemap/src/main.rs Added Empty alias and new bench_tests! cases for Principal and zero-length values.
benchmarks/btreemap/canbench_results.yml Updated YAML with results for the new benchmarks.
Comments suppressed due to low confidence (3)

benchmarks/src/random.rs:51

  • Add unit tests for the Random implementation of Principal to verify generated principals are valid and within MAX_LENGTH_IN_BYTES.
impl Random for Principal {

benchmarks/src/random.rs:62

  • Add a basic test for the Random impl of () to ensure it always returns () as expected.
impl Random for () {

benchmarks/btreemap/src/main.rs:31

  • [nitpick] The alias Empty is very generic; consider renaming it to something like UnitValue or EmptyValue for clearer intent.
type Empty = ();

Comment thread benchmarks/btreemap/src/main.rs
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jul 14, 2025

canbench 🏋 (dir: ./benchmarks/io_chunks) 5bdf9b7 2025-07-15 08:49:54 UTC

./benchmarks/io_chunks/canbench_results.yml is up to date
📦 canbench_results_io_chunks.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes 👍
    counts:   [total 18 | regressed 0 | improved 0 | new 0 | unchanged 18]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 18 | regressed 0 | improved 0 | new 0 | unchanged 18]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 18 | regressed 0 | improved 0 | new 0 | unchanged 18]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jul 14, 2025

canbench 🏋 (dir: ./benchmarks/btreeset) 5bdf9b7 2025-07-15 08:49:22 UTC

./benchmarks/btreeset/canbench_results.yml is up to date
📦 canbench_results_btreeset.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes 👍
    counts:   [total 100 | regressed 0 | improved 0 | new 0 | unchanged 100]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 100 | regressed 0 | improved 0 | new 0 | unchanged 100]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 100 | regressed 0 | improved 0 | new 0 | unchanged 100]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jul 14, 2025

canbench 🏋 (dir: ./benchmarks/vec) 5bdf9b7 2025-07-15 08:49:08 UTC

./benchmarks/vec/canbench_results.yml is up to date
📦 canbench_results_vec.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jul 14, 2025

canbench 🏋 (dir: ./benchmarks/nns) 5bdf9b7 2025-07-15 08:49:20 UTC

./benchmarks/nns/canbench_results.yml is up to date
📦 canbench_results_nns.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jul 14, 2025

canbench 🏋 (dir: ./benchmarks/memory_manager) 5bdf9b7 2025-07-15 08:49:04 UTC

./benchmarks/memory_manager/canbench_results.yml is up to date
📦 canbench_results_memory-manager.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes 👍
    counts:   [total 3 | regressed 0 | improved 0 | new 0 | unchanged 3]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 3 | regressed 0 | improved 0 | new 0 | unchanged 3]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 3 | regressed 0 | improved 0 | new 0 | unchanged 3]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jul 14, 2025

canbench 🏋 (dir: ./benchmarks/btreemap) 5bdf9b7 2025-07-15 08:50:52 UTC

./benchmarks/btreemap/canbench_results.yml is up to date
📦 canbench_results_btreemap.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   New benchmarks added ➕
    counts:   [total 303 | regressed 0 | improved 0 | new 18 | unchanged 285]
    change:   [max +188.14K | p75 0 | median -3.62K | p25 -4.03K | min -1.55M]
    change %: [max +0.02% | p75 0.00% | median -0.00% | p25 -0.00% | min -0.03%]

  heap_increase:
    status:   New benchmarks added ➕
    counts:   [total 303 | regressed 0 | improved 0 | new 18 | unchanged 285]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   New benchmarks added ➕
    counts:   [total 303 | regressed 0 | improved 0 | new 18 | unchanged 285]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------

Only significant changes:
| status | name                            | calls |     ins |  ins Δ% | HI |  HI Δ% | SMI |  SMI Δ% |
|--------|---------------------------------|-------|---------|---------|----|--------|-----|---------|
|  new   | btreemap_v2_contains_blob_32_0  |       | 329.57M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_contains_principal  |       | 349.06M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_contains_vec_32_0   |       | 355.32M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_get_blob_32_0       |       | 337.43M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_get_principal       |       | 356.43M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_get_vec_32_0        |       | 358.22M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_insert_blob_32_0    |       | 490.46M |         |  0 |        |   8 |         |
|  new   | btreemap_v2_insert_principal    |       | 503.30M |         |  0 |        |   8 |         |
|  new   | btreemap_v2_insert_vec_32_0     |       | 621.85M |         |  0 |        |  20 |         |
|  new   | btreemap_v2_pop_first_blob_32_0 |       | 751.49M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_pop_first_principal |       | 800.19M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_pop_first_vec_32_0  |       | 862.94M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_pop_last_blob_32_0  |       | 723.33M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_pop_last_principal  |       | 780.29M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_pop_last_vec_32_0   |       | 846.11M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_remove_blob_32_0    |       | 654.25M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_remove_principal    |       | 684.15M |         |  0 |        |   0 |         |
|  new   | btreemap_v2_remove_vec_32_0     |       | 834.33M |         |  0 |        |   0 |         |

ins = instructions, HI = heap_increase, SMI = stable_memory_increase, Δ% = percent change

---------------------------------------------------
CSV results saved to canbench_results.csv

@maksymar maksymar marked this pull request as ready for review July 14, 2025 14:40
@maksymar maksymar requested a review from a team as a code owner July 14, 2025 14:40
Comment thread benchmarks/src/random.rs Outdated
@maksymar maksymar enabled auto-merge (squash) July 15, 2025 08:42
@maksymar maksymar merged commit a479340 into main Jul 15, 2025
16 checks passed
@maksymar maksymar deleted the maksym/principal branch July 15, 2025 08:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants