Skip to content

feat: support loading stable memory from a file#76

Merged
dsarlis merged 4 commits intomainfrom
ielashi/stable_memory
Dec 6, 2024
Merged

feat: support loading stable memory from a file#76
dsarlis merged 4 commits intomainfrom
ielashi/stable_memory

Conversation

@ielashi
Copy link
Copy Markdown
Collaborator

@ielashi ielashi commented Dec 5, 2024

Problem

Running benchmarks that require a particular stable memory is currently not possible with canbench. I ran into this problem quite often when wanting to benchmark AI models, and I'm sure there are other use-cases for it as well.

Solution

This PR introduces a way to specify a binary file in canbench.yml that will be loaded as the canister's stable memory after initialization.

Example:

stable_memory:
  file:
    stable_memory.bin

@ielashi ielashi requested a review from a team as a code owner December 5, 2024 14:51
dsarlis
dsarlis previously approved these changes Dec 5, 2024
Comment thread canbench-rs/src/lib.rs Outdated
@dsarlis
Copy link
Copy Markdown
Contributor

dsarlis commented Dec 5, 2024

I think this also suffers from the same issue as #65. Namely that some benchmark is giving different results on different platforms.

@ielashi
Copy link
Copy Markdown
Collaborator Author

ielashi commented Dec 6, 2024

I think this also suffers from the same issue as #65. Namely that some benchmark is giving different results on different platforms.

I addressed by making the test only ensure success. We don't actually need to assert on the exact instruction count, as that is quite brittle.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Dec 6, 2024

canbench 🏋 (dir: examples/fibonacci)

No significant performance changes detected ✅

examples/fibonacci/canbench_results.yml is up to date ✅


---------------------------------------------------

Benchmark: fibonacci_20
  total:
    instructions: 2224 (no change)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: fibonacci_45
  total:
    instructions: 3011 (no change)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Dec 6, 2024

canbench 🏋 (dir: examples/btreemap_vs_hashmap)

No significant performance changes detected ✅

examples/btreemap_vs_hashmap/canbench_results.yml is up to date ✅


---------------------------------------------------

Benchmark: insert_users
  total:
    instructions: 2.57 B (no change)
    heap_increase: 871 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: remove_users
  total:
    instructions: 2.07 B (no change)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: pre_upgrade_bench
  total:
    instructions: 743.98 M (0.00%) (change within noise threshold)
    heap_increase: 519 pages (no change)
    stable_memory_increase: 184 pages (no change)

  serialize_state (scope):
    instructions: 729.05 M (0.00%) (change within noise threshold)
    heap_increase: 519 pages (no change)
    stable_memory_increase: 0 pages (no change)

  writing_to_stable_memory (scope):
    instructions: 14.93 M (no change)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 184 pages (no change)

---------------------------------------------------

@dsarlis dsarlis merged commit 442c800 into main Dec 6, 2024
@dsarlis dsarlis deleted the ielashi/stable_memory branch December 6, 2024 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants