Reproducibility 3/3 #1924
Reproducibility 3/3 #1924
Conversation
|
The documentation is not available anymore as the PR was closed or merged. |
…make_tests_deterministic
…make_tests_deterministic
patil-suraj
left a comment
patil-suraj
left a comment
There was a problem hiding this comment.
Great PR! It must have taken a lot of time to change all these tests! I mostly left some nits.
| ddim.to("cuda") | ||
|
|
||
| # create a generator for reproducibility | ||
| generator = torch.manual_seed(0) |
There was a problem hiding this comment.
Should we still use a torch.Generator here? Using manual_seed will set the global seed, no?
There was a problem hiding this comment.
Think torch.Generator does that as well
Co-authored-by: Suraj Patil <surajp815@gmail.com>
…face/diffusers into make_tests_deterministic
| return True | ||
|
|
||
|
|
||
| def print_tensor_test(tensor, filename="test_corrections.txt", expected_tensor_name="expected_slice"): |
There was a problem hiding this comment.
BTW @pcuenca @patil-suraj @yiyixuxu @williamberman
I added some functionality that helps to automatically correct slice-comparison tests.
All you need to do here is to add:
print_tensor_test(image_slice)and it'll write the correct tensor to a file: filename with the name expected_tensor_name.
The utils file here: https://github.com/huggingface/diffusers/pull/1924/files#diff-bd9ba404f47129f2218fc94c5e528cc81bfd10cef1eff1d6d543f6ffa0143367 can then take this file and will automatically overwrite the test with the correct results. This should save quite some time going forward :-)
There was a problem hiding this comment.
That's very cool, thanks a lot !
* make tests deterministic * run slow tests * prepare for testing * finish * refactor * add print statements * finish more * correct some test failures * more fixes * set up to correct tests * more corrections * up * fix more * more prints * add * up * up * up * uP * uP * more fixes * uP * up * up * up * up * fix more * up * up * clean tests * up * up * up * more fixes * Apply suggestions from code review Co-authored-by: Suraj Patil <surajp815@gmail.com> * make * correct * finish * finish Co-authored-by: Suraj Patil <surajp815@gmail.com>
* make tests deterministic * run slow tests * prepare for testing * finish * refactor * add print statements * finish more * correct some test failures * more fixes * set up to correct tests * more corrections * up * fix more * more prints * add * up * up * up * uP * uP * more fixes * uP * up * up * up * up * fix more * up * up * clean tests * up * up * up * more fixes * Apply suggestions from code review Co-authored-by: Suraj Patil <surajp815@gmail.com> * make * correct * finish * finish Co-authored-by: Suraj Patil <surajp815@gmail.com>
Last PR of the series after #1906.
In this PR, I'll update all tests to use CPU generators for better determinism and also add a reproducible doc page.