Skip to content

[Object Spilling] Skip flaky tests#13628

Merged
rkooo567 merged 4 commits intoray-project:masterfrom
amogkam:skip-object-spilling
Jan 22, 2021
Merged

[Object Spilling] Skip flaky tests#13628
rkooo567 merged 4 commits intoray-project:masterfrom
amogkam:skip-object-spilling

Conversation

@amogkam
Copy link
Contributor

@amogkam amogkam commented Jan 22, 2021

Why are these changes needed?

Related issue number

Checks

  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

rkooo567
rkooo567 approved these changes Jan 22, 2021
View reviewed changes
Copy link
Contributor

@rkooo567 rkooo567 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, test_delete_objects_delete_while_creating is also flaky I think. Interestingly, this fails only on Mac build... I suspect the issue is just simply the current test suites are too stressful for Mac hardware.

@amogkam
Copy link
Contributor Author

amogkam commented Jan 22, 2021

Oh yes good catch about test_delete_objects_delete_while_creating, I added a skip for that one as well. You could very well be right about why it's failing only Mac.

This test been very flaky (>30%) though so I think we should go ahead and merge this. It's up to you/core team on how to resolve it. Skipping the tests just for Mac does seem like a viable option.

@rkooo567
Copy link
Contributor

rkooo567 commented Jan 22, 2021

Yeah can you skip this only for Mac? I will make a PR that reduces the stress of tests and if it can fix the issue.

@amogkam
Copy link
Contributor Author

amogkam commented Jan 22, 2021

Ok I skipped only for Mac

@rkooo567 rkooo567 merged commit 00c14ce into ray-project:master Jan 22, 2021
fishbone pushed a commit to fishbone/ray that referenced this pull request Feb 16, 2021
@amogkam @fishbone
[Object Spilling] Skip flaky tests (ray-project#13628)
4d3c8d1 
* skip flaky tests

* lint

* skip one more

* fix
fishbone added a commit to fishbone/ray that referenced this pull request Feb 16, 2021
@fishbone
Revert "[Object Spilling] Skip flaky tests (ray-project#13628)"
450cf1b 
This reverts commit 4d3c8d1.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rkooo567 rkooo567 rkooo567 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@amogkam @rkooo567