Ignore "index with different options already exists" mongoengine errors #3202
Conversation
Right now we can't do anything about those errors, proper solution would involve, re-creating the index, but this could be very expensive and blocking.
|
To give some context - MongoDB 3.4 is much stricter with index validation and uid field index creation would fail on bootstrap when running on existing installation because index with the same name, but different fields already exist ( This would cause a failure, but the error is not actually fatal - old / existing index still works just fine. As mentioned above, real long-term solution is to re-work the ensure_indexes code to re-create offending indexes, but we will still need to be careful we don't mess with indexes in such manner unless is really needed because it re-creation would be very slow and expensive. |
Codecov Report
@@ Coverage Diff @@
## master #3202 +/- ##
==========================================
+ Coverage 77.83% 77.84% +<.01%
==========================================
Files 433 433
Lines 22398 22415 +17
==========================================
+ Hits 17433 17447 +14
- Misses 4965 4968 +3
Continue to review full report at Codecov.
|
| background: true | ||
| # We sleep a bit to wait for the background process to start and script to | ||
| # finish | ||
| - case $CIRCLE_NODE_INDEX in [1,2]) sleep 10 ;; esac |
There was a problem hiding this comment.
Can't we just make it a part of the previous line?
There was a problem hiding this comment.
Sadly no because it runs in a background (background: true) and that's why we need it to be a separate step.
There was a problem hiding this comment.
It can but I prefer separate lines to make it more readable (flow it's more sequential).
I already hate all the case lines which make it hard to read and follow so I plan to look into circleci-matrix or similar solution in the future so we can at the very least keep circle.yml more sane.
indexes. Sadly there is no way around it but to drop the offending index and re-create it See http://docs.mongoengine.org/upgrade.html#inheritance
|
It looks like there is sadly no way around it but to drop and re-create the offending index - ebcf8cb. Luckily this only affects the "uid" index. See https://docs.mongoengine.org/upgrade.html#inheritance for details on the types stuff. As you can see in the diff, I only scoped it to the uid index to be on the safe side although something like this would also be useful in other contexts in the future. |
m4dcoder
left a comment
m4dcoder
left a comment
There was a problem hiding this comment.
LGTM once the packages and CIs pass. I was wondering about this index error I was seeing on my dev system. I had to clean the mongoDB but obviously not good for upgrades. Thanks for the fix.
|
It looks like the e2e issue was the same issue as one here - #3211 (comment) Syncing with latest master should solve it. |
This pull request updates the code to ignore the errors about which we can't do anything right now.
Correct solution we should eventually do is to re-creating offending indexes. The problem with re-creation is that the operation is blocking and depending on the dataset size, it could take a very long time.