feat(object_store): Override DNS Resolution to Randomize IP Selection#7123
feat(object_store): Override DNS Resolution to Randomize IP Selection#7123tustvold merged 2 commits intoapache:mainfrom
Conversation
There was a problem hiding this comment.
Note that these dependencies aren't technically new, they were already indirect dependencies. They are mostly needed because the relevant reqwest types aren't public. I'll try to fix that in seanmonstar/reqwest#2549 , but I don't think we need to block this PR here on that upstream change.
Closes #7117.
crepererum
force-pushed
the
crepererum/issue7117
branch
from
6b159a3 to
60732d4
Compare
|
|
||
| use std::str::FromStr; | ||
|
|
||
| use hyper_util::client::legacy::connect::dns::{ |
There was a problem hiding this comment.
Is the legacy part a concern? Do we need to use this or could we just inline whatever it is doing if it isn't complex?
There was a problem hiding this comment.
It's used by reqwest as well. I think it mostly means that his is the hyper 0.x legacy behavior that you should eventually migrate away from -- which we could do eventually (see #7123 (comment) ).
However you're also right that we could just perform the std call ourselves, will try that.
There was a problem hiding this comment.
done, and the code is IMHO actually rather simple
tustvold
left a comment
tustvold
left a comment
There was a problem hiding this comment.
I've marked this as a breaking change, just to highlight it in the changelog. The next object_store release is breaking anyway
Makes apache#7123 compatible w/ an older version of `rand`.
|
FWIW this is actually a breaking change, as reqwest has a feature flag that can enable hickory DNS, which this will override |
tustvold
changed the title
* feat(object_store): random IP address selection Closes #7117. * refactor: directly call stdlib w/o hyper-util
2 participants
Which issue does this PR close?
Closes apache/arrow-rs-object-store#19.
Rationale for this change
See apache/arrow-rs-object-store#19.
What changes are included in this PR?
Opt-out, randomizing resolver config.
Are there any user-facing changes?
Load is now spread across multiple servers.