Azure deep storage does not work with datasource name containing non-ASCII chars#9525
Merged
clintropolis merged 4 commits intoapache:masterfrom Mar 19, 2020
Merged
Azure deep storage does not work with datasource name containing non-ASCII chars#9525clintropolis merged 4 commits intoapache:masterfrom
clintropolis merged 4 commits intoapache:masterfrom
Conversation
…ASCII chars Fixed a bug where recording the segment file location fails when using Azure Deep Storage, if the datasource has any special characters
Contributor
|
If this PR #9501 goes in, you can also run the integration test for Azure which will verify your fix. |
| return outSegment; | ||
| } | ||
|
|
||
| private Map<String, Object> makeLoadSpec(String container, String prefix) |
Member
There was a problem hiding this comment.
Could you make the other makeLoadSpec delegate to this method instead of duplicating it?
@Override
public Map<String, Object> makeLoadSpec(URI uri)
{
return makeLoadSpec(segmentConfig.getContainer(), uri.toString());
}
Also, is it necessary to pass container in since both invocations are using the same value (if you make the suggested modification)?
* address review comments
clintropolis
approved these changes
Mar 19, 2020
| "blobPath", | ||
| uri.toString() | ||
| ); | ||
| return makeLoadSpec(uri.toString()); |
Member
There was a problem hiding this comment.
It looks like hadoop indexing would still call this method so it might have issues with funny characters, but I don't think it is worth refactoring to fix the issue at this point.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #9515
Description
Fixed a bug where recording the segment file location fails when
using Azure Deep Storage, if the datasource has any special
characters. Verified fix with manual test.
This PR has: