Skip to content

Allow configuration of compression codec.#26

Merged
leventov merged 2 commits intometamx:masterfrom
gianm:compression-configure
Dec 7, 2016
Merged

Allow configuration of compression codec.#26
leventov merged 2 commits intometamx:masterfrom
gianm:compression-configure

Conversation

@gianm
Copy link
Copy Markdown
Contributor

@gianm gianm commented Dec 5, 2016

In Druid we've seen that for large resultsets, gzip compression / decompression time is substantial, and given that Druid is typically deployed on a LAN we could benefit from using a different codec or from not using compression at all. This patch adds configurability between gzip/deflate/identity (the ones currently supported by the client) rather than hard-coding gzip.

Allows changing the default through withCompressionCodec on the config builder, as well as overriding per-request through the Accept-Encoding header.

}
};

public abstract String getEncodingString();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can remove this method and use compressionCodec.name().toLowerCase() instead of getEncodingString(). Or even call enum constants lowercase and use just name().

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did this because I'm anticipating that in the future we might want to add LZ4, which would manifest as an enum value LZ4 but encoding string "x-lz4"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gianm why not add it now?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the http-client doesn't currently support LZ4 and adding it may be non-trivial work.

Copy link
Copy Markdown
Contributor

@leventov leventov Dec 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gianm ok, could you please add this explanation to getEncodingString() as a Javadoc comment?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@leventov added.

@leventov leventov merged commit 45f9166 into metamx:master Dec 7, 2016
@leventov
Copy link
Copy Markdown
Contributor

leventov commented Dec 7, 2016

@gianm this change is included into 1.0.6 release.

@leventov
Copy link
Copy Markdown
Contributor

leventov commented Dec 7, 2016

@gianm could you please review some of my recent PRs into Druid?

@gianm
Copy link
Copy Markdown
Contributor Author

gianm commented Dec 7, 2016

sure, I have been doing more development than reviewing this week, but was intending to get back to reviewing soon.

@gianm
Copy link
Copy Markdown
Contributor Author

gianm commented Dec 7, 2016

thank you for your review.

@gianm gianm deleted the compression-configure branch December 7, 2016 18:15
@esevastyanov
Copy link
Copy Markdown

@gianm could you please describe a bit how this code change controls the compression? As I understand the ACCEPT_ENCODING header means which content encoding the client is able to decode. So it can disable the response compression, but not the request compression.

@esevastyanov
Copy link
Copy Markdown

And where the compression logic is? Is it in Druid?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants