[Proposal] Skip writing value lengths in the indexes

[pjain1]

Druid uses format described in GenericIndexed class to serialize indexes. While writing index values, Druid writes the length of values in bytes before each value.

However, this may be skipped as end offsets of each serialized value is already written before the values are stored. This can lead to savings of (4 bytes * number of values in the index) for each index in the final segment. Some examples of GenericIndexed index are for writing Dictionary values, corresponding bitmaps etc.

Corresponding PR for discussion - #3069

[navis]

I've also did this once but found the difference was not big enough (~5%) to justify format inconsistency. I'm good on this change because we don't have any product cluster yet but it seemed hard to accepted for others.

[stale]

This issue has been marked as stale due to 280 days of inactivity. It will be closed in 2 weeks if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the dev@druid.apache.org list. Thank you for your contributions.

[stale]

This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.