Skip to content

Conversation

@pvary
Copy link
Contributor

@pvary pvary commented Sep 7, 2020

Metrics implements Serializable, but contains ByteBuffers which are not serializable.
This PR makes sure that the Metrics object remain serializable

@pvary pvary changed the title Core: Fix Metrics serialization API: Fix Metrics serialization Sep 7, 2020
@pvary
Copy link
Contributor Author

pvary commented Sep 7, 2020

@rdblue: Could you please review this?
Thanks,
Peter

ByteBuffer bb = entry.getValue();
byte[] bytes = new byte[bb.remaining()];
bb.get(bytes);
bb.position(bb.position() - bytes.length); // Restores the buffer position
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a ByteBuffers util located in the core module at org.apache.iceberg.util.ByteBuffers that handles the appropriate copying and restoration of ByteBuffers.

Please consider using that for consistency and for easier refactoring of ByteBuffer issues in the future. Here's a link to that class: https://github.com/apache/iceberg/blob/master/core/src/main/java/org/apache/iceberg/util/ByteBuffers.java

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @kbendick. You can call ByteBuffers.toByteArray instead of adding the logic to read here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kbendick! Have not noticed that class.
Had to refactor that and move it to the API package to reuse it. Please check if you agree with the move, or you think there is a better place for it.

Thanks,
Peter

@rdblue rdblue merged commit dece71c into apache:master Sep 15, 2020
@rdblue
Copy link
Contributor

rdblue commented Sep 15, 2020

Thanks for fixing this, @pvary!

@pvary pvary deleted the metrics branch September 15, 2020 06:52
@rdblue rdblue added this to the Java 0.10.0 Release milestone Nov 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants