Skip to content

Conversation

@AlenkaF
Copy link
Member

@AlenkaF AlenkaF commented Jun 1, 2022

This PR changes CMakeLists .txt files so that xxhash, portable-snippets are also installed. Plus it changes the name of int_util_internal.h to int_util_overflow.h for the same reason.

This step is needed for Python code to be moved to PyArrow in https://issues.apache.org/jira/browse/ARROW-16340.

@github-actions
Copy link

github-actions bot commented Jun 1, 2022

@kou
Copy link
Member

kou commented Jun 1, 2022

Renaming int_util_internal.h to int_util_overflow.h is OK if it only includes overflow related codes but the current int_util_internal.h has UpcastInt() and CheckSliceParams() that aren't related to overflow.

What functions do you want? Can we move only them to int_util.h from int_util_internal.h?

@AlenkaF
Copy link
Member Author

AlenkaF commented Jun 2, 2022

A file (to be moved out of C++) python_to_arrow.cc uses next functions:

  • MultiplyWithOverflow
  • AddWithOverflow
  • SubtractWithOverflow

I do not mind moving only these parts to int_util.h. But am not sure if leaving other overflow functions where they are makes sense?

@kou
Copy link
Member

kou commented Jun 2, 2022

How about moving only them and DivideWithOverflow() to int_util.h?

@AlenkaF
Copy link
Member Author

AlenkaF commented Jun 2, 2022

How about moving only them and DivideWithOverflow() to int_util.h?

I do not mind. @pitrou can you also share you thoughts on this?

@pitrou
Copy link
Member

pitrou commented Jun 2, 2022

Judging by the comment:

// "safe-math.h" includes <intsafe.h> from the Windows headers.

We don't want to move these functions to int_util.h. OTOH, the other functions (UpcastInt and CheckSliceParams) can be moved to int_util.h.

@AlenkaF
Copy link
Member Author

AlenkaF commented Jun 3, 2022

I do not think the fails are connected to the changes in this PR.

@pitrou @kou I added UpcastInt and CheckSliceParams to int_util.h and kept int_util_internal.h renamed to int_util_overflow.h. This should be ready for another look. Thanks!

#include <type_traits>

#include "arrow/status.h"
#include "arrow/util/int_util_overflow.h"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know why we should not include <intsafe.h> in our public headers but we should not include it here.
Only needed python/**/*.cc in #13311 should include it instead.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIR, intsafe.h includes Windows.h.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure I understand the comments, could you please clarify?

To note why arrow/util/int_util_overflow.h is included here: CheckSliceParams which is now moved to int_util.h needs AddWithOverflow from arrow/util/int_util_overflow.h.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, then CheckSliceParams can go to int_util_overflow.h probably?

What I meant above is that we don't want Windows.h to be included from our public headers, even transitively.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But currently this PR makes int_util_overflow.h (formerly int_util_internal.h) also a public header? (and so Windows.h gets included that way as well)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, we have to distribute it somehow for PyArrow (or duplicate the functionality, as was suggested elsewhere, but that doesn't sound ideal). However, as long as it's not included from other public headers, then it's unlikely to be a problem.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about creating cpp/src/arrow/util/slice_util_internal.h (or something) and move CheckSliceParams to the header instead of moving CheckSliceParams to int_util_overflow.h? I feel that moving CheckSliceParams to int_util_overflow.h is strange. Because CheckSliceParams is a validation function for Slice. It's not (almost) related to integer overflow.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds fine to me.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, will do 👍

@AlenkaF
Copy link
Member Author

AlenkaF commented Jun 7, 2022

I think the CI error isn't related.
Could you have another look at this PR pls? @kou @pitrou

Copy link
Member

@kou kou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@kou kou merged commit c32a1d1 into apache:master Jun 8, 2022
@AlenkaF AlenkaF deleted the ARROW-16609 branch June 8, 2022 06:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants