ARROW-6541: [Format][C++] Update Columnar.rst for two-part EOS, update C++ implementation #5361
Conversation
|
Trying to reproduce the failure that's occurring in Ursabot Python 2.7 build |
|
I can repro, not sure why it occurs in Python 2.7 but not 3.6/7 |
|
@pitrou @emkornfield I found that Python 2.7 wasn't returning aligned memory from |
| namespace internal { | ||
|
|
||
| // This 0xFFFFFFFF value is the first 4 bytes of a valid IPC message | ||
| constexpr int32_t kIpcContinuationToken = -1; |
There was a problem hiding this comment.
for some reason i thought there were linkage issues in C++11 using constexpr in a header.
There was a problem hiding this comment.
constexpr implies inline/static linkage. It's not an error (though I sort of expected it to be) in clang to declare
extern constexpr int32_t kIpcContinuationToken = -1;... if you want to declare something constexpr within this translation unit but also export it as a symbol for other translation units, but the expectation is that you'll just include the header into whatever TU needs that constant.
emkornfield
left a comment
emkornfield
left a comment
There was a problem hiding this comment.
Looks OK to me, feel free to merge when build passes.
Interesting. Python 3 doesn't make any such guarantee either, but probably it turns out ok in the current |
| return Write(&kEos, sizeof(kEos)); | ||
| constexpr int32_t kZeroLength = 0; | ||
| if (!options_.write_legacy_ipc_format) { | ||
| RETURN_NOT_OK(Write(&internal::kIpcContinuationToken, sizeof(int32_t))); |
There was a problem hiding this comment.
nit:
| RETURN_NOT_OK(Write(&internal::kIpcContinuationToken, sizeof(int32_t))); | |
| RETURN_NOT_OK(Write(&internal::kIpcContinuationToken, sizeof(internal::kIpcContinuationToken))); |
There was a problem hiding this comment.
I'm going to let this one slide -- I kind of prefer int32_t for the documentation aspect
…e C++ implementation Closes #5361 from wesm/two-part-eos and squashes the following commits: cd632ff <Wes McKinney> Copy metadata unconditionally since input buffers may not start on aligned offsets 70d1b3a <Wes McKinney> Remove unneeded header 4071528 <Wes McKinney> Use 2-part EOS in C++, update in format docs. Write old EOS if option is set Authored-by: Wes McKinney <wesm+git@apache.org> Signed-off-by: Wes McKinney <wesm+git@apache.org>
…e C++ implementation Closes #5361 from wesm/two-part-eos and squashes the following commits: cd632ff <Wes McKinney> Copy metadata unconditionally since input buffers may not start on aligned offsets 70d1b3a <Wes McKinney> Remove unneeded header 4071528 <Wes McKinney> Use 2-part EOS in C++, update in format docs. Write old EOS if option is set Authored-by: Wes McKinney <wesm+git@apache.org> Signed-off-by: Wes McKinney <wesm+git@apache.org>
…e C++ implementation Closes apache#5361 from wesm/two-part-eos and squashes the following commits: cd632ff <Wes McKinney> Copy metadata unconditionally since input buffers may not start on aligned offsets 70d1b3a <Wes McKinney> Remove unneeded header 4071528 <Wes McKinney> Use 2-part EOS in C++, update in format docs. Write old EOS if option is set Authored-by: Wes McKinney <wesm+git@apache.org> Signed-off-by: Wes McKinney <wesm+git@apache.org>
6 participants
No description provided.