[C++][FlightRPC] Buffer handling change in 12.0.1 causing a lot of warnings being printed

[RichardHaythorn]

Describe the bug, including details regarding any error messages, version, and platform.

Version: 12.0.1

"An input buffer was poorly aligned. This could lead to crashes or "
"poor performance on some hardware. Please ensure that all Acero "
"sources generate aligned buffers, or change the unaligned buffer "
"handling configuration to silence this warning."

When doing a do_get with pyarrow flight clients now, a warning is triggered for every column in every recordbatch with misaligned buffers, causing a lot of warnings to be printed to console when a table is created from the recordbatches. This is due to flight creating misaligned buffers #32276. I believe this was introduced by #35565 in how it handles unaligned buffers.

As there is no way to change this behaviour in pyarrow, the only I've found to sidestep it is to use copy.deepcopy on the recordbatches before making the pyarrow table.

Apologies for not being able to provide a MWE of the bug

Component(s)

C++, Python

[benkimpel]

@RichardHaythorn I'm also seeing this issue in our org, but so far only on windows. Is that, by any chance, the OS you're running?

[RichardHaythorn]

@benkimpel This problem is cropping up on Ubuntu 22 for me

[benkimpel]

Gotcha, thanks. That's good to know that it's not just windows.

[kou]

Does ACERO_ALIGNMENT_HANDLING=ignore environment variable work for you?

[RichardHaythorn]

Yes that seems to have done it! I didn't realise the new switch case in the commit was using environment variables

[kou]

Can we close this?

[RichardHaythorn]

I think it would be useful to update the warning message, " Please ensure that all Acero sources generate aligned buffers, or change the unaligned buffer handling configuration to silence this warning.", as this is unclear for Python users what they're suppose to change, as acero is not exposed on the pyarrow side. Also, this being an environment variable is not listed in the Python or C++ docs currently

[kou]

OK. Do you want to open a pull request to improve them?

[westonpace]

I agree that updating the warning message would be a good idea. It's located here if you want to propose some new wording (I'm not really sure how to describe this without using the word "Acero"). Also, just to be clear, this usage of unaligned buffers was not recently introduced. It is the way that Flight (well, technically, grpc I think) has always worked. That only thing that is new is that we've started checking for it and issuing a warning when we detect it. In most cases (on modern CPUs) this warning is fairly harmless.

[westonpace]

#36622 adds documentation to the environment variables page. We can leave this issue open in case someone wants to improve the error message.

[RichardHaythorn]

Yes it's not the first time I've come across this misaligned buffer issue, its caused a few downstream issues for me before, pola-rs/polars#6315, i do understand its not an arrow issue. The updated docs LGTM, I'll try and get round to looking at the arrow Python/Flight docs to see where it might be good to put a note about this.