Skip to content

Improve error messages for parallel parquet writer "Unable to send array to writer!" #11397

@alamb

Description

@alamb

@devinjdangelo noted on #11345 that if an error happens deep in the parallel parquet writer it often is not returned to the user and instead we get a generic message "Unable to send array to writer!"

          I think we need to update several map_err statements to propagate inner error messages rather than ignore them. E.g. 

col_array_channels[next_channel]
.send(c)
.await
.map_err(|_| {
DataFusionError::Internal("Unable to send array to writer!".into())

change to something like

            col_array_channels[next_channel]
                .send(c)
                .await
                .map_err(|e| internal_datafusion_err!("Unable to send array to writer due to error {e}"))

Originally posted by @devinjdangelo in #11345 (comment)

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions