Panic when inserting array literal with NULL element into a table where the matching column has a non-nullable element #12598
Description
Describe the bug
I have a custom data sink whose schema contains an array whose element type is non-nullable. If I attempt to insert an array literal with a null element into this data sink via SQL, the process exits with a panic. This behavior can be reproduced using MemTable.
To Reproduce
The following example code can be added to datafusion-examples/src and run via cargo run --example array-non-nullable-element:
use std::sync::Arc;
use arrow::datatypes::{DataType, Field, Schema};
use arrow::record_batch::RecordBatch;
use datafusion::common::Result;
use datafusion::prelude::*;
#[tokio::main]
async fn main() -> Result<()> {
let schema = Arc::new(Schema::new(vec![
Field::new("id", DataType::Int64, false),
Field::new("arr", DataType::new_list(DataType::Int64, false), true),
]));
let ctx = SessionContext::new();
ctx.register_batch("t", RecordBatch::new_empty(Arc::clone(&schema)))?;
let _ = ctx.table("t").await?;
let _ = ctx
.sql("INSERT INTO t VALUES (1, [1, 2, 3]);")
.await?
.collect()
.await?;
let _ = ctx
.sql("INSERT INTO t VALUES (2, [1, NULL, 3]);")
.await?
.collect()
.await?;
Ok(())
}
Running with RUST_BACKTRACE=1 indicates that there is an unwrap() occurring in ListArray::new.
3: core::result::Result<T,E>::unwrap
at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/result.rs:1102:23
4: arrow_array::array::list_array::GenericListArray<OffsetSize>::new
at $HOME/.cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-array-53.0.0/src/array/list_array.rs:228:9
5: arrow_cast::cast::list::cast_list_values
at $HOME/.cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-cast-53.0.0/src/cast/list.rs:144:17
6: arrow_cast::cast::cast_with_options
at $HOME/.cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-cast-53.0.0/src/cast/mod.rs:750:32
7: datafusion_expr_common::columnar_value::ColumnarValue::cast_to
at $DATAFUSION/datafusion/expr-common/src/columnar_value.rs:195:17
Expected behavior
I would expect this query to bubble a DataFusionError up to the call site instead of panicking the process.
Additional context
No response