-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
Describe the bug
A clear and concise description of what the bug is.
Some aggregation plans will fail to deserialize. This is caused by a rewrite done by the TypeCoercion optimizer.
To Reproduce
Steps to reproduce the behavior:
#[tokio::test]
async fn roundtrip_logical_plan_aggregation() -> Result<(), DataFusionError> {
let ctx = SessionContext::new();
let schema = Schema::new(vec![
Field::new("a", DataType::Int64, true),
Field::new("b", DataType::Decimal128(15,2), true)
]);
ctx.register_csv("t1", "testdata/test.csv", CsvReadOptions::default().schema(&schema)).await?;
let query = "SELECT a, SUM(b + 1) FROM t1 GROUP BY a";
let plan = ctx.sql(query).await?.to_logical_plan()?;
let bytes =
logical_plan_to_bytes(&plan)?;
let logical_round_trip =
logical_plan_from_bytes(&bytes, &ctx)?;
assert_eq!(
format!("{:?}", plan),
format!("{:?}", logical_round_trip)
);
Ok(())
This fails with
Error: SchemaError(FieldNotFound { qualifier: None, name: "SUM(t1.b + Int64(1))", valid_fields: Some(["t1.a", "SUM(t1.b + Decimal128(Some(100),23,2))", "t1.a", "t1.b"]) })
The problem is there is a projection after this which projects the original expression name. Typically this doesn't matter since the schema is unchanged and everything still aligns. But when we deserialize we have to build the plan again and it will fail to validate.
Expected behavior
A clear and concise description of what you expected to happen.
Additional context
Add any other context about the problem here.
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working