feat: Add `datafusion-spark` crate#14392

Closed

andygrove wants to merge 69 commits intoapache:mainfrom

andygrove:add-spark-expr-crate

Commits on Jul 10, 2024

feat: Create new `datafusion-comet-spark-expr` crate containing Spark-compatible DataFusion expressions (#638)
andygrove
authored

Commits on Jul 11, 2024

feat: Move `IfExpr` to `spark-expr` crate (#653)
andygrove
authored

Commits on Jul 12, 2024

Commits on Jul 15, 2024

chore: Move temporal kernels and expressions to spark-expr crate (#660)
andygrove
authored
fix: Optimize some functions to rewrite dictionary-encoded strings (#627)
vaibhawvipul
authored

Commits on Jul 16, 2024

Change suffix on some expressions from Exec to Expr (#673)
andygrove
authored

Commits on Jul 20, 2024

chore: Disable abs and signum because they return incorrect results (#695)
andygrove
authored

Commits on Jul 21, 2024

chore: Make rust clippy happy (#701)
Xuanwo
authored

Commits on Jul 24, 2024

perf: Optimize IfExpr by delegating to CaseExpr (#681)
andygrove
authored

Commits on Jul 27, 2024

chore: make Cast's logic reusable for other projects (#716)
Blizzara
authored

Commits on Jul 28, 2024

chore: move scalar_funcs into spark-expr (#712)
Blizzara
authored

Commits on Jul 31, 2024

chore: Add criterion benchmark for decimal_div (#743)
andygrove
authored

Commits on Aug 1, 2024

Commits on Aug 2, 2024

feat: Implement basic version of RLIKE (#734)
andygrove
authored

Commits on Aug 3, 2024

feat: Add GetStructField expression (#731)
Kimahriman
authored

Commits on Aug 8, 2024

fix: Optimize read_side_padding (#772)
kazuyukitanimura
authored

Commits on Aug 9, 2024

feat: Optimze CreateNamedStruct preserve dictionaries (#789)
eejbyfeldt
authored

Commits on Aug 21, 2024

feat: Enable `clippy::clone_on_ref_ptr` on `proto` and `spark_exprs` crates (#859)
comphead
authored

Commits on Aug 28, 2024

Commits on Aug 30, 2024

feat: Support sort merge join with a join condition (#553)
viirya
authored

Commits on Sep 5, 2024

Commits on Sep 16, 2024

feat: date_add and date_sub functions (#910)
mbutrovich
authored

Commits on Oct 7, 2024

feat: Support `GetArrayStructFields` expression (#993)
Kimahriman
authored

Commits on Oct 14, 2024

chore: Bump arrow-rs to 53.1.0 and datafusion (#1001)
kazuyukitanimura
authored

Commits on Oct 30, 2024

chore: Use twox-hash 2.0 xxhash64 oneshot api instead of custom implementation (#1041)
NoeB
authored

Commits on Nov 5, 2024

chore: Upgrade to DataFusion 43.0.0-rc1 (#1057)
andygrove
authored

Commits on Nov 8, 2024

feat: Implement CAST from struct to string (#1066)
andygrove
authored

Commits on Nov 11, 2024

feat: Implement CAST between struct types (#1074)
andygrove
authored

Commits on Nov 19, 2024

fix: Unsigned type related bugs (#1095)
kazuyukitanimura
authored

Commits on Nov 22, 2024

feat: support array_insert (#1073)

SemyonSinchenko
and
andygrove
authored

Commits on Dec 6, 2024

chore: Refactor cast to use SparkCastOptions param (#1146)
andygrove
authored

Commits on Dec 9, 2024

chore: Move more expressions from core crate to spark-expr crate (#1152)
andygrove
authored

Commits on Dec 12, 2024

Commits on Dec 17, 2024

feat: Add Spark-compatible implementation of SchemaAdapterFactory (#1169)
andygrove
authored

Commits on Dec 19, 2024

fix: stddev_pop should not directly return 0.0 when count is 1.0 (#1184)
viirya
authored

Commits on Dec 28, 2024

chore: Upgrade to DataFusion 44.0.0-rc2 (#1154)

andygrove
and
alamb
authored

Commits on Jan 6, 2025

Commits on Jan 7, 2025

Commits on Jan 8, 2025

Commits on Jan 9, 2025

chore: extract hash_funcs expressions to folders based on spark grouping (#1221)

rluvaton
and
andygrove
authored

Commits on Jan 18, 2025

chore: extract conversion_funcs, conditional_funcs, bitwise_funcs and array_funcs expressions to folders based on spark grouping (#1223)
rluvaton
authored

Commits on Jan 19, 2025

fix: partially fix consistency issue of hash functions with decimal input (#1295)
wForget
authored

Commits on Jan 20, 2025

chore: extract math_funcs expressions to folders based on spark grouping (#1219)
rluvaton
authored

Commits on Jan 21, 2025

chore: merge comet-parquet-exec branch into main (#1318)
andygrove
authored

Commits on Jan 22, 2025

fix merge conflicts (#1320)
andygrove
authored

Commits on Jan 23, 2025

chore: Fix merge conflicts from merging comet-parquet-exec into main (#1323)
mbutrovich
authored

Commits on Jan 30, 2025

chore: Prepare for DataFusion 45 (bump to DataFusion rev 5592834 + Arrow 54.0.0) (#1332)
andygrove
authored