Skip to content

[SPARK-55519][SQL][TESTS] Fix V2FunctionBenchmark #54540

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
pan3793 wants to merge 3 commits into from
+60 −58
Closed

[SPARK-55519][SQL][TESTS] Fix V2FunctionBenchmark #54540

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9857b9f
[SPARK-55744][SQL][TESTS] Fix V2FunctionBenchmark
pan3793 Feb 27, 2026
6d862e5
Benchmark results for org.apache.spark.sql.connector.functions.V2Func…
pan3793 Feb 27, 2026
f175798
Benchmark results for org.apache.spark.sql.connector.functions.V2Func…
pan3793 Feb 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 28 additions & 28 deletions sql/core/benchmarks/V2FunctionBenchmark-jdk21-results.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,44 +1,44 @@
OpenJDK 64-Bit Server VM 21.0.8+9-LTS on Linux 6.11.0-1018-azure
OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
scalar function (long + long) -> long, result_nullable = true codegen = true: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------------------------
native_long_add 9308 9710 686 53.7 18.6 1.0X
java_long_add_default 22865 23994 1622 21.9 45.7 0.4X
java_long_add_magic 11505 11983 804 43.5 23.0 0.8X
java_long_add_static_magic 11395 11452 49 43.9 22.8 0.8X
scala_long_add_default 23153 23807 968 21.6 46.3 0.4X
scala_long_add_magic 11562 11585 38 43.2 23.1 0.8X
native_long_add 9357 9828 787 53.4 18.7 1.0X
java_long_add_default 23136 23301 258 21.6 46.3 0.4X
java_long_add_magic 11555 11895 550 43.3 23.1 0.8X
java_long_add_static_magic 11457 11505 52 43.6 22.9 0.8X
scala_long_add_default 23402 23436 48 21.4 46.8 0.4X
scala_long_add_magic 11533 11657 149 43.4 23.1 0.8X

OpenJDK 64-Bit Server VM 21.0.8+9-LTS on Linux 6.11.0-1018-azure
OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
scalar function (long + long) -> long, result_nullable = false codegen = true: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------------
native_long_add 10290 10340 49 48.6 20.6 1.0X
java_long_add_default 22328 22368 42 22.4 44.7 0.5X
java_long_add_magic 11485 11572 118 43.5 23.0 0.9X
java_long_add_static_magic 9962 9980 19 50.2 19.9 1.0X
scala_long_add_default 22314 22377 104 22.4 44.6 0.5X
scala_long_add_magic 11503 12078 705 43.5 23.0 0.9X
native_long_add 10313 10344 34 48.5 20.6 1.0X
java_long_add_default 22388 22880 536 22.3 44.8 0.5X
java_long_add_magic 11491 11582 156 43.5 23.0 0.9X
java_long_add_static_magic 9980 13846 6696 50.1 20.0 1.0X
scala_long_add_default 22311 22344 30 22.4 44.6 0.5X
scala_long_add_magic 11541 11627 77 43.3 23.1 0.9X

OpenJDK 64-Bit Server VM 21.0.8+9-LTS on Linux 6.11.0-1018-azure
OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
scalar function (long + long) -> long, result_nullable = true codegen = false: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------------
native_long_add 22531 22615 108 22.2 45.1 1.0X
java_long_add_default 26677 26707 32 18.7 53.4 0.8X
java_long_add_magic 32551 32589 55 15.4 65.1 0.7X
java_long_add_static_magic 30687 30774 95 16.3 61.4 0.7X
scala_long_add_default 26292 26366 86 19.0 52.6 0.9X
scala_long_add_magic 32777 32981 245 15.3 65.6 0.7X
native_long_add 22692 22767 99 22.0 45.4 1.0X
java_long_add_default 26553 26619 68 18.8 53.1 0.9X
java_long_add_magic 32637 32804 285 15.3 65.3 0.7X
java_long_add_static_magic 30751 30819 60 16.3 61.5 0.7X
scala_long_add_default 26453 26489 42 18.9 52.9 0.9X
scala_long_add_magic 32736 32891 263 15.3 65.5 0.7X

OpenJDK 64-Bit Server VM 21.0.8+9-LTS on Linux 6.11.0-1018-azure
OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
scalar function (long + long) -> long, result_nullable = false codegen = false: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------------------------------
native_long_add 22593 22807 331 22.1 45.2 1.0X
java_long_add_default 26325 26370 46 19.0 52.7 0.9X
java_long_add_magic 32796 32827 42 15.2 65.6 0.7X
java_long_add_static_magic 30699 30874 163 16.3 61.4 0.7X
scala_long_add_default 26341 26371 45 19.0 52.7 0.9X
scala_long_add_magic 32794 32982 258 15.2 65.6 0.7X
native_long_add 22670 22722 79 22.1 45.3 1.0X
java_long_add_default 26424 26464 46 18.9 52.8 0.9X
java_long_add_magic 32737 33464 1195 15.3 65.5 0.7X
java_long_add_static_magic 30840 31079 380 16.2 61.7 0.7X
scala_long_add_default 26384 26482 123 19.0 52.8 0.9X
scala_long_add_magic 32628 32932 455 15.3 65.3 0.7X

56 changes: 28 additions & 28 deletions sql/core/benchmarks/V2FunctionBenchmark-results.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,44 +1,44 @@
OpenJDK 64-Bit Server VM 17.0.16+8-LTS on Linux 6.11.0-1018-azure
OpenJDK 64-Bit Server VM 17.0.18+8-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
scalar function (long + long) -> long, result_nullable = true codegen = true: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------------------------
native_long_add 9211 9245 49 54.3 18.4 1.0X
java_long_add_default 21996 22193 184 22.7 44.0 0.4X
java_long_add_magic 10647 10743 91 47.0 21.3 0.9X
java_long_add_static_magic 10559 10596 37 47.4 21.1 0.9X
scala_long_add_default 23193 23214 20 21.6 46.4 0.4X
scala_long_add_magic 10666 10686 30 46.9 21.3 0.9X
native_long_add 9112 9184 81 54.9 18.2 1.0X
java_long_add_default 22468 22546 89 22.3 44.9 0.4X
java_long_add_magic 10635 10671 31 47.0 21.3 0.9X
java_long_add_static_magic 10507 10579 63 47.6 21.0 0.9X
scala_long_add_default 23158 23211 83 21.6 46.3 0.4X
scala_long_add_magic 10646 10654 9 47.0 21.3 0.9X

OpenJDK 64-Bit Server VM 17.0.16+8-LTS on Linux 6.11.0-1018-azure
OpenJDK 64-Bit Server VM 17.0.18+8-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
scalar function (long + long) -> long, result_nullable = false codegen = true: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------------
native_long_add 9988 10158 200 50.1 20.0 1.0X
java_long_add_default 22418 22479 70 22.3 44.8 0.4X
java_long_add_magic 10631 10657 23 47.0 21.3 0.9X
java_long_add_static_magic 9864 9928 67 50.7 19.7 1.0X
scala_long_add_default 22269 22354 76 22.5 44.5 0.4X
scala_long_add_magic 10614 10670 52 47.1 21.2 0.9X
native_long_add 9953 9984 28 50.2 19.9 1.0X
java_long_add_default 22049 22151 105 22.7 44.1 0.5X
java_long_add_magic 10611 10636 27 47.1 21.2 0.9X
java_long_add_static_magic 9880 10707 1205 50.6 19.8 1.0X
scala_long_add_default 22129 22175 78 22.6 44.3 0.4X
scala_long_add_magic 10622 10712 95 47.1 21.2 0.9X

OpenJDK 64-Bit Server VM 17.0.16+8-LTS on Linux 6.11.0-1018-azure
OpenJDK 64-Bit Server VM 17.0.18+8-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
scalar function (long + long) -> long, result_nullable = true codegen = false: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------------
native_long_add 22860 23095 317 21.9 45.7 1.0X
java_long_add_default 28130 28318 282 17.8 56.3 0.8X
java_long_add_magic 30689 32298 2467 16.3 61.4 0.7X
java_long_add_static_magic 30929 30969 35 16.2 61.9 0.7X
scala_long_add_default 26355 27156 1356 19.0 52.7 0.9X
scala_long_add_magic 30825 30861 33 16.2 61.7 0.7X
native_long_add 23353 23427 116 21.4 46.7 1.0X
java_long_add_default 28263 28280 15 17.7 56.5 0.8X
java_long_add_magic 31114 31224 166 16.1 62.2 0.8X
java_long_add_static_magic 31335 31963 1035 16.0 62.7 0.7X
scala_long_add_default 26408 27086 882 18.9 52.8 0.9X
scala_long_add_magic 31166 31292 136 16.0 62.3 0.7X

OpenJDK 64-Bit Server VM 17.0.16+8-LTS on Linux 6.11.0-1018-azure
OpenJDK 64-Bit Server VM 17.0.18+8-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
scalar function (long + long) -> long, result_nullable = false codegen = false: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------------------------------
native_long_add 21895 21957 74 22.8 43.8 1.0X
java_long_add_default 25861 26173 537 19.3 51.7 0.8X
java_long_add_magic 30646 30681 31 16.3 61.3 0.7X
java_long_add_static_magic 30226 30256 32 16.5 60.5 0.7X
scala_long_add_default 26041 26108 70 19.2 52.1 0.8X
scala_long_add_magic 30668 30765 130 16.3 61.3 0.7X
native_long_add 22149 22214 110 22.6 44.3 1.0X
java_long_add_default 26026 26087 74 19.2 52.1 0.9X
java_long_add_magic 30800 30875 67 16.2 61.6 0.7X
java_long_add_static_magic 30936 31876 872 16.2 61.9 0.7X
scala_long_add_default 26061 26084 35 19.2 52.1 0.8X
scala_long_add_magic 30832 31138 451 16.2 61.7 0.7X

6 changes: 4 additions & 2 deletions sql/core/src/test/scala/org/apache/spark/sql/connector/functions/V2FunctionBenchmark.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,8 +108,10 @@ object V2FunctionBenchmark extends SqlBasedBenchmark {
case class NativeAdd(
left: Expression,
right: Expression,
override val nullable: Boolean) extends BinaryArithmetic {
override val evalContext: NumericEvalContext = NumericEvalContext(EvalMode.LEGACY)
override val nullable: Boolean,
override val evalContext: NumericEvalContext = NumericEvalContext(EvalMode.LEGACY))
extends BinaryArithmetic {

override def inputType: AbstractDataType = NumericType
override def symbol: String = "+"

Expand Down