Core: Upgrade Jetty to 12.1.5 by nastra · Pull Request #10837 · apache/iceberg

nastra · 2024-08-01T13:49:12Z

Now that we fixed #10338 and moved to JDK 17 we're finally able to upgrade Jetty and the Servlet API to the latest version

github-actions · 2024-11-10T00:16:21Z

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

nastra · 2026-01-13T15:55:06Z

Looks like Spark tests are failing because Spark internally still depends on an older Jetty version:


jakarta/servlet/SingleThreadModel
java.lang.NoClassDefFoundError: jakarta/servlet/SingleThreadModel
	at org.sparkproject.jetty.servlet.ServletHolder.setServlet(ServletHolder.java:173)
	at org.sparkproject.jetty.servlet.ServletHolder.<init>(ServletHolder.java:120)
	at org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:121)
	at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:69)
	at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:253)
	at org.apache.spark.SparkContext.<init>(SparkContext.scala:521)
	at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:3076)
	at org.apache.spark.sql.classic.SparkSession$Builder.$anonfun$build$2(SparkSession.scala:1015)
	at scala.Option.getOrElse(Option.scala:201)
	at org.apache.spark.sql.classic.SparkSession$Builder.build(SparkSession.scala:1006)
	at org.apache.spark.sql.classic.SparkSession$Builder.getOrCreate(SparkSession.scala:1035)
	at org.apache.spark.sql.classic.SparkSession$Builder.getOrCreate(SparkSession.scala:908)
	at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:923)
	at org.apache.iceberg.spark.TestBase.startMetastoreAndSpark(TestBase.java:83)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: java.lang.ClassNotFoundException: jakarta.servlet.SingleThreadModel
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
	... 16 more


jakarta.servlet.SingleThreadModel
java.lang.ClassNotFoundException: jakarta.servlet.SingleThreadModel
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
	at org.sparkproject.jetty.servlet.ServletHolder.setServlet(ServletHolder.java:173)
	at org.sparkproject.jetty.servlet.ServletHolder.<init>(ServletHolder.java:120)
	at org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:121)
	at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:69)
	at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:253)
	at org.apache.spark.SparkContext.<init>(SparkContext.scala:521)
	at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:3076)
	at org.apache.spark.sql.classic.SparkSession$Builder.$anonfun$build$2(SparkSession.scala:1015)
	at scala.Option.getOrElse(Option.scala:201)
	at org.apache.spark.sql.classic.SparkSession$Builder.build(SparkSession.scala:1006)
	at org.apache.spark.sql.classic.SparkSession$Builder.getOrCreate(SparkSession.scala:1035)
	at org.apache.spark.sql.classic.SparkSession$Builder.getOrCreate(SparkSession.scala:908)
	at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:923)
	at org.apache.iceberg.spark.TestBase.startMetastoreAndSpark(TestBase.java:83)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

manuzhang · 2026-01-21T04:34:22Z

Can the jetty depended by Iceberg tests be separated from that by Spark? I don't think iceberg-spark-runtime needs to include jetty dependency from Iceberg.

manuzhang · 2026-02-03T15:00:24Z

@nastra Are you still working on this?

nastra · 2026-02-03T15:31:46Z

@nastra Are you still working on this?

I'm not actively working on this right now

manuzhang · 2026-02-04T02:38:08Z

@nastra I can pick this up if you don't have time currently.

nastra · 2026-02-04T05:56:34Z

@manuzhang sure, just go ahead and open a separate PR

manuzhang · 2026-02-13T06:11:37Z

This work is continued in #15232, please help review @nastra

nastra · 2026-02-13T15:39:19Z

@manuzhang #15232 contained too many unrelated changes and was also missing a bunch of other places where the UI config had to be disabled. However, the idea of disabling the UI is correct and I've added those here and added you as the author.

@singhpk234 @amogh-jahagirdar we should merge the PR using "Rebase and merge" so that the individual commits get merged separately

nastra · 2026-02-13T15:44:16Z

        CreateMultipartUploadRequest.builder().bucket(BUCKET).key("random/multipart-key").build());
  }

+  @SuppressWarnings("removal")


this is needed because GzipHandler is now deprecated. We'll be switching to Jetty's new Compression API in #15043

nastra · 2026-02-13T15:45:50Z

    this.spark =
        SparkSession.builder()
-            .config("spark.ui.enabled", false)
+            .config(TestBase.DISABLE_UI)


The main issue with Spark is that Spark brings its own (older) Jetty version for the UI, which conflicts with the Jetty version we're using. The downside is that we need to disable the UI pretty much everywhere

Spark only upgraded to jetty 12 after 4.2, so I believe it's worthwhile. As a follow-up, I plan to create a util method to start SparkSession such that we don't need to copy the same config everywhere.

Does this mean it's going to be an issue for downstream users? If not why don't we switch these benchmarks to use the fat jar?

nastra · 2026-02-16T07:28:51Z

@singhpk234 @amogh-jahagirdar can you guys please review this one?

nastra · 2026-02-16T16:23:06Z

    testImplementation project(path: ':iceberg-hive-metastore', configuration: 'testArtifacts')
    testImplementation project(path: ":iceberg-spark:iceberg-spark-${sparkMajorVersion}_${scalaVersion}", configuration: 'testArtifacts')
-    testImplementation (project(path: ':iceberg-open-api', configuration: 'testFixturesRuntimeElements')) {
-      transitive = false


this was initially done because there were conflicts. However, given that we don't have any conflicts anymore we can remove this

manuzhang · 2026-02-17T06:28:04Z

@nastra I prefer to merge this in one commit since the first commit doesn't pass test and we can't revert the commits separately.

manuzhang · 2026-03-07T14:41:44Z

@nastra can you please rebase this PR?

…Jetty version

RussellSpitzer

I would hope we could get away without disabling the UI, this seems like a bit of a code smell. If our builds can't handle multiple versions are we sure this is going to work with end users of the fat jar/ using their own packages?

I don't often use the UI for debugging tests but it feels like we need to make sure we aren't just masking what would be a downstream issue.

RussellSpitzer

Ok I looked closer, I see that we are really just replacing a constant. We were disabling the UI before in most cases anyway.

amogh-jahagirdar · 2026-03-30T19:49:32Z

Yeah sorry for the late review @nastra , I checked this out last week and had the same realization as @RussellSpitzer above that the UI was disabled anyways, I think it looks good to me

With the new Jetty upgrade in apache/iceberg#10837 we're running into the following: Failing CI: https://github.com/apache/iceberg-python/actions/runs/24416422109/job/71326697962 ```json Malformed request: { "message":"Suspicious Path Character", "url":"http://rest:8181/v1/namespaces/default%1Ftest_positional_mor_deletes_v2/tables/branch_without_5?snapshots=all", "status":"400" } ``` The `namespace-seperator` has been introduced afther the 1.10.x branch (apache#2826), so it isn't out in public, meaning the Spark integration tests are not respecting this config. This requires us to downgrade for now. I think disallowing the default (`%1F`) separator is problematic, since it might break compatibility.

github-actions Bot added core build labels Aug 1, 2024

nastra commented Aug 1, 2024

View reviewed changes

Comment thread build.gradle

nastra commented Aug 1, 2024

View reviewed changes

Comment thread core/src/test/java/org/apache/iceberg/rest/RESTCatalogServlet.java

nastra commented Aug 1, 2024

View reviewed changes

Comment thread gradle/libs.versions.toml Outdated

nastra changed the title ~~Core: Upgrade Jetty and Servlet API~~ Core: Upgrade Jetty to 12.0.12 and Servlet API Aug 2, 2024

nastra changed the title ~~Core: Upgrade Jetty to 12.0.12 and Servlet API~~ Core: Upgrade Jetty to 12.0.12 Aug 2, 2024

nastra force-pushed the bump-jetty-and-servlet-api branch from b0cf998 to 0f7e32c Compare August 2, 2024 12:19

github-actions Bot added the stale label Nov 10, 2024

nastra added not-stale and removed stale labels Nov 10, 2024

nastra force-pushed the bump-jetty-and-servlet-api branch from 0f7e32c to 1a902a0 Compare January 13, 2026 12:36

nastra changed the title ~~Core: Upgrade Jetty to 12.0.12~~ Core: Upgrade Jetty to 12.1.5 Jan 13, 2026

nastra requested review from amogh-jahagirdar and singhpk234 January 13, 2026 12:38

nastra force-pushed the bump-jetty-and-servlet-api branch from 1a902a0 to eb3162e Compare January 13, 2026 12:42

github-actions Bot added AWS OPENAPI labels Jan 13, 2026

nastra mentioned this pull request Jan 13, 2026

AWS, Core: Switch Jetty to use new Compression API for GZIP #15043

Merged

nastra force-pushed the bump-jetty-and-servlet-api branch from eb3162e to 24c1805 Compare January 13, 2026 13:54

github-actions Bot added the spark label Jan 13, 2026

nastra force-pushed the bump-jetty-and-servlet-api branch 2 times, most recently from f33bd66 to 24c1805 Compare January 13, 2026 15:54

nastra marked this pull request as draft January 13, 2026 15:58

manuzhang mentioned this pull request Feb 12, 2026

Core: Upgrade Jetty to 12.1.5 #15232

Closed

nastra force-pushed the bump-jetty-and-servlet-api branch 2 times, most recently from 6b15c1a to c9d14a6 Compare February 13, 2026 15:35

nastra marked this pull request as ready for review February 13, 2026 15:41

nastra commented Feb 13, 2026

View reviewed changes

nastra force-pushed the bump-jetty-and-servlet-api branch 2 times, most recently from c9a3108 to 49cdaf1 Compare February 13, 2026 17:45

nastra force-pushed the bump-jetty-and-servlet-api branch from 49cdaf1 to b4e9819 Compare February 16, 2026 16:20

nastra commented Feb 16, 2026

View reviewed changes

manuzhang approved these changes Feb 17, 2026

View reviewed changes

nastra and others added 2 commits March 9, 2026 16:05

Build: Bump Jetty to 12.1.5

085a0f9

Spark: Disable UI configs to avoid dependency conflicts with Spark's …

a025f53

…Jetty version

nastra force-pushed the bump-jetty-and-servlet-api branch from 9002df6 to a025f53 Compare March 9, 2026 15:09

RussellSpitzer reviewed Mar 30, 2026

View reviewed changes

RussellSpitzer approved these changes Mar 30, 2026

View reviewed changes

amogh-jahagirdar approved these changes Mar 30, 2026

View reviewed changes

nastra merged commit 3550bce into apache:main Apr 3, 2026
35 checks passed

nastra deleted the bump-jetty-and-servlet-api branch April 3, 2026 14:14

Fokko mentioned this pull request Apr 14, 2026

Build: Bump daft from 0.6.14 to 0.7.7 apache/iceberg-python#3232

Merged

Fokko mentioned this pull request Apr 14, 2026

CI: Lock REST Fixture to 1.10.1 apache/iceberg-python#3240

Merged

Conversation

nastra commented Aug 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Nov 10, 2024

Uh oh!

nastra commented Jan 13, 2026

Uh oh!

manuzhang commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

manuzhang commented Feb 3, 2026

Uh oh!

nastra commented Feb 3, 2026

Uh oh!

manuzhang commented Feb 4, 2026

Uh oh!

nastra commented Feb 4, 2026

Uh oh!

manuzhang commented Feb 13, 2026

Uh oh!

nastra commented Feb 13, 2026

Uh oh!

nastra Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

nastra Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

manuzhang Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

RussellSpitzer Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

nastra commented Feb 16, 2026

Uh oh!

nastra Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

manuzhang commented Feb 17, 2026

Uh oh!

manuzhang commented Mar 7, 2026

Uh oh!

RussellSpitzer left a comment

Choose a reason for hiding this comment

Uh oh!

RussellSpitzer left a comment

Choose a reason for hiding this comment

Uh oh!

amogh-jahagirdar commented Mar 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

nastra commented Aug 1, 2024 •

edited

Loading

manuzhang commented Jan 21, 2026 •

edited

Loading