Skip to content

Upgrade nb-javac to JDK 25b31#8572

Merged
mbien merged 4 commits intoapache:masterfrom
mbien:nb-javac25ea
Jul 14, 2025
Merged

Upgrade nb-javac to JDK 25b31#8572
mbien merged 4 commits intoapache:masterfrom
mbien:nb-javac25ea

Conversation

@mbien
Copy link
Member

@mbien mbien commented Jun 9, 2025

Upgrade javac integration to nb-javac based on JDK 25 build 31

this is not meant to be integrated yet, the wrapped nb-javac is an experimental build which was backported to JDK 17 instead of 8 to reduce the delta to upstream. It was built from this old branch I created a while ago.

nb-javac 25 is not available yet, uses a temporary build for now with extra patches.

refs to javac regressions which were discovered during testing:

@mbien mbien added Java [ci] enable extra Java tests (java.completion, java.source.base, java.hints, refactoring.java, form) Upgrade Library Library (Dependency) Upgrade ci:all-tests [ci] enable all tests ci:dev-build [ci] produce a dev-build zip artifact (7 days expiration, see link on workflow summary page) labels Jun 9, 2025
@mbien
Copy link
Member Author

mbien commented Jun 9, 2025

javadoc doesn't build on JDK 25, i saw that already in #8482 but hoped that it would work as soon we bump nb-javac (like it was the case in the past)

 /home/runner/work/netbeans/netbeans/nbbuild/javadoctools/template.xml:284: Warning: Could not find file /home/runner/work/netbeans/netbeans/nbbuild/build/javadoc/org-netbeans-api-annotations-common/resource-files/glass.png to copy.

see #8633 for javadoc fix

for hints LambdaTest and ConvertVarToExplicitTypeTest need looking at.
java.completion test data needs updating most likely.

@mbien mbien added JavaDoc [ci] enable java/javadoc tests and build-javadoc target LSP [ci] enable Language Server Protocol tests and removed ci:all-tests [ci] enable all tests labels Jun 9, 2025
@lahodaj
Copy link
Contributor

lahodaj commented Jun 16, 2025

FWIW, I've opened:
JaroslavTulach/nb-javac#30

@mbien
Copy link
Member Author

mbien commented Jun 16, 2025

FWIW, I've opened:
JaroslavTulach/nb-javac#30

@lahodaj thanks! I can refresh this PR once a staged build is available and drop the experimental nb-javac17 bits.

@mbien
Copy link
Member Author

mbien commented Jul 3, 2025

switched to a temporary hosted build of nb-javac 25b29 until its available, the same tests are failing as before (when the experimental build was used)

@mbien mbien added this to the NB27 milestone Jul 5, 2025
@mbien mbien removed the JavaDoc [ci] enable java/javadoc tests and build-javadoc target label Jul 6, 2025
@mbien
Copy link
Member Author

mbien commented Jul 6, 2025

ComputeImportsTest failed due to CCE in javac Lint class. Likely due to openjdk/jdk#22056, disabled test case since it influenced code scanning of other tests.

@lahodaj
Copy link
Contributor

lahodaj commented Jul 6, 2025

ComputeImportsTest failed due to CCE in javac Lint class. Likely due to openjdk/jdk#22056, disabled test case since it influenced code scanning of other tests.

Thanks. FWIW, I've opened:
openjdk/jdk#26142

(The failure in LSP tests/ServerTest also seems to be caused by this.)

@mbien
Copy link
Member Author

mbien commented Jul 6, 2025

will take a look at the (hopefully) last remaining failing test later (java.source.base).

update:

MRJARCachingFileManagerTest#testJavac is failing with:

java.lang.IllegalArgumentException: Path component should be '/'
Working directory: /home/mbien/NetBeansProjects/netbeans/java/java.source.base/build/test/unit/work/o.n.m.j.s.p.M/j
	at java.base/sun.nio.fs.UnixFileSystemProvider.checkUri(UnixFileSystemProvider.java:81)
	at java.base/sun.nio.fs.UnixFileSystemProvider.newFileSystem(UnixFileSystemProvider.java:90)
	at java.base/java.nio.file.FileSystems.newFileSystem(FileSystems.java:339)
	at com.sun.tools.javac.file.JavacFileManager$ArchiveContainer.<init>(JavacFileManager.java:586)
	at com.sun.tools.javac.file.JavacFileManager.getContainer(JavacFileManager.java:337)
	at com.sun.tools.javac.file.JavacFileManager.pathsAndContainers(JavacFileManager.java:1101)
	at com.sun.tools.javac.file.JavacFileManager.indexPathsAndContainersByRelativeDirectory(JavacFileManager.java:1056)
	at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1220)
	at com.sun.tools.javac.file.JavacFileManager.pathsAndContainers(JavacFileManager.java:1044)
	at com.sun.tools.javac.file.JavacFileManager.list(JavacFileManager.java:792)
	at org.netbeans.modules.java.source.parsing.MRJARCachingFileManagerTest.testJavac(MRJARCachingFileManagerTest.java:176)

which is due to this new transformation within nb-javac:

java.nio.file.FileSystems.newFileSystem($path, $env, $classloader) :: $path instanceof java.nio.file.Path
=>
java.nio.file.FileSystems.newFileSystem($path.toUri(), $env, $classloader)
;;

the fascinating detail here is that the second factory method fails, since the uri checks are different from the path checks:

        // JDK 13+ API
        FileSystem fs1 = FileSystems.newFileSystem(
                Path.of("/mnt/ram/netbeans/java/java.source.base/build/test/unit/work/o.n.m.j.s.p.M/testJavac/mr.jar"),
                Map.of(),
                (ClassLoader)null
        );
        // JDK 8 API
        FileSystem fs2 = FileSystems.newFileSystem(
                Path.of("/mnt/ram/netbeans/java/java.source.base/build/test/unit/work/o.n.m.j.s.p.M/testJavac/mr.jar").toUri(),
                Map.of(),
                (ClassLoader)null
        );

first call creates a ZipFileSystem, I believe the second attempts to create a UnixFileSystem, given the call into UnixFileSystemProvider.

Javadoc makes also slightly different promises path variant, uri variant.

javac 24 worked since it didn't use the env param and used the path variant, essentially ending up doing something similar to the fs1 call.

Easiest fix would be to bump nb-javac to 17 to drop some of the transformations which aren't needed for NB anyway. But we would have to find a new maven namespace for that most likely.

Second option is to use the path+env variant on JDK 13+ via reflection, and the path (without env) variant on <13.

@neilcsmith-net
Copy link
Member

neilcsmith-net commented Jul 7, 2025

Easiest fix would be to bump nb-javac to 17 to drop some of the transformations which aren't needed for NB anyway. But we would have to find a new maven namespace for that most likely.

This library exists to support NetBeans, so why is upgrading to JDK 17 an issue?! If it needs to move to new hosting and to a new Maven namespace to ensure it's properly responsive to our requirements, then let's work out how that happens. Perhaps a dev@ thread? I had previously mentioned to @geertjanw that FoAN would be a better home for this.

Of course, doing all that within the week left to get this required change in before freeze might be tricky! Still, this upgrade needs to be in NB27.

@mbien
Copy link
Member Author

mbien commented Jul 7, 2025

taking a look at JavaRefactoringActionsProviderTest. Its the last test of refactoring.java which is failing

update:
reason seems to be similar to #8572 (comment), the test case intentionally uses code with error (class declaration commented out) and javac 25 thinks it is a "compact java file" which leads to different results

testcase is named after 190101, unfortunately I can't find the issue outside of https://github.com/emilianbold/netbeans-releases/commit/dc3c8858a8b5a507227370ebd7968df9b6f1cc78

will likely turn the test off

@mbien
Copy link
Member Author

mbien commented Jul 7, 2025

yey, all java tests green!

will rebase/squash to a reasonable amount of commits, remove merge commits and give co-author credits.

PR ready to review, all what is missing is a nb-javac release + updating artifact locations. (keeping it as draft so that nobody merges it by accident with the dependency pointing to my raspi)

@mbien
Copy link
Member Author

mbien commented Jul 7, 2025

noticed during code scanner smoke test:

    public static void main(String[] args) {
        new WeakReference<>(null) {
            private Object hardRef = null;
        };
    }

will now cause a

modifier sealed not allowed here

javac error. WeakReference is non-sealed. Not sure if this is intended, but its not nb-specific.

@lahodaj
Copy link
Contributor

lahodaj commented Jul 8, 2025

noticed during code scanner smoke test:

    public static void main(String[] args) {
        new WeakReference<>(null) {
            private Object hardRef = null;
        };
    }

will now cause a

modifier sealed not allowed here

javac error. WeakReference is non-sealed. Not sure if this is intended, but its not nb-specific.

Uhhhh. Thanks!!! Filled as:
https://bugs.openjdk.org/browse/JDK-8361570
fix is coming.

@mbien
Copy link
Member Author

mbien commented Jul 8, 2025

squashed into logical units, the idea is to revert or drop the last commit with the next nb-javac / JDK build.

@mbien mbien added the ci:all-tests [ci] enable all tests label Jul 8, 2025
@mbien
Copy link
Member Author

mbien commented Jul 8, 2025

last showstopper is having no line numbers in stack traces

the emitted classes in nb-javac do seem to have a line number table (javap -v), so on first glance everything is ok (but I haven't diffed the javap outputs).

stack trace looks like:

Caused: java.lang.AssertionError: Unexpected tree: (ERROR) ? (ERROR) > entry : (ERROR) with kind: CONDITIONAL_EXPRESSION within: (ERROR) ? (ERROR) > entry : (ERROR) with kind: CONDITIONAL_EXPRESSION
	at com.sun.tools.javac.util.Assert.error(Unknown Source)
	at com.sun.tools.javac.comp.Attr$TypeAnnotationsValidator.validateAnnotatedType(Unknown Source)
	at com.sun.tools.javac.comp.Attr$TypeAnnotationsValidator.visitVarDef(Unknown Source)
	at com.sun.tools.javac.tree.JCTree$JCVariableDecl.accept(Unknown Source)
	at com.sun.tools.javac.tree.TreeScanner.scan(Unknown Source)
	at com.sun.tools.javac.tree.TreeScanner.scan(Unknown Source)
	at com.sun.tools.javac.comp.Attr$TypeAnnotationsValidator.visitMethodDef(Unknown Source)
	at com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(Unknown Source)
	at com.sun.tools.javac.tree.TreeScanner.scan(Unknown Source)
	at com.sun.tools.javac.comp.Attr$TypeAnnotationsValidator.visitClassDef(Unknown Source)
	at com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(Unknown Source)
	at com.sun.tools.javac.comp.Attr.validateTypeAnnotations(Unknown Source)

building nb-javac without passing --enable-preview to frgaal does seem to work:

Caused: java.lang.AssertionError: Unexpected tree: (ERROR) ? (ERROR) > entry : (ERROR) with kind: CONDITIONAL_EXPRESSION within: (ERROR) ? (ERROR) > entry : (ERROR) with kind: CONDITIONAL_EXPRESSION
	at com.sun.tools.javac.util.Assert.error(Assert.java:162)
	at com.sun.tools.javac.comp.Attr$TypeAnnotationsValidator.validateAnnotatedType(Attr.java:5937)
	at com.sun.tools.javac.comp.Attr$TypeAnnotationsValidator.visitVarDef(Attr.java:5776)
	at com.sun.tools.javac.tree.JCTree$JCVariableDecl.accept(JCTree.java:1066)
	at com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:50)
	at com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:58)
	at com.sun.tools.javac.comp.Attr$TypeAnnotationsValidator.visitMethodDef(Attr.java:5766)
	at com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:960)
	at com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:50)
	at com.sun.tools.javac.comp.Attr$TypeAnnotationsValidator.visitClassDef(Attr.java:5829)
	at com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:859)
	at com.sun.tools.javac.comp.Attr.validateTypeAnnotations(Attr.java:5726)

other curiosity:

  • setting -g --enable-preview breaks the bytecode?
    [junit] java.lang.ClassFormatError: Illegal field name "" in class com/sun/tools/javac/comp/Check
    [junit] 	at com.sun.tools.javac.comp.Modules.<init>(Modules.java:195)
    [junit] 	at com.sun.tools.javac.comp.Modules.instance(Modules.java:183)
  • setting -g:lines --enable-preview produces traces without lines as shown above
  • setting -g does appear to work, and was the config we ran with so far

opened JaroslavTulach/nb-javac#32

@mbien
Copy link
Member Author

mbien commented Jul 12, 2025

there is now a nb-javac build based on 25b31 available

  • going to switch the coordinates to maven central
  • and drop mbien@60166d3

everything should be still green after that

@mbien mbien changed the title Upgrade nb-javac to JDK 25-ea Upgrade nb-javac to JDK 25b31 Jul 12, 2025
@mbien mbien force-pushed the nb-javac25ea branch 2 times, most recently from a41c4a8 to f3cb49c Compare July 12, 2025 16:09
@mbien mbien removed the ci:all-tests [ci] enable all tests label Jul 12, 2025
@mbien
Copy link
Member Author

mbien commented Jul 12, 2025

something is wrong, JDK-8361445 doesn't seem to be in the nb-javac build

2025-07-12T17:48:06.4357931Z     [junit] java.lang.ClassCastException: class com.sun.tools.javac.code.Attribute$Error cannot be cast to class com.sun.tools.javac.code.Attribute$Constant (com.sun.tools.javac.code.Attribute$Error and com.sun.tools.javac.code.Attribute$Constant are in unnamed module of loader 'app')
2025-07-12T17:48:06.4358120Z     [junit] 	at com.sun.tools.javac.code.Lint.suppressionsFrom(Lint.java:533)
2025-07-12T17:48:06.4358377Z     [junit] 	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
2025-07-12T17:48:06.4358628Z     [junit] 	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
2025-07-12T17:48:06.4358815Z     [junit] 	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
2025-07-12T17:48:06.4359109Z     [junit] 	at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
2025-07-12T17:48:06.4359359Z     [junit] 	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
2025-07-12T17:48:06.4359630Z     [junit] 	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
2025-07-12T17:48:06.4359897Z     [junit] 	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
2025-07-12T17:48:06.4360173Z     [junit] 	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
2025-07-12T17:48:06.4360416Z     [junit] 	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
2025-07-12T17:48:06.4360670Z     [junit] 	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
2025-07-12T17:48:06.4360852Z     [junit] 	at com.sun.tools.javac.code.Lint.suppressionsFrom(Lint.java:524)
2025-07-12T17:48:06.4361095Z     [junit] 	at com.sun.tools.javac.code.Lint.suppressionsFrom(Lint.java:511)
2025-07-12T17:48:06.4361246Z     [junit] 	at com.sun.tools.javac.code.Lint.augment(Lint.java:82)
2025-07-12T17:48:06.4361462Z     [junit] 	at com.sun.tools.javac.comp.Attr.attribClass(Attr.java:5414)

sources of the Lint class do also not seem to include the fix:

    // Given a @SuppressWarnings annotation, extract the recognized suppressions
    private EnumSet<LintCategory> suppressionsFrom(Attribute.Compound suppressWarnings) {
        EnumSet<LintCategory> result = LintCategory.newEmptySet();
        Attribute.Array values = (Attribute.Array)suppressWarnings.member(names.value);
        for (Attribute value : values.values) {
            Optional.of((String)((Attribute.Constant)value).value)

^^L 533

JDK diff shows that it should be in it openjdk/jdk@jdk-25+29...jdk-25+31

edit: decompiled the class to double check and it seems to confirm it that the release isn't based on build 31:

$ java -jar cfr-0.152.jar /tmp/nb-javac-jdk-25+31/com/sun/tools/javac/code/Lint.class --methodname suppressionsFrom
...
private EnumSet<LintCategory> suppressionsFrom(Attribute.Compound suppressWarnings) {
    EnumSet<LintCategory> result = LintCategory.newEmptySet();
    Attribute.Array values = (Attribute.Array)suppressWarnings.member(this.names.value);
    for (Attribute value : values.values) {
        Optional.of((String)((Attribute.Constant)value).value).flatMap(LintCategory::get).filter(lc -> lc.annotationSuppression).ifPresent(result::add);
    }
    return result;
}

here the JDK commit for comparison (compare missing filter stage):
openjdk/jdk@21cb2ac

@mbien mbien added the do not merge Don't merge this PR, it is not ready or just demonstration purposes. label Jul 12, 2025
@mbien mbien force-pushed the nb-javac25ea branch 2 times, most recently from 39d5d08 to a793032 Compare July 13, 2025 17:58
mbien and others added 4 commits July 14, 2025 15:37
 - SourceUtils: update main method detector to work on JDK 25 without
   preview flag set
 - ErrorHintsProviderTest: update for javac 25 (error code changed)
 - account for internal API changes
 - fixes ConvertVarToExplicitTypeTest and LambdaTest

Co-authored-by: Michael Bien <mbien42@gmail.com>
 - mostly whitespace changes
JavaRefactoringActionsProviderTest#test190101:

 - javac 25 assumes everything without a class declaration is a compact
   source file -> disabled

JavaCompletionTaskXXXFeaturesTest:

 - Records (final) are a JDK 16+ feature. Reflecting this in the tests.
 - moving into the JDk 16 completion tests, using the appropriate
   source level.

Co-authored-by: Jan Lahoda <jan.lahoda@oracle.com>
@mbien mbien removed the do not merge Don't merge this PR, it is not ready or just demonstration purposes. label Jul 14, 2025
@mbien mbien marked this pull request as ready for review July 14, 2025 13:43
@mbien
Copy link
Member Author

mbien commented Jul 14, 2025

second attempt to switch to release artifacts (25+31.1)

Copy link
Contributor

@lahodaj lahodaj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks!

assertEquals(++expectedCount, called.get());
}

// hg blame: "#190101: preventing exceptions from various refactorings for very broken sources (without a top-level class)"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe there should be a GH Issue for this (although we have many GH issues, of course).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could potentially fix the test in a followup PR if we know what it should do.

My thoughts were that I wasn't sure if it was still needed. Because if javac inserts the compact-file body, the original scenario can't really happen anymore most likely (refactoring plugins throwing exceptions when the class decl line was missing).

btw the reason it failed was because assertNull(ui); was not null.

I couldn't find the issue the test referenced unfortunately (#8572 (comment))

@mbien
Copy link
Member Author

mbien commented Jul 14, 2025

thanks a lot for the help @lahodaj and review! merging.

@mbien mbien merged commit 0effd7f into apache:master Jul 14, 2025
37 checks passed
9FB1613756338EE706C23D53F1423E6925D127C8 com.dukescript.nbjavac:nb-javac:jdk-24-ga
AD9963C5C1DA19B72628F48364562C014A972B69 com.dukescript.nbjavac:nb-javac:jdk-24-ga:api
CD9EA8DDE23DF41A129D563A51C9A2E502BD85BF com.dukescript.nbjavac:nb-javac:jdk-25+31.1
183E5391BCCC0A27AE0F5014A3A9FA7F93214911 com.dukescript.nbjavac:nb-javac:jdk-25+31.1:api
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:dev-build [ci] produce a dev-build zip artifact (7 days expiration, see link on workflow summary page) Java [ci] enable extra Java tests (java.completion, java.source.base, java.hints, refactoring.java, form) LSP [ci] enable Language Server Protocol tests Upgrade Library Library (Dependency) Upgrade VSCode Extension

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

Comments