Skip to content

Conversation

@som-snytt
Copy link
Contributor

If running a single test, the pool shutdown
is likely to subvert waitFor the javac process.
Catch the InterruptedException and try again
with a timed wait of generous but finite duration,
to accommodate testing by developers. This quick
fix does not correct the race, which presumably
does not matter because of the order in which tests
are ordinarily submitted.

For example, tests/pos-java-interop/t2940:

sbt:scala3> testCompilation t2940
[snip]
[info] Test dotty.tools.dotc.CompilationTests.posTwice started
[                                        ] completed (0/1, 0 failed, 2s)java.lang.InterruptedException
        at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1640)compiler / Test / testOnly 3s
        at java.base/java.lang.ProcessImpl.waitFor(ProcessImpl.java:447)
[snip]
java.util.concurrent.ExecutionException: java.lang.InterruptedException
        at java.base/java.util.concurrent.ForkJoinTask.reportExecutionException(ForkJoinTask.java:581)
        at java.base/java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:993)
        at dotty.tools.vulpix.ParallelTesting.dotty$tools$vulpix$ParallelTesting$Test$$_$executeTestSuite$$anonfun$4(ParallelTesting.scala:687)
[snip long stack trace]

@som-snytt som-snytt marked this pull request as ready for review April 29, 2023 23:33
@anatoliykmetyuk anatoliykmetyuk requested a review from dwijnand May 1, 2023 13:47
If running a single test, the pool shutdown
is likely to subvert `waitFor` the javac process.
Catch the `InterruptedException` and try again
with a timed wait of generous but finite duration,
to accommodate testing by developers. This quick
fix does not correct the race, which presumably
does not matter because of the order in which tests
are ordinarily submitted.
@som-snytt som-snytt force-pushed the tweak/vulpix-spurious-interrupt branch from 05aa3ab to 0975d29 Compare May 3, 2023 22:25
@dwijnand dwijnand merged commit 983a314 into scala:main May 4, 2023
@som-snytt som-snytt deleted the tweak/vulpix-spurious-interrupt branch May 4, 2023 15:11
@som-snytt
Copy link
Contributor Author

Locally, I just saw a couple of interrupted tests while scala3-bootstrapped/test.

[info] Test dotty.tools.dotc.BootstrappedOnlyCompilationTests.posMacros started
[======================================> ] completed (327/328, 0 failed, 21s)java.lang.InterruptedException
        at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1722)
        at java.base/java.lang.ProcessImpl.waitFor(ProcessImpl.java:447)
        at dotty.tools.vulpix.ParallelTesting$Test.compileWithOtherCompiler(ParallelTesting.scala:628)
        at dotty.tools.vulpix.ParallelTesting$CompilationLogic.compileTestSource$$anonfun$1$$anonfun$1(ParallelTesting.scala:238)
        at scala.collection.immutable.List.map(List.scala:246)
        at dotty.tools.vulpix.ParallelTesting$CompilationLogic.compileTestSource$$anonfun$1(ParallelTesting.scala:238)
        at scala.util.Try$.apply(Try.scala:210)
        at dotty.tools.vulpix.ParallelTesting$CompilationLogic.dotty$tools$vulpix$ParallelTesting$CompilationLogic$$compileTestSource(ParallelTesting.scala:239)
        at dotty.tools.vulpix.ParallelTesting$$anon$4.checkTestSource$$anonfun$1(ParallelTesting.scala:285)
        at dotty.tools.vulpix.ParallelTesting$$anon$4.checkTestSource$$anonfun$adapted$1(ParallelTesting.scala:288)
        at scala.Function0.apply$mcV$sp(Function0.scala:42)
        at dotty.tools.vulpix.ParallelTesting$Test.tryCompile(ParallelTesting.scala:462)
        at dotty.tools.vulpix.ParallelTesting$$anon$4.checkTestSource(ParallelTesting.scala:288)
        at dotty.tools.vulpix.ParallelTesting$Test$LoggedRunnable.run(ParallelTesting.scala:358)
        at dotty.tools.vulpix.ParallelTesting$Test$LoggedRunnable.run$(ParallelTesting.scala:340)
        at dotty.tools.vulpix.ParallelTesting$$anon$4.run(ParallelTesting.scala:283)
        at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1403)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)
Interrupted (probably running after shutdown)
java.util.concurrent.ExecutionException: java.lang.InterruptedException

and

[info] Test dotty.tools.dotc.CompilationTests.explicitNullsRun started
[===============================>        ] completed (8/10, 0 failed, 0s)java.nio.channels.ClosedByInterruptException
        at java.base/java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:199)
        at java.base/sun.nio.ch.FileChannelImpl.endBlocking(FileChannelImpl.java:171)
        at java.base/sun.nio.ch.FileChannelImpl.size(FileChannelImpl.java:430)
        at java.base/java.nio.file.Files.createFileChannelLinesStream(Files.java:4123)
        at java.base/java.nio.file.Files.lines(Files.java:4109)
        at dotty.tools.utils$package$.toolArgsFor$$anonfun$1(utils.scala:68)
        at scala.collection.LinearSeqOps.foldLeft(LinearSeq.scala:183)
        at scala.collection.LinearSeqOps.foldLeft$(LinearSeq.scala:179)
        at scala.collection.immutable.List.foldLeft(List.scala:79)
        at dotty.tools.utils$package$.toolArgsFor(utils.scala:74)
        at dotty.tools.vulpix.ParallelTesting$TestSource.allToolArgs(ParallelTesting.scala:103)

@yilinwei
Copy link
Contributor

yilinwei commented Feb 7, 2024

According to the javadoc for the ExecutorService, this shouldn't happen.

The doc for shutdown states that:

Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted.

Since we call awaitTermination straight afterwards with a long timeout, all the tasks ought to be finished.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants