Importing mutation tests warnings

[mgonzcast]

which versions are you using: 8.9.1 LTS
what are you trying to achieve: Making sure the quality of unit tests
what have you tried so far to achieve this: Use mutation pitests
I am checking on a Java code to see if unit tests are good and get a handle of the current situation in regards on unit tests.

I took someone else´s quality profile rules for Java and added mutation pitest rules.

Mutation tests are reporting quite poor unit tests (50% strength of unit tests).

When I try to import the pitests I get a bunch of warning saying:

WARN: Found unknown mutation operator: org.pitest.mutationtest.engine.gregor.mutators.returns.NullReturnValsMutator

or

WARN: Found unknown mutation operator: WARN: Found unknown mutation operator: org.pitest.mutationtest.engine.gregor.mutators.returns.EmptyObjectReturnValsMutator

and many similar warnings.

I run the scanner like this:

“C:\software\PackSonar\sonar-scanner-4.6.0.2311-windows\bin\sonar-scanner.bat” -D"sonar.projectKey=ProjectName" -D"sonar.sources=." -D"sonar.host.url=http://localhost:9000" -D"sonar.login=fffffffffffffffffffffffffffffffffffffffffffff"
-D"sonar.dependencyCheck.htmlReportPath=.\ProjectName\dependency-check-report.html" -D"sonar.dependencyCheck.xmlReportPath=.\ProjectName\dependency-check-report.xml" -D"sonar.coverage.jacoco.xmlReportPaths=.\ProjectName\jacoco.xml" -D"dc5.mutationAnalysis.pitest.sensor.reports.directory=.\ProjectName\target\pit-reports\202110281422"

I am wondering if there is any issue that I should be aware and if so, how to avoid it

[mgonzcast]

anyone there @gmuecke @artsok ?

[gmuecke]

Hi @mgonzcast , there has been a PR that should fix this issue #52
May I ask you, to check out the version from development branch & build & test it on your setup if it solves your problem?

[mgonzcast]

Thanks @gmuecke for answering. I am using pitest 1.7.1 so I am not sure if this applies entirely in my case. Should I switch to pitest 1.6.9 to avoid those messages?

Do I have to build myself the development version 1.6? Any instructions for building?

Wouldn´t it make more sense to backport these features in the 1.5.x production branch?

[gmuecke]

Hi @mgonzcast , the warnings come from the mutators got refacotored into different packages, causing this message.
Reverting to 1.6.9 would probably help - as a workaround.

I plan to release the plugin 1.6 soon, so default upgrade path would be to upgrade via SQ marketplace - as usual.
But I want to be sure, that problem is really fixed prior to release.

So if you're familiar with java & maven, just checkout and run mvn verify and copy the jar file to the extensions/plugin folder of SQ. If not, then just wait until the 1.6 is released (if I find the time, it should be within the next 2 weeks)

[mgonzcast]

Thanks @gmuecke !

Since I am using an old version of Maven (3.0.4) I had to add these entries in the pom.xml file right before , otherwise Maven complained of using http instead of https. I am using JDK 1.8:

<pluginRepositories>
        <pluginRepository>
            <id>central</id>
            <name>Central Repository</name>
            <url>https://repo.maven.apache.org/maven2</url>
            <layout>default</layout>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
            <releases>
                <updatePolicy>never</updatePolicy>
            </releases>
        </pluginRepository>
    </pluginRepositories>
    <repositories>
        <repository>
            <id>central</id>
            <name>Central Repository</name>
            <url>https://repo.maven.apache.org/maven2</url>
            <layout>default</layout>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </repository>
    </repositories>
</project>

I got the following error:

Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.076 sec - in ch.devcon5.sonar.plugins.mutationanalysis.sensors.TestMetricsWriterTest
Running ch.devcon5.sonar.plugins.mutationanalysis.StreamsTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - in ch.devcon5.sonar.plugins.mutationanalysis.StreamsTest

Results :

Failed tests:
  PitestSensorTest.execute_coverageThresholdRuleActive_belowThreshold_moreMutantsMissing:361 No element found matching assertions
  PitestSensorTest.execute_coverageThresholdRuleActive_belowThreshold_twoMutantMissing:337 No element found matching assertions
  RulesProcessorTest.processRules_coverageThresholdRuleActive_customEffortFactor_coverageTooLow_issueCreated:363->assertIssueAtLine:457->assertIssueAtLine:480 expected:<...ion coverage from 75[.0% to 80.]0%> but was:<...ion coverage from 75[,0% to 80,]0%>
  RulesProcessorTest.processRules_coverageThresholdRuleActive_defaultEffortFactor_coverageTooLow_issueCreated:340->assertIssueAtLine:457->assertIssueAtLine:480 expected:<...ion coverage from 75[.0% to 80.]0%> but was:<...ion coverage from 75[,0% to 80,]0%>
  RulesProcessorTest.processRules_coverageThresholdRuleActive_gapIsSamllerThan05_RoundedUp_issueCreated:446->assertIssueAtLine:457->assertIssueAtLine:480 expected:<...ion coverage from 77[.8% to 80.]0%> but was:<...ion coverage from 77[,8% to 80,]0%>

Tests run: 274, Failures: 5, Errors: 0, Skipped: 0

[gmuecke]

It's the same issue as described here #53 (comment) ... What is your system's locale?

[mgonzcast]

Thanks @gmuecke! As you have guessed my system locale is "es" :)

I have successfully build the 1.6 plugin and I don´t get more warnings of unknown mutation operators

I still get "WARNING : Minion exited abnormally due to TIMED_OUT"

Should I increase 10000 ms of timeout?

Thanks again for your quick responses!

[gmuecke]

Is fixed in Release 1.6,
public announcement https://community.sonarsource.com/t/new-release-mutation-analysis-1-6/54174

Soon to be available in Market Place

Thank you very much for reporting the issue and the contribution!