Unit tests following the NewSimpleTest model for C++ programs are not processed correctly.

[DKnoto]

Apache NetBeans version

Apache NetBeans 14

What happened

I use NetBeans 8.2 plugins to work on C/C++ projects. I implemented the unit tests using the GoogleTest library according to the guidelines in the documentation Adding Unit Tests to a C Project. This worked without any major problems until NB version 12.4. Unfortunately, since version 12.5, support for unit tests compatible with this model has collapsed.

NetBeans 14 Tests Results:

NetBeans 14 Output Clean Build:

NetBeans 14 Output Build Test:

"/usr/bin/make" -f nbproject/Makefile-Debug.mk QMAKE= SUBPROJECTS= .build-conf make[1]: Wejście do katalogu '/home/dknoto/Projekty/Ovation2D-Pointers' "/usr/bin/make" -f nbproject/Makefile-Debug.mk dist/Debug/GNU-Linux/libovation2d-pointers.a make[2]: Wejście do katalogu '/home/dknoto/Projekty/Ovation2D-Pointers' make[2]: 'dist/Debug/GNU-Linux/libovation2d-pointers.a' jest aktualne. make[2]: Opuszczenie katalogu '/home/dknoto/Projekty/Ovation2D-Pointers' make[1]: Opuszczenie katalogu '/home/dknoto/Projekty/Ovation2D-Pointers' "/usr/bin/make" -f nbproject/Makefile-Debug.mk SUBPROJECTS= .build-tests-conf make[1]: Wejście do katalogu '/home/dknoto/Projekty/Ovation2D-Pointers' "/usr/bin/make" -f nbproject/Makefile-Debug.mk dist/Debug/GNU-Linux/libovation2d-pointers.a make[2]: Wejście do katalogu '/home/dknoto/Projekty/Ovation2D-Pointers' make[2]: 'dist/Debug/GNU-Linux/libovation2d-pointers.a' jest aktualne. make[2]: Opuszczenie katalogu '/home/dknoto/Projekty/Ovation2D-Pointers' make[1]: Opuszczenie katalogu '/home/dknoto/Projekty/Ovation2D-Pointers' "/usr/bin/make" -f nbproject/Makefile-Debug.mk SUBPROJECTS= .test-conf make[1]: Wejście do katalogu '/home/dknoto/Projekty/Ovation2D-Pointers' %SUITE_STARTING% Highway Pointers Tests %SUITE_STARTED% %TEST_STARTED% CheckPointer1::EmptyInputNotInitialized (newsimpletest) %TEST_FINISHED% time=1 CheckPointer1::EmptyInputNotInitialized (newsimpletest) %TEST_STARTED% CheckPointer1::EmptyInputInitialized (newsimpletest) %TEST_FINISHED% time=1 CheckPointer1::EmptyInputInitialized (newsimpletest) %TEST_STARTED% CheckPointer1::BadFirstChar (newsimpletest) %TEST_FINISHED% time=1 CheckPointer1::BadFirstChar (newsimpletest) %TEST_STARTED% CheckPointer1::GoodFirstCharButEmptySecondChar (newsimpletest) %TEST_FINISHED% time=1 CheckPointer1::GoodFirstCharButEmptySecondChar (newsimpletest) %TEST_STARTED% CheckPointer1::GoodFirstCharButSecondCharDiff_H_or_h (newsimpletest) %TEST_FINISHED% time=1 CheckPointer1::GoodFirstCharButSecondCharDiff_H_or_h (newsimpletest) %TEST_STARTED% CheckPointer1::GoodFirstCharSecondCharEqualHiHAndFirstDigitEmpty (newsimpletest) %TEST_FINISHED% time=1 CheckPointer1::GoodFirstCharSecondCharEqualHiHAndFirstDigitEmpty (newsimpletest) %TEST_STARTED% CheckPointer1::GoodFirstCharSecondCharEqualLoHAndFirstDigitEmpty (newsimpletest) %TEST_FINISHED% time=1 CheckPointer1::GoodFirstCharSecondCharEqualLoHAndFirstDigitEmpty (newsimpletest) %TEST_STARTED% CheckPointer1::GoodFirstAndSecondCharButFirstDigitIsLessThanNine (newsimpletest) %TEST_FINISHED% time=1 CheckPointer1::GoodFirstAndSecondCharButFirstDigitIsLessThanNine (newsimpletest) %TEST_STARTED% CheckPointer1::GoodFirstAndSecondCharButFirstDigitIsGreaterThanNine (newsimpletest) %TEST_FINISHED% time=1 CheckPointer1::GoodFirstAndSecondCharButFirstDigitIsGreaterThanNine (newsimpletest) %TEST_STARTED% CheckPointer1::GoodFirstAndSecondCharButIndexLessThanOne (newsimpletest) %TEST_FINISHED% time=1 CheckPointer1::GoodFirstAndSecondCharButIndexLessThanOne (newsimpletest) %TEST_STARTED% CheckPointer1::GoodFirstAndSecondCharAndIndexEqualMinimum (newsimpletest) %TEST_FINISHED% time=1 CheckPointer1::GoodFirstAndSecondCharAndIndexEqualMinimum (newsimpletest) %TEST_STARTED% CheckPointer1::GoodFirstAndSecondAndThirdCharButSecondDigitIsLessThanZero (newsimpletest) %TEST_FINISHED% time=1 CheckPointer1::GoodFirstAndSecondAndThirdCharButSecondDigitIsLessThanZero (newsimpletest) %TEST_STARTED% CheckPointer1::GoodFirstAndSecondAndThirdCharButSecondDigitIsGreaterThanNine (newsimpletest) %TEST_FINISHED% time=1 CheckPointer1::GoodFirstAndSecondAndThirdCharButSecondDigitIsGreaterThanNine (newsimpletest) %TEST_STARTED% CheckPointer1::GoodAllChars (newsimpletest) %TEST_FINISHED% time=1 CheckPointer1::GoodAllChars (newsimpletest) %TEST_STARTED% CheckPointer1::GoodAllCharsMaximumPointerIndex (newsimpletest) %TEST_FINISHED% time=1 CheckPointer1::GoodAllCharsMaximumPointerIndex (newsimpletest) %TEST_STARTED% CheckPointer1::GoodAllCharsButIntputTooLong (newsimpletest) %TEST_FINISHED% time=1 CheckPointer1::GoodAllCharsButIntputTooLong (newsimpletest) %TEST_STARTED% CheckPointer1::GoodAllCharsButIntputTooLong2 (newsimpletest) %TEST_FINISHED% time=1 CheckPointer1::GoodAllCharsButIntputTooLong2 (newsimpletest) %TEST_STARTED% CheckPointer2::EmptyInputNotInitialized (newsimpletest) %TEST_FINISHED% time=1 CheckPointer2::EmptyInputNotInitialized (newsimpletest) %TEST_STARTED% CheckPointer2::EmptyInputInitialized (newsimpletest) %TEST_FINISHED% time=1 CheckPointer2::EmptyInputInitialized (newsimpletest) %TEST_STARTED% CheckPointer2::BadFirstChar (newsimpletest) %TEST_FINISHED% time=1 CheckPointer2::BadFirstChar (newsimpletest) %TEST_STARTED% CheckPointer2::GoodFirstCharButEmptySecondChar (newsimpletest) %TEST_FINISHED% time=1 CheckPointer2::GoodFirstCharButEmptySecondChar (newsimpletest) %TEST_STARTED% CheckPointer2::GoodFirstCharButSecondCharDiff_H_or_h (newsimpletest) %TEST_FINISHED% time=1 CheckPointer2::GoodFirstCharButSecondCharDiff_H_or_h (newsimpletest) %TEST_STARTED% CheckPointer2::GoodFirstCharSecondCharEqualHiHAndFirstDigitEmpty (newsimpletest) %TEST_FINISHED% time=1 CheckPointer2::GoodFirstCharSecondCharEqualHiHAndFirstDigitEmpty (newsimpletest) %TEST_STARTED% CheckPointer2::GoodFirstCharSecondCharEqualLoHAndFirstDigitEmpty (newsimpletest) %TEST_FINISHED% time=1 CheckPointer2::GoodFirstCharSecondCharEqualLoHAndFirstDigitEmpty (newsimpletest) %TEST_STARTED% CheckPointer2::GoodFirstAndSecondCharButFirstDigitIsLessThanNine (newsimpletest) %TEST_FINISHED% time=1 CheckPointer2::GoodFirstAndSecondCharButFirstDigitIsLessThanNine (newsimpletest) %TEST_STARTED% CheckPointer2::GoodFirstAndSecondCharButFirstDigitIsGreaterThanNine (newsimpletest) %TEST_FINISHED% time=1 CheckPointer2::GoodFirstAndSecondCharButFirstDigitIsGreaterThanNine (newsimpletest) %TEST_STARTED% CheckPointer2::GoodFirstAndSecondCharButIndexLessThanOne (newsimpletest) %TEST_FINISHED% time=1 CheckPointer2::GoodFirstAndSecondCharButIndexLessThanOne (newsimpletest) %TEST_STARTED% CheckPointer2::GoodFirstAndSecondCharAndIndexEqualMinimum (newsimpletest) %TEST_FINISHED% time=1 CheckPointer2::GoodFirstAndSecondCharAndIndexEqualMinimum (newsimpletest) %TEST_STARTED% CheckPointer2::GoodFirstAndSecondAndThirdCharButSecondDigitIsLessThanZero (newsimpletest) %TEST_FINISHED% time=1 CheckPointer2::GoodFirstAndSecondAndThirdCharButSecondDigitIsLessThanZero (newsimpletest) %TEST_STARTED% CheckPointer2::GoodFirstAndSecondAndThirdCharButSecondDigitIsGreaterThanNine (newsimpletest) %TEST_FINISHED% time=1 CheckPointer2::GoodFirstAndSecondAndThirdCharButSecondDigitIsGreaterThanNine (newsimpletest) %TEST_STARTED% CheckPointer2::GoodAllChars (newsimpletest) %TEST_FINISHED% time=1 CheckPointer2::GoodAllChars (newsimpletest) %TEST_STARTED% CheckPointer2::GoodAllCharsMaximumPointerIndex (newsimpletest) %TEST_FINISHED% time=1 CheckPointer2::GoodAllCharsMaximumPointerIndex (newsimpletest) %TEST_STARTED% CheckPointer2::GoodAllCharsButIntputTooLong (newsimpletest) %TEST_FINISHED% time=1 CheckPointer2::GoodAllCharsButIntputTooLong (newsimpletest) %TEST_STARTED% CheckPointer2::GoodAllCharsButIntputTooLong2 (newsimpletest) %TEST_FINISHED% time=1 CheckPointer2::GoodAllCharsButIntputTooLong2 (newsimpletest) %TEST_STARTED% CheckPointer3::EmptyInputNotInitialized (newsimpletest) %TEST_FINISHED% time=1 CheckPointer3::EmptyInputNotInitialized (newsimpletest) %TEST_STARTED% CheckPointer3::EmptyInputInitialized (newsimpletest) %TEST_FINISHED% time=1 CheckPointer3::EmptyInputInitialized (newsimpletest) %TEST_STARTED% CheckPointer3::BadFirstChar (newsimpletest) %TEST_FINISHED% time=1 CheckPointer3::BadFirstChar (newsimpletest) %TEST_STARTED% CheckPointer3::GoodFirstCharButEmptySecondChar (newsimpletest) %TEST_FINISHED% time=1 CheckPointer3::GoodFirstCharButEmptySecondChar (newsimpletest) %TEST_STARTED% CheckPointer3::GoodFirstCharButSecondCharDiff_H_or_h (newsimpletest) %TEST_FINISHED% time=1 CheckPointer3::GoodFirstCharButSecondCharDiff_H_or_h (newsimpletest) %TEST_STARTED% CheckPointer3::GoodFirstCharSecondCharEqualHiHAndFirstDigitEmpty (newsimpletest) %TEST_FINISHED% time=1 CheckPointer3::GoodFirstCharSecondCharEqualHiHAndFirstDigitEmpty (newsimpletest) %TEST_STARTED% CheckPointer3::GoodFirstCharSecondCharEqualLoHAndFirstDigitEmpty (newsimpletest) %TEST_FINISHED% time=1 CheckPointer3::GoodFirstCharSecondCharEqualLoHAndFirstDigitEmpty (newsimpletest) %TEST_STARTED% CheckPointer3::GoodFirstAndSecondCharButFirstDigitIsLessThanNine (newsimpletest) %TEST_FINISHED% time=1 CheckPointer3::GoodFirstAndSecondCharButFirstDigitIsLessThanNine (newsimpletest) %TEST_STARTED% CheckPointer3::GoodFirstAndSecondCharButFirstDigitIsGreaterThanNine (newsimpletest) %TEST_FINISHED% time=1 CheckPointer3::GoodFirstAndSecondCharButFirstDigitIsGreaterThanNine (newsimpletest) %TEST_STARTED% CheckPointer3::GoodFirstAndSecondCharButIndexLessThanOne (newsimpletest) %TEST_FINISHED% time=1 CheckPointer3::GoodFirstAndSecondCharButIndexLessThanOne (newsimpletest) %TEST_STARTED% CheckPointer3::GoodFirstAndSecondCharAndIndexEqualMinimum (newsimpletest) %TEST_FINISHED% time=1 CheckPointer3::GoodFirstAndSecondCharAndIndexEqualMinimum (newsimpletest) %TEST_STARTED% CheckPointer3::GoodFirstAndSecondAndThirdCharButSecondDigitIsLessThanZero (newsimpletest) %TEST_FINISHED% time=1 CheckPointer3::GoodFirstAndSecondAndThirdCharButSecondDigitIsLessThanZero (newsimpletest) %TEST_STARTED% CheckPointer3::GoodFirstAndSecondAndThirdCharButSecondDigitIsGreaterThanNine (newsimpletest) %TEST_FINISHED% time=1 CheckPointer3::GoodFirstAndSecondAndThirdCharButSecondDigitIsGreaterThanNine (newsimpletest) %TEST_STARTED% CheckPointer3::GoodAllChars (newsimpletest) %TEST_FINISHED% time=1 CheckPointer3::GoodAllChars (newsimpletest) %TEST_STARTED% CheckPointer3::GoodAllCharsMaximumPointerIndex (newsimpletest) %TEST_FINISHED% time=1 CheckPointer3::GoodAllCharsMaximumPointerIndex (newsimpletest) %TEST_STARTED% CheckPointer3::GoodAllCharsButIntputTooLong (newsimpletest) %TEST_FINISHED% time=1 CheckPointer3::GoodAllCharsButIntputTooLong (newsimpletest) %TEST_STARTED% CheckPointer3::GoodAllCharsButIntputTooLong2 (newsimpletest) %TEST_FINISHED% time=1 CheckPointer3::GoodAllCharsButIntputTooLong2 (newsimpletest) %SUITE_FINISHED% time=0 make[1]: Opuszczenie katalogu '/home/dknoto/Projekty/Ovation2D-Pointers'

NetBeans 12.4 Tests Results:

NetBeans 14 Output Build Test:

The content is identical to that of version 14.

How to reproduce

Implement unit tests using my NBNewSimpleTestPrinter class:

`/*

• File: RunAllTests.cpp
• Author: DKnoto
  */

#include

#include <gtest/gtest.h>
#include <stdlib.h>

namespace {
const char* TEST_SUITE_NAME = "Highway Pointers Tests";
}

using ::testing::EmptyTestEventListener;
using ::testing::InitGoogleTest;
using ::testing::Test;
using ::testing::TestCase;
using ::testing::TestEventListeners;
using ::testing::TestInfo;
using ::testing::TestPartResult;
using ::testing::UnitTest;

namespace
{
class NBNewSimpleTestPrinter : public EmptyTestEventListener
{
std::string currentTestName;

private:
    void OnTestProgramStart(const UnitTest& unit_test) override
    {
    }

    void OnTestProgramEnd(const UnitTest& unit_test) override
    {
        fprintf(stdout, "%%SUITE_FINISHED%% time=0\n");
        fflush(stdout);
    }

    void OnTestStart(const TestInfo& test_info) override
    {
        currentTestName.assign(test_info.test_case_name());
        currentTestName.append("::");
        currentTestName.append(test_info.name());
        fprintf(stdout,
            "%%TEST_STARTED%% %s (newsimpletest)\n",
            currentTestName.c_str());
        fflush(stdout);
    }

    void OnTestPartResult(const TestPartResult& test_part_result) override
    {
        if (!test_part_result.failed())
            return;
        fprintf(stdout,
            "%%TEST_FAILED%% time=0 testname=%s (newsimpletest) message=%s:%d, %s\n",
            currentTestName.c_str(),
            test_part_result.file_name(),
            test_part_result.line_number(),
            test_part_result.summary());
        fflush(stdout);
    }

    void OnTestEnd(const TestInfo& test_info) override
    {
        fprintf(stdout,
            "%%TEST_FINISHED%% time=1 %s (newsimpletest)\n",
            currentTestName.c_str());
        fflush(stdout);
    }
};

}

void initGoogleTest(int argc, char *argv, const char suiteName)
{
fprintf(stdout, "%%SUITE_STARTING%% %s\n", suiteName);
fflush(stdout);

testing::InitGoogleTest(&argc, argv);

UnitTest& unit_test = *UnitTest::GetInstance();
TestEventListeners& listeners = unit_test.listeners();
delete listeners.Release(listeners.default_result_printer());
listeners.Append(new NBNewSimpleTestPrinter);

fprintf(stdout, "%%SUITE_STARTED%%\n");
fflush(stdout);

}

int main(int argc, char **argv)
{
initGoogleTest(argc, argv, TEST_SUITE_NAME);
return RUN_ALL_TESTS();
}
`

Did this work correctly in an earlier version?

Apache NetBeans 12.4

Operating System

Linux 5.17.12-100.fc34.x86_64 #1 SMP PREEMPT

JDK

Java 11.0.15; OpenJDK 64-Bit Server VM 11.0.15+10

Apache NetBeans packaging

Apache NetBeans provided installer

Anything else

No response

Are you willing to submit a pull request?

No

Code of Conduct

Yes

[neilcsmith-net]

I'm not sure there is anything we can do about this. Migrating CND modules to Apache is still a work in progress. In the interim there is the cpplite cluster.

[DKnoto]

So, am I to understand that the unit test output analysis is supported in the plug-in with NB 8.2 and it cannot be improved?

[mbien]

So, am I to understand that the unit test output analysis is supported in the plug-in with NB 8.2 and it cannot be improved?

the CND plugin was written for an older version of NetBeans and hasn't changed since then. It would have to be updated and moved to the master branch first before there can be a release compatible with current NetBeans versions again.

Tracking bugs for CND doesn't make sense yet since nobody can fix them, going to close this. But lets say there will be a new release of CND one day and this issue still applies, feel free to reopen it.

[DKnoto]

Thanks for explanation. It's a pity, I will have to maintain two versions of NB, 12.4 for C / C ++ programming and NB 14 for Java :(

[DKnoto]

Problem solved. This was not a bug in the CND plugin but in the Apache NetBeans documentation. In the old documentation we have in the examples the use of stdout output, printf("%%..."), and in the new versions of Apache NetBeans you should use stderr. After such a change, fprintf(stderr, "%%..."), it works.