Skip to content

Add test summary for the end of lein test#164

Merged
nmburgan merged 1 commit intomainfrom
test_summary
Jan 16, 2026
Merged

Add test summary for the end of lein test#164
nmburgan merged 1 commit intomainfrom
test_summary

Conversation

@nmburgan
Copy link
Copy Markdown
Member

The test suite generates quite a lot of errors that are actually expected and not test failures, so it's hard to pick out what the real failures were. This adds a wrapper that collects all of the real failures and prints them out at the end of the test run.

Sample output of failures:

Ran 685 tests containing 13534 assertions.
3 failures, 0 errors.


========== FAILURE SUMMARY ==========

fail: puppetlabs.puppetdb.http.index-test/index-queries
  at index_test.clj:79
  expected: (re-matches #"(?s)Json parse error at line 1, column 17:\n\n
                \[\"from\",\"foobar\"\n
                \s+\^\n\n
                Unexpected end-of-input: expected close marker for Array
                \(start marker at \[Source: .*; line: 1, column: 1\]\)" body)
    actual: (not (re-matches #"(?s)Json parse error at line 1, column 17:\n\n
                \[\"from\",\"foobar\"\n
                \s+\^\n\n
                Unexpected end-of-input: expected close marker for Array
                \(start marker at \[Source: .*; line: 1, column: 1\]\)" "Json parse error at line 1, column 17:\n\n[\"from\",\"foobar\"\n               ^\n\nUnexpected end-of-input: expected close marker for Array (start marker at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 1])"))

fail: puppetlabs.puppetdb.http.index-test/index-queries
  at index_test.clj:79
  expected: (re-matches #"(?s)Json parse error at line 1, column 17:\n\n
                \[\"from\",\"foobar\"\n
                \s+\^\n\n
                Unexpected end-of-input: expected close marker for Array
                \(start marker at \[Source: .*; line: 1, column: 1\]\)" body)
    actual: (not (re-matches #"(?s)Json parse error at line 1, column 17:\n\n
                \[\"from\",\"foobar\"\n
                \s+\^\n\n
                Unexpected end-of-input: expected close marker for Array
                \(start marker at \[Source: .*; line: 1, column: 1\]\)" "Json parse error at line 1, column 17:\n\n[\"from\",\"foobar\"\n               ^\n\nUnexpected end-of-input: expected close marker for Array (start marker at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 1])"))

fail: puppetlabs.puppetdb.http.reports-test/query-report-with-malformed-json
  at reports_test.clj:222
  expected: (= (str "Json parse error at line 1, column 5:\n\n" "[\"=\"\n" "   ^\n\n" "Unexpected end-of-input: expected close marker for Array " "(start marker at [Source: (StringReader); line: 1, column: 1])") body)
    actual: (not (= "Json parse error at line 1, column 5:\n\n[\"=\"\n   ^\n\nUnexpected end-of-input: expected close marker for Array (start marker at [Source: (StringReader); line: 1, column: 1])" "Json parse error at line 1, column 5:\n\n[\"=\"\n   ^\n\nUnexpected end-of-input: expected close marker for Array (start marker at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 1])"))

======================================

Subprocess failed (exit code: 1)
Error encountered performing task 'test' with profile(s): 'defaults,dev-settings'
Subprocess failed (exit code: 1)
++ /tmp/code/ext/bin/pdbbox-env pg_ctl stop
waiting for server to shut down.... done
server stopped
+ rm -rf /tmp/code/core-test-jQ5ZR2/box
+ rm -rf /tmp/code/core-test-jQ5ZR2
Command failed! Command: docker exec -u postgres openvoxdb-test /bin/bash --login -c 'cd /tmp/code && NO_ACCEPTANCE=true ci/bin/run core+ext/openjdk17/pg-17', Exit code: 1

The test suite generates quite a lot of errors that are actually expected and not test failures, so it's hard to pick out what the real failures were. This adds a wrapper that collects all of the real failures and prints them out at the end of the test run.
@nmburgan nmburgan merged commit 902a917 into main Jan 16, 2026
11 of 27 checks passed
@nmburgan nmburgan deleted the test_summary branch January 16, 2026 20:40
bastelfreak added a commit to bastelfreak/openvox-server that referenced this pull request Jan 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants