HTML report is green while console report shows defects

[AnastasiiaSergienko]

Description

When I run this command java -jar "$tmp_file" trace -o html > report.html, the report is green:

When I run this command java -jar "$tmp_file" trace, the report contains defects:

not ok - 170 total, 6 defect

And the second report is right because I have some typos and missing tracking comments in the code.

Steps to Reproduce

Can be reproduced on the following commit with the commands described above: exasol/exasol-rest-api@ff918b4

Expected behavior

I expect to see the same report in HTML format and on the console

Environment

• OFT: 3.3.0
• OS: Ubuntu 20.04.2 LTS
• Java Version: openjdk 11.0.11 2021-04-20

[redcatbear]

@kaklakariada , please check if this is still reproducible.

[kaklakariada]

Problem still exists with the latest OFT version from main.

Output of oft trace -o html > /tmp/report.html shows no errors: report.html

Output of `oft trace` shows errors

% oft trace
not ok [ in:  2 /  2 ✔ | out:  1 /  1 ✔ ] dsn~get-tables-headers~1 (impl, -itest)

  The endpoint requires `Authorization` header with an API token to handle requests.

  [covered shallow  ] ← impl~get-tables-headers-2891709974~0
  [covered shallow  ] ← impl~get-tables-headers-3531088591~0
  [covers           ] → req~get-tables~1

not ok [ in:  3 /  3 ✔ | out:  1 /  1 ✔ ] dsn~get-tables-response-body~1 (impl, utest, -itest)

  The endpoint has the following JSON response body format:
  
  ```json
  {
    "status": <"ok" | "error">,
    "tablesList": [
      {
        "schemaName": "<schema>",
        "tableName": "<table>"
      },
      ...
    ],
    // in case of "error"
    "exception": "<error code and message>"
  }

[covered shallow ] ← implget-tables-response-body-13556443380
[covered shallow ] ← implget-tables-response-body-19697447240
[covers ] → reqsupport-json-request-and-response-format1
[covered shallow ] ← utestget-tables-response-body-19748562770

not ok [ in: 0 / 0 | out: 1 / 1 ✔ ] dsninsert-row-endpoint1 (-impl, -itest)

ERA provides the following endpoint to the API users: /api/v1/row [post]

[covers ] → reqinsert-row1

not ok [ in: 3 / 3 ✔ | out: 1 / 1 ✔ ] dsnupdate-rows-endpoint1 (itest, -impl)

ERA provides the following endpoint to the API users: /api/v1/rows [put]

[covered shallow ] ← itestupdate-rows-endpoint-12131957010
[covered shallow ] ← itestupdate-rows-endpoint-17539442880
[covered shallow ] ← itestupdate-rows-endpoint-31475673810
[covers ] → requpdate-rows1

not ok [ in: 1 / 1 ✔ | out: 1 / 1 ✔ ] dsnupdate-rows-response-body1 (impl, -itest)

The endpoint has the following JSON response body format:

{
  "status": <"ok" | "error">,
  // in case of "error"
  "exception": "<error code and message>"
}

[covered shallow ] ← implupdate-rows-response-body-11345362440
[covers ] → reqsupport-json-request-and-response-format1

not ok [ in: 9 / 9 ✔ | out: 0 / 0 ] featexasol-rest-endpoints1 (req)

ERA provides REST API endpoints that allow API Users to interact with the Exasol database.

[covered shallow ] ← reqcommunication-with-exasol1
[covered shallow ] ← reqdelete-rows1
[covered shallow ] ← reqexecute-query1
[covered shallow ] ← reqexecute-statement1
[covered shallow ] ← reqget-rows1
[covered shallow ] ← reqget-tables1
[covered shallow ] ← reqinsert-row1
[covered shallow ] ← reqsupport-json-request-and-response-format1
[covered shallow ] ← requpdate-rows1

not ok [ in: 2 / 2 ✔ | out: 1 / 1 ✔ ] reqget-tables1 (dsn)

API users can get a list of table names by a schema name.

[covered shallow ] ← dsnget-tables-endpoint1
[covered shallow ] ← dsnget-tables-headers1
[covers ] → featexasol-rest-endpoints1

not ok [ in: 2 / 2 ✔ | out: 1 / 1 ✔ ] reqinsert-row1 (dsn)

API users can insert a single row into a table.

[covered shallow ] ← dsninsert-row-endpoint1
[covered shallow ] ← dsninsert-row-headers1
[covers ] → featexasol-rest-endpoints1

not ok [ in: 11 / 11 ✔ | out: 1 / 1 ✔ ] reqsupport-json-request-and-response-format1 (dsn)

ERA supports a JSON request and response format.

[covered shallow ] ← dsndelete-rows-request-body1
[covered shallow ] ← dsndelete-rows-response-body1
[covered shallow ] ← dsnexecute-query-response-body1
[covered shallow ] ← dsnexecute-statement-request-body1
[covered shallow ] ← dsnexecute-statement-response-body1
[covered shallow ] ← dsnget-rows-response-body1
[covered shallow ] ← dsnget-tables-response-body1
[covered shallow ] ← dsninsert-row-request-body1
[covered shallow ] ← dsninsert-row-response-body1
[covered shallow ] ← dsnupdate-rows-request-body1
[covered shallow ] ← dsnupdate-rows-response-body1
[covers ] → featexasol-rest-endpoints1

not ok [ in: 2 / 2 ✔ | out: 1 / 1 ✔ ] requpdate-rows1 (dsn)

API users can update rows from a table based on a condition.

[covered shallow ] ← dsnupdate-rows-endpoint1
[covered shallow ] ← dsnupdate-rows-headers1
[covers ] → featexasol-rest-endpoints1

not ok - 162 total, 10 defect

</details>

[kaklakariada]

Root cause: File design.md in the project contains text like <endpoint>. OFT copies this to the HTML report which confuses the browser and causes the rendering to stop, leading to an incompletely rendered HTML report. Removing the tags from design.md fixes this. The report is then rendered correctly:

A possible solution could be to replace chars like < and > with HTML entities <and >.