Add documentation for JsonImporter

[Phanindra899]

This PR adds documentation for the JsonImporter.

The documentation explains:

• Expected JSON file structure
• Required and optional fields
• Configuration example
• Behavior and filtering logic
• Limitations and constraints

Closes #128

Let me know if any changes are needed.

[Phanindra899]

Hi @henrikingo,

I've added documentation for JsonImporter based on the current implementation.

Could you please review and let me know if any changes are needed?

Thanks!

[Phanindra899]

Hi @henrikingo,

It looks like the GitHub Actions workflow didn’t run due to fork security restrictions.

Could you please approve the workflow so the checks can run?

Thanks!

[henrikingo]

Thank you

Can you please execute the commands in the new documentation and copy paste the output in this discussion.

If there are files involved thrn please cat or head their contents.

[Phanindra899]

@henrikingo — I've verified the commands from the JSON documentation and confirmed they work as expected. Below is the minimal example I used for validation.

---

JSON File (test_data/sample.json)

[
  {
    "timestamp": 1711929600,
    "metrics": [
      { "name": "throughput", "value": 100 }
    ],
    "attributes": {
      "branch": "main"
    }
  }
]

---

Config File (otava.yaml)

tests:
  my_benchmark:
    type: json
    file: test_data/sample.json
    base_branch: main

---

Command Executed

otava analyze my_benchmark --config otava.yaml

---

Output

INFO: Computing change points for test my_benchmark...
time                       branch      throughput
-------------------------  --------  ------------
2024-04-01 00:00:00 +0000  main               100

---

Everything works as documented. Let me know if any adjustments are needed.

[henrikingo]

Yes, now with the added files it makes more sense :-)

Can you move the new files so that they are under otava/examples/json/*

I will add some other comments too, but now we have some meaningful direction to start from.

[henrikingo]

Json and Csv both have this property.

[henrikingo]

Please change this to the command that you are using in your reply today:

otava analyze my_benchmark --config otava.yaml

[henrikingo]

...and this under /examples/json/config/

[Phanindra899]

Hi @henrikingo, I’ve addressed all the requested changes:
• Moved example files to otava/examples/json/data/ and config/
• Updated configuration paths in documentation
• Fixed the example usage command
• Added required license headers

Please let me know if anything else needs improvement. Thanks!

[henrikingo]

Sorry I missed this the first time. Actually, this should be the time that the commit was merged to the branch we are tracking. Typically this is recorded in git's history as a merge-commit.

In most cases this should NOT be the time that the benchmark was run. In particular, it often happens that you want to rerun a benchmark with the system under test ghecked out and built from this same githash. In such a case, the new results should be recorded with this same timestamp. For the same githash, the timestamp should be constant, regardless of when or how many times the benchmark was run.

[henrikingo]

Ok thanks! The structure is clearer now and I think this will work.

I have several more comments now on wording or style issues. This is normal when you contribute to a project the first time, even for experienced developers.

I tried to be thorough. Hopefully after this, there won't be many or any more changes.

[henrikingo]

I suggest removing the first sentence. It sound like it is describing the intrenals of how our code works. But this is a user manual, they don't need to understand that. The second sentence can stay though.

[henrikingo]

=> So really you are saying Required: no

[henrikingo]

Suggest to change:

"Analyze test results stored in JSON format."

[henrikingo]

This needs updating. The user should be able to try the provided example and it should work.

[henrikingo]

You'd better add more than one data point if you want to demo change point detection :-)

[henrikingo]

Two points also is not enough to trigger a change point...

Perhaps you should use the same data that is in the csv example (local_sample.csv). Just convert that to json.

[Phanindra899]

HI @henrikingo ,Thanks for the detailed feedback! I’ve addressed all the points:
• clarified timestamp semantics
• removed internal implementation details
• updated wording (runs → results where applicable)
• simplified attributes section
• fixed config file paths
• improved sample JSON with multiple data points

Please let me know if anything else needs adjustment.

[Phanindra899]

Hi @henrikingo, I’ve updated the JSON example to include multiple realistic data points based on the CSV example, so it now supports meaningful change-point detection.

Let me know if anything else should be improved.

[Phanindra899]

Hi @henrikingo, I realized that some unrelated Sphinx documentation changes were included in this PR when merging branches.

If you’d prefer, I can separate those into a different PR and keep this one focused only on the JsonImporter documentation changes.

Please let me know how you’d like me to proceed

[henrikingo]

Sounds like you want to learn about the rebase command so you can undo that last merge com. Once you manage to do that, use git push -f to push a corrected series of commits to this same PR.

[Phanindra899]

@henrikingo — Done. I used interactive rebase to drop the unrelated Sphinx commits and force pushed the corrected branch. The PR now contains only the 6 commits related to JsonImporter documentation. Sorry for the confusion.

[henrikingo]

Sorry but now you have both the old and new sample.json in 2 different places.

[Phanindra899]

@henrikingo I removed the duplicate sample.json under examples/json/data/, and now only the canonical file under otava/examples/json/data/sample.json remains.