feat: add HTML report exporter for SuiteResult#48
Conversation
Learn moreAll Green is an AI agent that automatically: ✅ Addresses code review comments ✅ Fixes failing CI checks ✅ Resolves merge conflicts |
WalkthroughA new HTML reporting module is introduced with a Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Pre-merge checks and finishing touches❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
src/agentunit/reporting/html.py (2)
87-88: Consider adding accessibility attributes to the bar chart.The visualization bars are purely visual
<div>elements without text content or ARIA attributes, making them inaccessible to screen readers. While the text summary above provides the information, adding semantic meaning to the bars would improve accessibility.🔎 Suggested enhancement
Add ARIA attributes and/or text content to the bars:
-<div class="bar passed"></div> -<div class="bar failed"></div> +<div class="bar passed" role="img" aria-label="Passed: {passed_runs} out of {total_runs} runs">Passed: {passed_runs}</div> +<div class="bar failed" role="img" aria-label="Failed: {failed_runs} out of {total_runs} runs">Failed: {failed_runs}</div>This makes the visualization understandable to screen reader users and provides text alternatives.
34-35: Add viewport meta tag for better mobile rendering.Including a viewport meta tag would improve the report's display on mobile devices.
🔎 Suggested addition
<head> <meta charset="UTF-8"> + <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>AgentUnit Report</title>
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
src/agentunit/reporting/html.pysrc/agentunit/reporting/results.py
🧰 Additional context used
🧬 Code graph analysis (2)
src/agentunit/reporting/results.py (1)
src/agentunit/reporting/html.py (1)
render_html_report(11-102)
src/agentunit/reporting/html.py (1)
src/agentunit/reporting/results.py (2)
SuiteResult(72-141)success_rate(42-45)
🔇 Additional comments (3)
src/agentunit/reporting/results.py (1)
12-13: LGTM! Clean implementation following existing patterns.The import and
to_html()method follow the established pattern of other exporters in this file. The implementation is consistent withto_json(),to_markdown(), andto_junit()methods: creates parent directories, generates content via dedicated renderer, writes UTF-8 encoded output, and returns the Path.Also applies to: 133-141
src/agentunit/reporting/html.py (2)
1-8: LGTM! Proper handling of imports.The use of
TYPE_CHECKINGcorrectly avoids circular imports while maintaining type safety, and importinghtml.escapedemonstrates good security practices for rendering user-controlled content.
11-30: LGTM! Solid computation and escaping.The metrics computation is accurate, and the use of
html.escape()for scenario names properly prevents XSS vulnerabilities. The row generation logic correctly formats all data for HTML output.
| .passed {{ | ||
| background-color: #4CAF50; | ||
| width: {passed_runs * 20}px; | ||
| }} | ||
| .failed {{ | ||
| background-color: #F44336; | ||
| width: {failed_runs * 20}px; |
There was a problem hiding this comment.
Bar chart width calculation creates UX issues with larger test suites.
The fixed-pixel width calculation (passed_runs * 20px) causes horizontal scrolling when test suites have many runs. For example, a suite with 100 test runs produces 2000px-wide bars, making the report difficult to view. Consider using percentage-based widths or a capped maximum width with proportional scaling.
🔎 Proposed fix using percentage-based widths
Modify the CSS and HTML to use a container with proportional bars:
.bar {{
- height: 18px;
+ display: inline-block;
+ height: 20px;
margin: 6px 0;
+ color: white;
+ padding: 2px 8px;
+ font-size: 0.85em;
}}
+ .bar-container {{
+ display: flex;
+ width: 100%;
+ max-width: 600px;
+ margin: 12px 0;
+ }}
.passed {{
background-color: #4CAF50;
- width: {passed_runs * 20}px;
}}
.failed {{
background-color: #F44336;
- width: {failed_runs * 20}px;
}}And update the HTML section:
<h2>Summary</h2>
<p>Passed: {passed_runs} · Failed: {failed_runs}</p>
-<div class="bar passed"></div>
-<div class="bar failed"></div>
+<div class="bar-container">
+ <div class="bar passed" style="flex: {passed_runs}">Passed: {passed_runs}</div>
+ <div class="bar failed" style="flex: {failed_runs}">Failed: {failed_runs}</div>
+</div>This approach uses flexbox to proportionally distribute space and includes text labels within the bars.
Also applies to: 87-88
There was a problem hiding this comment.
Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!
|
Hi @aviralgarg05 , i have created this please review it and merge it or recommend changes if required . I am a participating in kwoc so please add it as a kwoc contributer. |
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
hi @aviralgarg05 thanks for merging this PR , but my KWOC student profile still says 0 commits and 0 PR , what can be done about this to solve this issue as 31 dec is the time for mid evaluation |
I think that would get shown in a few minutes, because I had already merged pr's for kwoc for this project and it shows for them |
3 participants
Description
This PR introduces a self-contained HTML report exporter for SuiteResult, allowing users to generate a browser-viewable report of AgentUnit runs.
The HTML report complements existing JSON, Markdown, and JUnit exporters and provides a human-friendly summary of results, including scenario success rates and overall execution details.
Fixes #29
Type of Change
Please delete options that are not relevant:
Changes Made
Added an HTML exporter under agentunit.reporting
Introduced SuiteResult.to_html(path) API for generating reports
Generated self-contained HTML with:
Execution summary (duration, total runs)
Scenario-wise success rates
Inline visual pass/fail chart (no external dependencies)
Ensured implementation follows existing exporter patterns and project style guidelines
Testing
Please describe the tests you ran to verify your changes:
pytest)Test Configuration
Test Results
Code Quality
pre-commit run --all-filesand addressed any issuesDocumentation
Breaking Changes
If this PR introduces breaking changes, please describe:
Dependencies
Performance Impact
Describe any performance implications:
Screenshots/Recordings (if applicable)
N/A (HTML output can be generated locally via SuiteResult.to_html("report.html")
Additional Context
Add any other context about the PR here:
This feature aligns with existing exporter patterns (to_json, to_markdown, to_junit)
The HTML output is fully offline and dependency-free
Future enhancements could include richer charts or per-run details
Checklist
Reviewer Notes
Please pay special attention to:
Post-Merge Tasks
Tasks to complete after merging (if any):
OPEN SOURCE PROGRAM KWOC
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.