Adding Bar Graph Visualization for games/applications by danielleodigie · Pull Request #24 · Qiskit/red-queen

danielleodigie · 2022-07-19T14:28:57Z

This pull request adds bar graph visualization for the applications the QxQ interns are working on. I added to the README, but to run it, after getting your results JSON, run:

python visualization/view.py <JSON FILE> <DATA TYPE>

For example:

python visualization/view.py results/0001_bench.json fidelity

Currently, I only have fidelity and meantime as available data types, but I'm open to adding more, I just wasn't sure which ones would be useful. If you have any suggestions, it would help a ton! :)

Also, I noticed it didn't work as well with the mapping benchmarks, so if you can help with that, that would be awesome

danielleodigie · 2022-07-19T14:32:54Z

This is what I get when I run the results from run_bv.py with fidelity:

Yelissal

I ran your code on my computer and everything looks great! I checked your fidelities and they are within a similar range of the paper. Good job!

Lementknight

The benchmark name for benchmark fidelity breaks down when you have multiple json with data from multiple benchmarks. Could you rearrange the table to be able to deal with this?

Lementknight · 2022-08-02T20:29:02Z

This is the data from the json I used
data_table.txt

Lementknight · 2022-08-03T15:05:17Z

I used the visualization code and it no longer works. I noticed that you said that you added dual file comparison, but that wasn't the issue that I was experiencing. The issue that I had was with visualization of multiple fidelities in one json file. When you run multiple benchmark together in one call:
pytest red_queen/games/applications/grovers.py red_queen/games/applications/run_bv.py red_queen/games/applications/run_ft.py --store
The json file has the fidelities of each individual benchmark, so the code you had prior to the adding dual file comparison commit, struggled to visual multiple benchmark names.

Your code was perfectly fine, it just needed to be slightly modified to deal with multiple concurrent benchmark names in the fidelity. I hope that this comment offers clarity to my issue.

danielleodigie · 2022-08-03T15:09:42Z

I used the visualization code and it no longer works. I noticed that you said that you added dual file comparison, but that wasn't the issue that I was experiencing. The issue that I had was with visualization of multiple fidelities in one json file. When you run multiple benchmark together in one call: pytest red_queen/games/applications/grovers.py red_queen/games/applications/run_bv.py red_queen/games/applications/run_ft.py --store The json file has the fidelities of each individual benchmark, so the code you had prior to the adding dual file comparison commit, struggled to visual multiple benchmark names.

Your code was perfectly fine, it just needed to be slightly modified to deal with multiple concurrent benchmark names in the fidelity. I hope that this comment offers clarity to my issue.

ahh, okay I didn't know you could run multiple benchmarks at once, I didn't write this with that in mind, and your problem is that the fidelity breaks down when you have multiple different benchmarks in one json file?

Lementknight · 2022-08-03T15:52:14Z

It's more so that the names of the benchmarks aren't displayed in a legible manor

I believe that this could be fixed by redesigning the graph structure, since the data displayed in the visualization is great.

danielleodigie · 2022-08-03T16:10:04Z

It's more so that the names of the benchmarks aren't displayed in a legible manor

I believe that this could be fixed by redesigning the graph structure, since the data displayed in the visualization is great.

oh it's just how it looks, I rotated the names as such, to see the bottom you may have to go into the subplot config thingy but that's alright

mtreinish

I've just put a few comments inline. You'll need to git mv the file in to the red_queen directory. The reason lint is passing right now is that it isn't being run on the file because it's outside the red queen package.

mtreinish · 2022-08-05T19:58:55Z

@@ -0,0 +1,153 @@
+import sys
+import json
+from turtle import color


Is turtle in the requirements list? I think this will fail unless people manually install turtle so we should add it to the requirements list https://github.com/Qiskit/red-queen/blob/main/requirements.txt so that it gets installed with red queen. That being said I don;t think anything is actually using this.

mtreinish · 2022-08-05T19:59:37Z

+            print("Invalid Data Type! Allowed parameters are: 'fidelity' and 'meantime'")
+if __name__ == "__main__":
+    main(sys.argv)
+
+
+
+


Suggested change

print("Invalid Data Type! Allowed parameters are: 'fidelity' and 'meantime'")

if __name__ == "__main__":

main(sys.argv)

print("Invalid Data Type! Allowed parameters are: 'fidelity' and 'meantime'")

if __name__ == "__main__":

main(sys.argv)

mtreinish · 2022-08-05T20:02:02Z

+        try:
+            bench_name = {}
+            bench_fid= {}
+            for i,bench in enumerate(contents["benchmarks"]):
+                bench_name[bench["name"]] = i + 1
+                if bench_fid.get(bench["name"]):
+                    bench_fid[bench["name"]] = bench_fid.get(bench["name"]) + bench["stats"]["quality"]["fidelity"]
+                else:
+                    bench_fid[bench["name"]] = bench["stats"]["quality"]["fidelity"]
+            fidavg = []
+            for i in bench_fid:
+                fidavg.append(bench_fid[i]/bench_name[i])
+            return bench_name.keys(), fidavg
+        except Exception as e:
+            print("Uh, Oh! Something went wrong")
+            print(e)


Is there a reason to just not have the exception here raise?

mtreinish · 2022-08-05T20:02:40Z

+    if len(args) == 4:
+        viewtype = args[1]
+        file = args[2]
+        datatype = args[3]
+    if len(args) == 5:
+        viewtype = args[1]
+        file = args[2]
+        file2 = args[3]
+        datatype = args[4]


Can you leverage argparse for this so we have a --help to explain the options to users?

If need be, I could help you with this @danielleodigie.

danielleodigie added 6 commits July 11, 2022 13:02

adding dj

c5ceb54

fixing deustch jozsa

ac22583

fixing lint stuff

b296c82

fixing lint issues again

1d7acae

adding bar graph visualization

07e7a9a

fixing files

a924195

Yelissal approved these changes Jul 25, 2022

View reviewed changes

Merge branch 'Qiskit:main' into visualization

5621b4b

Lementknight suggested changes Aug 2, 2022

View reviewed changes

danielleodigie added 2 commits August 3, 2022 09:33

adding dual file comparison

6f0fb64

adding doc

e1d38ed

fixing labels

1baee45

mtreinish reviewed Aug 5, 2022

View reviewed changes

Conversation

danielleodigie commented Jul 19, 2022

Uh oh!

danielleodigie commented Jul 19, 2022

Uh oh!

Yelissal left a comment

Choose a reason for hiding this comment

Uh oh!

Lementknight left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Lementknight commented Aug 2, 2022

Uh oh!

Lementknight commented Aug 3, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danielleodigie commented Aug 3, 2022

Uh oh!

Lementknight commented Aug 3, 2022

Uh oh!

danielleodigie commented Aug 3, 2022

Uh oh!

mtreinish left a comment

Choose a reason for hiding this comment

Uh oh!

mtreinish Aug 5, 2022

Choose a reason for hiding this comment

Uh oh!

mtreinish Aug 5, 2022

Choose a reason for hiding this comment

Uh oh!

mtreinish Aug 5, 2022

Choose a reason for hiding this comment

Uh oh!

mtreinish Aug 5, 2022

Choose a reason for hiding this comment

Uh oh!

Lementknight Aug 6, 2022

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Lementknight left a comment •

edited

Loading

Lementknight commented Aug 3, 2022 •

edited

Loading