tables data on images

[will-moore]

OMERO.tables data for Images in Wells

Previously webclient has only tried to show the OMERO.tables data for Wells (not other objects).
E.g. Browse SPW -> show Well in right panel -> expand 'Tables' panel
This was in contrast to other annotations E.g. Tags etc that could only be applied to Images in Wells.

When images were shown "out of SPW context", (e.g. result of image search) we have tried to check whether the image is in a Well, then query for parent Screen/Plate table with row that matches that Well. (see #4774)

Now, we have changed the UI for SPW, such that right panel shows Well OR Image, but not both. #4863
This allows us to only show annotations on the Well OR Image (not show Image annotations for Well or Well annotations for Image).
Therefore, this PR reverts the behaviour of #4774 but adds support for querying Image rows of tables.
When Image OR Well is shown in right panel, we query Screen and Plate parents for OMERO.tables then query OMERO.table rows that match Image-id or Well-id.

Testing this PR

1. Test Image
   • Create a csv file with an Image column that contains Image IDs (Images within a Screen & Plate). Just select a few images from Plate (not all)! E.g.

Image, Well Type, Drug, Concentration
63009, Control, DMSO, 0
63010, Treatment, Toxin, 5
63011, Treatment, Taxol, 10

- Attach this to the parent Screen
- Click the scripts button above annotations, check the checkbox for csv and run Populate_metadata script
- When done, click the Well then select Image from bottom panel.
- Image in right panel, expand the 'Tables' pane. Should see corresponding data from table.

2. Test Well

   • Create similar csv with Well column instead of Image, using A1, A2 etc instead of IDs.
   • Attach this to the Plate from above.
   • Run the populat_metadata script with this csv on Plate.
   • Browse to Wells in that plate and check 'Tables' tab for data.

3. Test wrong image ID

   • Use an invalid Image ID in the csv file with Image column
   • Attach and run the populate_metadata as before
   • Error output from script should contain:
     ERROR:omero.util.populate_metadata:Image Id: 630010 not found!

NB: Since we only query a single OMERO.tables file at a time, we ONLY show the most recent. Therefore it's not possible to have E.g. one table with Image column and one table with Well column, both on the same Plate and both being displayed on child Images & Wells.
In the test scenario above, we attach one table to Screen and one to Plate to avoid this clash.
In future, we should probably try to show data from ALL OMERO.tables on the parent.

NB: This means that search results of Images (from within Wells) won't now show OMERO.tables data that is on the parent Well. Searching for Wells themselves is on the trello card below.

NB: I haven't added Robot tests for Images because it is much harder to create a csv file with Image IDs than to create one with Well positions. Would be nice if populate_metadata supported Image Name column or Well + field?

Related reading

See http://lists.openmicroscopy.org.uk/pipermail/ome-users/2016-October/006232.html
which was added to card: https://trello.com/c/GxqDnD0P/211-spw-follow-up

cc @joshmoore @manics

[manics]

I think images are supported on metadata52: #4637

[gusferguson]

@will-moore

Tested using OMERO.insight-5.3.0-m4-872-2a4a926-ice35-b479-mac on 10.0.51.106 user-1 to attach and run scripts. Results checked in Web client.

Attached script to Screen ID 1:

Image, Well Type, Drug, Concentration
13, Control, DMSO, 0
22, Treatment, Toxin, 5
23, Treatment, Taxol, 10

• ran successfully
  • tables with appropriate data visible in Web client as expected
  • no table attachments visible in Insight

Attached script to Plate ID 1:

Well, Well Type, Drug, Concentration
D3, Control-well, DMSO-well, 0
D7, Treatment-well, Toxin-well, 5
D11, Treatment-well, Taxol-well, 10

• ran successfully
• tables with appropriate data visible in Web client as expected
• no table attachments visible in Insight

Attached script with wrong Image ID to Screen ID 1:

Image, Well Type, Drug, Concentration
1111, Control, DMSO, 0
22, Treatment, Toxin, 5
23, Treatment, Taxol, 10

Error with Image ID (expected):

ERROR:omero.util.populate_metadata:Image Id: 1111 not found!
ERROR:omero.util.populate_metadata:Missing row or column for image name population!
Traceback (most recent call last):
  File "./script", line 130, in <module>
    message = populate_metadata(client, conn, scriptParams)
  File "./script", line 86, in populate_metadata
    ctx.parse_from_handle(file_handle)
  File "/home/omero/workspace/OMERO-server/OMERO.server/lib/python/omero/util/populate_metadata.py", line 497, in parse_from_handle
    self.post_process()
  File "/home/omero/workspace/OMERO-server/OMERO.server/lib/python/omero/util/populate_metadata.py", line 595, in post_process
    image = image[image_column.values[i]]
KeyError: -1L

Behaves as expected.
Need to flag Insight to add Table panel to RHP (cc: @dominikl )

Good to merge from UI perspective.

[emilroz]

@will-moore what about one table for Screen and one table for Plate? You could have then a table with "well info" attached to Screen with e.g. protocol info, annotated controls, drug names, drug concentrations, etc; and then a table attached to Plate with Image data - analysis results, etc?

Edit: I should really formalise my question :). Can we show both tables in this case: one with well info from Screen and one with Image info from Plate (and vice versa)?

[will-moore]

@emilroz Show both tables when you're looking at an Image? And just show the Well data when looking at a Well?
That might make sense, but the scenario I've described above is really just a limited work-around to the problem of "only show data from one table". It's not very intuitive to attach some Table data to a Screen and other data to a Plate, just to get it to show up in the UI. And it still limits you to just 2 Tables (wouldn't ever show repeats of the same analysis with tables on the same parent).

However, the idea of optionally showing the annotations on parent object might be a nice solution to the issue we were discussing the other day about "showing your context" - E.g. I'm looking at images in a Dataset and I want to know what tags are on the Images AND the Dataset.

[emilroz]

@will-moore yes that's exactly what I was thinking about. Of course it doesn't have to happen in this PR.

[jburel]

@will-moore: do you have the "show annotation of parents scenario" on a card?

[will-moore]

@jburel No, not yet.

[jburel]

Could you create a card on 5.3.x?

[will-moore]

Card at https://trello.com/c/5od1gJYj/143-right-panel-parent-annotations

[jburel]

Thanks @will-moore