Api experimenters groups by will-moore · Pull Request #5315 · ome/openmicroscopy

will-moore · 2017-05-30T15:22:50Z

What this PR does

Adds initial support for /experimenters/ and /experimentergroups/ to the JSON API.

This includes updating the getQueryString() for the BlitzGateway ExperimenterWrapper and ExperimenterGroupWrapper classes, so that by default they load experimenters without groups and load groups without experimenters.

Need to use the opts to load experimenters / groups (same as for Image 'load_pixels' etc).

conn.getObjects("ExperimenterGroup", opts={load_experimenters=True})
conn.getObjects("Experimenter", opts={load_groups=True})

In several places in webadmin we are using conn.getObjects("ExperimenterGroup") to list groups (e.g. in the experimenter/new/ form) when we don't need to load_experimenters. Now that these are not being loaded, the loading time for that page is ~2 x faster for me.

I've also updated the group.copyGroupExperimenterMap() and experimenter.copyGroupExperimenterMap() to lazy load if not previously loaded.
This means that doing the following is slower because of lazy loading, but at least it won't fail

for e in conn.getObjects("Experimenter"):
    e.copyGroupExperimenterMap()
    # or this also needs to load groups
    e.isAdmin()

TODOs:

Support for single Experimenter or Group
Tests
Loading experimenterGroupMap etc?? (see below)
Query experimenters or groups by name? E.g. to look up your own experimenter ID from omeName or lastName? E.g. /experimenters/?omeName=will

Testing this PR

Browse to /api/v0/m/experimenters/ to list all.
filter with ?group=3 or with /api/v0/m/groups/3/experimenters
Browse to /api/v0/m/experimentergroups/ to list all.
filter groups with ?experimenter=3 or with /api/v0/m/experimenters/3/experimentergroups
Follow links to see single experimenter or group: /api/v0/m/experimenters/3/ or /api/v0/m/groups/3/
Need to test webclient and webadmin in places where we are loading experimenters / groups to check that we don't get unloaded exceptions:
- Main /webclient/ page - Groups/Users menu
- Search page (groups select option)
- webadmin listing and editing groups
- webadmin listing and editing experimenters
- User settings in webclient (including for group leaders)

Related reading

To discuss:

Do we expose the full name experimenterGroups anywhere (urls, filters etc) or do we simply stick with groups?
Currently omero-marshal doesn't support loading groupExperimenterMap for Experimenters OR Groups. This means you can't choose to list Groups with Experimenters loaded (or list Experimenters with Groups loaded). Also, can't see which members are Owners of a Group.

cc @chris-allan @jburel

will-moore · 2017-05-30T15:23:25Z

Closing until after #POTN and more complete...

jburel · 2017-06-21T14:11:52Z


./components/tools/OmeroWeb/omeroweb/api/urls.py:327:24: E127 continuation line over-indented for visual indent
./components/tools/OmeroWeb/omeroweb/webadmin/views.py:102:80: E501 line too long (82 > 79 characters)

mtbc · 2017-07-03T09:20:13Z

excluding to let #5349 through now #5101 is in

…groups

will-moore · 2017-08-14T22:47:12Z

@jburel Could you remove the "exclude" label from this PR? Thanks

…ps')

…ap() In PR ome#5315 this now lazy loads group experimenters if unloaded

will-moore · 2017-08-22T21:41:18Z

                if d.child.id.val != self.getUserId():
                    yield ExperimenterWrapper(self, d.child)

-    def groupSummary(self, gid=None, exclude_self=False):


Seems that webtagging is still using this.
So I will leave it until webtagging has updated.

Agreed, but removal of any method from omero.gateway should include a deprecation so if you foresee dropping it, perhaps add the deprecation now?

It has been marked as deprecated for some time already. That's why I decided to remove it.

Apologies, you're of course right. I had confused this with https://github.com/openmicroscopy/openmicroscopy/pull/5402/files#diff-5586c69038567a9c1960ae4f6d731278L509 where I convinced myself that the removal was less critical since it was in webclient

Since this is still being used by webtagging, wait till that has been updated before removing. This reverts commit b40e970.

rgozim · 2017-08-30T10:40:39Z

API's tested and working well.

Only concern, as noted by @will-moore, might be the ability of a public user being able to view experimenter and experimenter group data via the API (no authentication required to query the API).

…ters/

jburel · 2017-08-24T20:54:46Z


        Returns a tuple of (query, clauses, params).
+        Supported opts: 'group': <group_id> to filter by ExperimenterGroup
+                        'load_groups': <bool> to load ExperimenterGroups'


jburel · 2017-08-24T20:56:09Z

        group.
        Returns a tuple of (query, clauses, params).
+        Supported opts: 'experimenter': <experimenter_id> to filter by
+                                        ExperimenterGroup


filter by Experimenter

jburel · 2017-08-24T20:56:18Z

        Returns a tuple of (query, clauses, params).
+        Supported opts: 'experimenter': <experimenter_id> to filter by
+                                        ExperimenterGroup
+                        'load_experimenters': <bool> to load Experimenters'


jburel · 2017-08-30T10:53:45Z

That is a serious concern

will-moore · 2017-08-30T13:00:59Z

Hmmm - let's discuss the public user issue. If you've configured the public url regex then chances are that /api/ urls are not public unless desired. E.g. they're not public on IDR.

joshmoore · 2017-08-30T13:06:13Z

It would be useful if a API definition itself (whether this one, or one introduced by third parties) could have some "awareness" of its critical nature so that the burden isn't externalized to the sysadmin to always have a regex set correctly.

will-moore · 2017-09-01T11:42:49Z

Re: discussion of guest user.
Logging in as guest, then using the session ID to access the JSON API gives the same exception as trying to load data via the CLI:

$ bin/omero user list
Using session 2960bf65-c0b4-4748-8b47-40eac3fdc22f (guest@localhost:4064). Idle timeout: 10 min. Current group: guest 
...
  File "/Users/wmoore/Desktop/OMERO/openmicroscopy/dist/lib/python/omero_api_IAdmin_ice.py", line 864, in lookupExperimenters
    return _M_omero.api.IAdmin._op_lookupExperimenters.invoke(self, ((), _ctx))
omero.SecurityViolation: exception ::omero::SecurityViolation
message = No matching roles found in [guest] for session 2960bf65-c0b4-4748-8b47-40eac3fdc22f (allowed: [user])

also seen at:
/api/v0/m/projects/?bsession=2960bf65-c0b4-4748-8b47-40eac3fdc22f&server=1

will-moore · 2017-09-04T12:44:10Z

Need more discussion of experimenters/groups visibility for public & other users.
No need to get this into 5.4.0, closing for now...

PR ome/openmicroscopy#5315

will-moore added 9 commits May 29, 2017 22:39

Move to API_VERSION 0.2-DEV

cf5708f

Support for listing Experimenters

b35ee4e

getObjects('Experimenter') doesn't load groups by default

49774c3

Blitz getObject('Experimenter') load_groups=True by default

c3082b5

Use load_groups = False in ExperimentersView

1c4ccf9

Blitz getQueryString 'Experimenter' supports 'group' filter

b8b8fd5

Experimenters view filters by 'group_id'

aaa8d3d

Blitz ExperimenterGroup getQueryString with 'experimenter' filter

9dc231b

ExperimenterGroup view and urls

ce83def

will-moore closed this May 30, 2017

will-moore added 3 commits May 30, 2017 23:06

Add urls and ObjectView for Group & Experimenter

7074e50

Blitz fix missing opts is None check

c20646d

Add links to 'experimenters' and 'groups' to base JSON

afe9a55

will-moore reopened this Jun 21, 2017

flake8 fixes

24ad4e5

will-moore force-pushed the api_experimenters_groups branch from 6aef59b to 24ad4e5 Compare June 21, 2017 22:49

jburel added the develop label Jun 23, 2017

mtbc added the exclude label Jul 3, 2017

will-moore added 3 commits August 14, 2017 22:12

Merge remote-tracking branch 'origin/develop' into api_experimenters_…

d40edcc

…groups

Default load_groups = False in Experimenter getQueryString()

e65d3eb

Use load_experimenters: True in webclient Group/Users menu

0c9524f

jburel removed the exclude label Aug 15, 2017

will-moore added 4 commits August 15, 2017 13:00

load_experimenters = False by default in getObjects('ExperimenterGrou…

4f01489

…ps')

Use 'load_groups': True in conn.getUser()

afcba49

Test fixes with load_experimenters and load_groups True

f4e5299

Remove deprecated conn.groupSummary()

b40e970

will-moore added 2 commits August 18, 2017 13:19

Fix 'url:groups' typo

fa30166

Fix removal of urls in assert_objects() testing

821480b

will-moore added a commit to will-moore/openmicroscopy that referenced this pull request Aug 18, 2017

webclient_gateway.setOwners() uses Blitz group.copyGroupExperimenterM…

f4eb7de

…ap() In PR ome#5315 this now lazy loads group experimenters if unloaded

flake8 fix

0731d14

will-moore commented Aug 22, 2017

View reviewed changes

will-moore added 4 commits August 22, 2017 22:43

Revert "Remove deprecated conn.groupSummary()"

9455a86

Since this is still being used by webtagging, wait till that has been updated before removing. This reverts commit b40e970.

Added new test_api_experimenters_groups.py

ce32988

Add test to test_api_experimenters_groups.py

1e29543

Use 'experimentergroups' in urls

2670d5c

Fix /groups/{id}/experimenters/ to /experimentergroups/{id}/experimen…

175ba3e

…ters/

jburel reviewed Aug 30, 2017

View reviewed changes

Fix typos

47cefae

snoopycrimecop mentioned this pull request Sep 4, 2017

Preview rdef start end ints #5484

Merged

will-moore closed this Sep 4, 2017

This was referenced Sep 4, 2017

Api eventContext #5487

Closed

public url regex default #5491

Merged

will-moore deleted the api_experimenters_groups branch February 18, 2019 04:13

will-moore mentioned this pull request Apr 4, 2019

Api get roi #5960

Merged

will-moore added a commit to will-moore/openmicroscopy that referenced this pull request Jul 4, 2019

Test load_experimenters:False from PR ome#5315

49add6a

will-moore mentioned this pull request Mar 6, 2020

api experimenters groups ome/omero-py#196

Merged

will-moore added a commit to will-moore/omero-web that referenced this pull request Mar 6, 2020

Ported omero-web code from api_experimenters_groups

485de7c

PR ome/openmicroscopy#5315

will-moore mentioned this pull request Mar 6, 2020

api_experimenters_groups ome/omero-web#148

Merged

will-moore added a commit to will-moore/omero-web that referenced this pull request Mar 6, 2020

Ported omero-web code from api_experimenters_groups

1c2de93

PR ome/openmicroscopy#5315

will-moore mentioned this pull request Mar 6, 2020

Add tests for api_experimenters_groups #6224

Merged

Conversation

will-moore commented May 30, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does

Testing this PR

Related reading

Uh oh!

will-moore commented May 30, 2017

Uh oh!

jburel commented Jun 21, 2017

Uh oh!

mtbc commented Jul 3, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

will-moore commented Aug 14, 2017

Uh oh!

will-moore Aug 22, 2017

Choose a reason for hiding this comment

Uh oh!

joshmoore Aug 23, 2017

Choose a reason for hiding this comment

Uh oh!

will-moore Aug 23, 2017

Choose a reason for hiding this comment

Uh oh!

joshmoore Aug 23, 2017

Choose a reason for hiding this comment

Uh oh!

rgozim commented Aug 30, 2017

Uh oh!

jburel Aug 24, 2017

Choose a reason for hiding this comment

Uh oh!

jburel Aug 24, 2017

Choose a reason for hiding this comment

Uh oh!

jburel Aug 24, 2017

Choose a reason for hiding this comment

Uh oh!

jburel commented Aug 30, 2017

Uh oh!

will-moore commented Aug 30, 2017

Uh oh!

joshmoore commented Aug 30, 2017

Uh oh!

will-moore commented Sep 1, 2017

Uh oh!

will-moore commented Sep 4, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

will-moore commented May 30, 2017 •

edited

Loading

mtbc commented Jul 3, 2017 •

edited

Loading