Add all microservices

[joshmoore]

Resurrect #297

---

• omero-ms-thumbnail 0.5.2
• omero-ms-imageregion 0.5.2 (https://github.com/manics/docker-example-omero-microservices/blob/1549d55923637bb34e3f047f18b17fe5233caaa8/imageregion/Dockerfile, manually built and pushed to Docker Hub)
• omero-ms-zarr 0.1.6

The front-end Nginx configuration is in a separate commits in case we want to split this PR and deploy the microservices internally without exposing them.

omero-ms-imageregion and omero-ms-zarr are currently broken for anything that requires the IDR fork of BioFormats.

BioFormatsCache is read-only inside the docker images, so if you open a large image it will probably time out. Update: presumably the version of BioFormats in the micro-services will use a different memo version anyway, so even if it was writeable it'd delete the old cache?

[sbesson]

Briefly tested on a deployment including this configuration. Confirmation that all studies requiring the IDR-specific Bio-Formats changes are neither loading in the viewer (via omero-ms-region) nor loading via omero-ms-zarr. Additionally for those loading

2021-05-08 06:25:55,588 [render-image-region-pool-10] ERROR ome.io.nio.PixelsService - Error instantiating pixel buffer: /data/OMERO/ManagedRepository/demo_2/2017-03/06/14-24-32.180/data/idr-metadata/idr0033-rohban-pathways/screens/41744.screen
ome.conditions.ResourceError: FormatException: /data/OMERO/ManagedRepository/demo_2/2017-03/06/14-24-32.180/data/idr-metadata/idr0033-rohban-pathways/screens/41744.screen
Unknown file format: /data/OMERO/ManagedRepository/demo_2/2017-03/06/14-24-32.180/data/idr-metadata/idr0033-rohban-pathways/screens/41744.screen
	at ome.io.bioformats.BfPixelBuffer.reader(BfPixelBuffer.java:76)

For the other ones, it looks like the cache regeneration is not happening at all (due to the read-only permissions likely) and the initialization is happening for every call.

The thumbnails micro-service is loading the thumbs as expected and from a quick audit, the performance is at least as good as the one in production IDR. From my side, this is the micro-service which is probably the closest to deploy in production.

For the two other ones, the two first outstanding questions are:

• will we need to start maintaining some IDR fork or is there some mechanism to share/copy the JARs from the server
• how to allow the micro-service to read from the Bio-Formats cache directory?

[joshmoore]

• will we need to start maintaining some IDR fork or is there some mechanism to share/copy the JARs from the server

Sounds like to match our process, we'd need ansible tasks that apply our jars to the existing zips and repackages, no?

• how to allow the micro-service to read from the Bio-Formats cache directory?

From omero-ms-image-region's beanRefContext.xml:

  <bean name="imageRegionPixelsService" class="com.glencoesoftware.omero.ms.image.region.PixelsService"
        parent="filesystem">
    <!-- index=0 "path" comes from parent -->
    <constructor-arg index="1" ref="MemoizerWait"/>
    <constructor-arg ref="omeroFilePathResolver"/>
    <constructor-arg ref="backOff"/>
    <constructor-arg ref="tileSizes"/>
    <constructor-arg><null /></constructor-arg>
    <property name="metrics" ref="metrics"/>
  </bean>

for

    public PixelsService(
            String path, long memoizerWait, FilePathResolver resolver,
            BackOff backOff, TileSizes sizes, IQuery iQuery) {
        super(
            path, true, new File(new File(path), "BioFormatsCache"),
            memoizerWait, resolver, backOff, sizes, iQuery);
        log.info("Using image region PixelsService");
    }

which doesn't pass omero.pixeldata.memoizer.dir like here:

src/main/resources/ome/services/service-ome.io.nio.PixelsService.xml:    <constructor-arg value="${omero.pixeldata.memoizer.dir}"/>

So I think we'll need to either create a symlink (?!) or open a PR against the repo to take omero.pixeldata.memoizer.dir

[joshmoore]

See glencoesoftware/omero-ms-image-region#75

[will-moore]

Presumably this needs to be updated to use https://github.com/ome/docker-example-omero-microservices instead of Simon's (non-existent one).

[will-moore]

Actually, that doesn't look like a production ready docker, not updated for 3 or 4 years etc. What's the equivalent we should use here (or do we need to create one)?

[will-moore]

What's the role of omero-ms-zarr here? It is required for the functionality of the other microservices? It provides zarr API endpoints which aren't available in the current IDR deployment.

[joshmoore]

It's no longer needed. In fact, I'd assume we should strip this down to only the endpoints we need if we're going to test it.

[will-moore]

I think the only one we need is imageregion. I don't think we need thumbnails at this point (could be added later if desired).

[will-moore]

Latest release of omero-ms-thumbnail is 0.5.7 https://hub.docker.com/r/openmicroscopy/omero-ms-thumbnail/tags

[will-moore]

Latest release is v0.8.5 - see https://github.com/glencoesoftware/omero-ms-image-region/blame/v0.8.5/src/dist/conf/config.yaml
But I'm not sure where the version of omero-ms-image-region is specified?

[pwalczysko]

This branch has conflicts.

[sbesson]

Closing as this is not targeted for prod122 and created confusion