dev_4_4: Update upgrade check handling in OMERO.web

[chris-allan]

This pull requests updates the base socket timeout semantics of the OMERO.py upgrade check and allows for upgrade check configurability in OMERO.web. OMERO tickets addressed include:

• #12097 (https://trac.openmicroscopy.org.uk/ome/ticket/12097)

To test timeouts, you will need a merge build with the changes included. Running under bin/omero shell:

from omero.util.upgrade_check import UpgradeCheck
from datetime import datetime

uc = UpgradeCheck('omero', url='http://134.36.64.142/')
print datetime.now()
uc.run()
print datetime.now()

134.36.64.142 is a University of Dundee firewalled IP address that can be used for internal or external testing. Default timeouts should be in the neighbourhood of 3 seconds. Example output:

$ bin/omero shell
WARNING:omero.util.UpgradeCheck:UPGRADE AVAILABLE:Please upgrade to 5.0.0 See http://trac.openmicroscopy.org.uk/omero for the latest version

Python 2.7.6 (default, Feb 18 2014, 10:06:14) 
Type "copyright", "credits" or "license" for more information.

IPython 1.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: %cpaste
Pasting code; enter '--' alone on the line to stop or use Ctrl-D.
:from omero.util.upgrade_check import UpgradeCheck
:from datetime import datetime
:
:uc = UpgradeCheck('omero', url='http://134.36.64.142/')
:print datetime.now()
:uc.run()
:print datetime.now()
:<EOF>
2014-03-18 13:27:49.661802
ERROR:omero.util.UpgradeCheck:<urlopen error timed out>
2014-03-18 13:27:52.663269

If needed the timeout can be adjusted and checked (example is a timeout of 10 seconds) by using:

...
uc = UpgradeCheck('omero', timeout=10.0, url='http://134.36.64.142/')
...

OMERO.web upgrade check configurability can also be tested with a merge build. Running under DJANGO_SETTINGS_MODULE=settings PYTHONPATH=$PYTHONPATH:lib/python/omeroweb bin/omero shell:

from omeroweb.webadmin.webadmin_utils import upgradeCheck
from datetime import datetime

print datetime.now()
upgradeCheck()
print datetime.now()

Disabled upgrade checks should return instantly. Example output:

$ bin/omero config set omero.upgrades.url ''
$ bin/omero config get
omero.upgrades.url=
$ DJANGO_SETTINGS_MODULE=settings PYTHONPATH=$PYTHONPATH:lib/python/omeroweb bin/omero shell
WARNING:omero.util.UpgradeCheck:UPGRADE AVAILABLE:Please upgrade to 5.0.0 See http://trac.openmicroscopy.org.uk/omero for the latest version

Python 2.7.6 (default, Feb 18 2014, 10:06:14) 
Type "copyright", "credits" or "license" for more information.

IPython 1.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: %cpaste
Pasting code; enter '--' alone on the line to stop or use Ctrl-D.
:from omeroweb.webadmin.webadmin_utils import upgradeCheck
:from datetime import datetime
:
:print datetime.now()
:upgradeCheck()
:print datetime.now()
:<EOF>
2014-03-18 13:35:25.614548
2014-03-18 13:35:25.622826

The source branch for this pull request has been based atop the v.4.4.0 tag so that it can be merged cleanly into:

• dev_4_4 (dev_4_4: Update upgrade check handling in OMERO.web #2175)
• dev_5_0 (dev_5_0: Update upgrade check handling in OMERO.web #2176)
• develop (develop: Update upgrade check handling in OMERO.web #2177)

---

--rebased-from #2176

[joshmoore]

After a first glance, these things occur to me, but can be in follow-up PRs:

• Modify the insight/server timeouts to match?
• Remove the "MilestoneDownloads" URL from webadmin?
• Any way to get double usage of omero.upgrades.url=http://upgrade.openmicroscopy.org.uk in etc/omero.properties?

[chris-allan]

I have no strong opinion on any specific timeout so if we have an OMERO.insight timeout we'd like to mirror that's trivial.

I had the same thoughts about addressing the milestone download URLs. This branch will start to fall apart really quickly if we start making string modifications unfortunately. There have been several commits fiddling with them. Right now this merges cleanly across all three branches and I'd be really loathed to loose that.

Double usage in what sense?

[chris-allan]

If I understand what you're asking about with respect to double usage this certainly works @joshmoore:

diff --git a/components/tools/OmeroWeb/omeroweb/settings.py b/components/tools/OmeroWeb/omeroweb/settings.py
index 3005b73..5f6bdb7 100755
--- a/components/tools/OmeroWeb/omeroweb/settings.py
+++ b/components/tools/OmeroWeb/omeroweb/settings.py
@@ -227,7 +227,7 @@ CUSTOM_SETTINGS_MAPPINGS = {
     "omero.web.webgateway_cache": ["WEBGATEWAY_CACHE", None, leave_none_unset],
     "omero.web.session_engine": ["SESSION_ENGINE", DEFAULT_SESSION_ENGINE, check_session_engine],
     "omero.web.debug": ["DEBUG", "false", parse_boolean],
-    "omero.web.upgrades_url": ["UPGRADES_URL", "http://upgrade.openmicroscopy.org.uk/", str],
+    "omero.upgrades.url": ["UPGRADES_URL", "http://upgrade.openmicroscopy.org.uk/", str],
     "omero.web.email_host": ["EMAIL_HOST", None, identity],
     "omero.web.email_host_password": ["EMAIL_HOST_PASSWORD", None, identity],
     "omero.web.email_host_user": ["EMAIL_HOST_USER", None, identity],

Reading the default out of etc/omero.properties is of course a non-starter as we all have experienced. The above does nothing for OMERO.py or OMERO.cli but will it work for the server?

[joshmoore]

@chris-allan: 4858115 is exactly what I meant. Don't mind the hard-coding at all, but want to ease the burden on users if they've already set omero.upgrades.url. In any event, seems that everyone is happy with this as well as gh-2176 and gh-2177 so merging all (especially for release of 5.0.1)

[joshmoore]

As an aside, the multiple merge of a single branch from v.4.4.0 into dev_4_4 and dev_5_0 has changed the value of git merge-base origin/dev_4_0 origin/dev_5_0 and so has broken http://ci.openmicroscopy.org/job/SCC-check-prs/ -- Currently looking into possible solutions with @sbesson. Sorry for not having noticed this sooner, but since we're in a rebase-workflow not a merge one, this might need to be force-pushed away and/or avoided in the future.

[joshmoore]

ome/scc#158 points to perhaps this usage just makes merge-base unreliable across the board and we'll have to move to something more sophisticated. See some of the related logic here: http://stackoverflow.com/questions/1527234/finding-a-branch-point-with-git