Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 29 additions & 27 deletions fixed-requirements.txt
Original file line number Diff line number Diff line change
@@ -1,40 +1,42 @@
# Packages versions fixed for the whole st2 stack
# Note: greenlet is used by eventlet
greenlet>=0.4.10,<0.5
eventlet>=0.18.4,<0.19
gunicorn==19.6.0
kombu==3.0.37
greenlet==0.4.12
# Note: 0.20.0 removed select.poll() on which some of our code and libraries we
# depend on rely
eventlet==0.19.0
gunicorn==19.7.1
kombu==4.0.2
# Note: amqp is used by kombu
amqp==1.4.9
amqp==2.2.1
oslo.config>=1.12.1,<1.13
oslo.utils<3.1.0
six==1.10.0
pyyaml>=3.12,<4.0
requests[security]>=2.14.1,<2.15
requests[security]>=2.18.1,<2.19
apscheduler==3.3.1
gitpython==2.1.3
jsonschema>=2.5.0,<2.6
mongoengine==0.11.0
gitpython==2.1.5
jsonschema==2.6.0
mongoengine==0.13.0
Copy link
Member

@arm4b arm4b Jun 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the same change which we reverted before: #3402
It caused the regression for Xenial when 'st2ctl register' failed to register rules on a fresh DB.

If you could investigate and fix the root cause of StackStorm/st2-packages#445 without adding timeouts, - I'm good with that.
Otherwise let's please avoid the same mongoengine update we reverted before.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, plan is to merge it once more testing is done and that issue is resolved.

Copy link
Member Author

@Kami Kami Jun 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this change should do it - 60a9dde.

As mentioned in that ticket and discussed on Slack, it seems like there is some weird race and internal trigger types are not registered yet when the script runs.

Registering internal triggers is an idempotent operation we do at every service setup / init phase so doing it again here should have no negative consequences (perhaps just a small performance overhead, but should be fine).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, as we just discussed offline, you are correct, this script shouldn't rely on other services to start first and with this change this is now indeed the case.

In the past we didn't do that, because registering internal trigger types talks to the database and we thought register-content will also be able to run on other servers without st2 and talk to st2 over API, but that's not the case and other functionality of the script also requires to be run from the server where st2 components are running.

And if we ever move it to the API, that's fine as well because then script won't require DB access anymore.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We believe #3542 should resolve the actual underlying race issue so we should be good, but we will still of course do more testing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

pymongo==3.4.0
passlib==1.6.5
lockfile>=0.10.2,<0.11
python-gnupg==0.3.9
jsonpath-rw>=1.3.0
pyinotify>=0.9.5,<=0.10
semver==2.7.2
stevedore>=1.7.0,<1.8
paramiko>=2.1.2,<2.2
networkx==1.10
passlib==1.7.1
lockfile==0.12.2
python-gnupg==0.4.1
jsonpath-rw==1.4.0
pyinotify==0.9.6
semver==2.7.7
stevedore==1.22.0
paramiko==2.2.1
networkx==1.11
python-keyczar==0.716
retrying>=1.3,<1.4
retrying==1.3.3
# Note: We use latest version of virtualenv which uses pip 9.0
virtualenv==15.1.0
sseclient==0.0.12
python-editor==1.0.1
prompt-toolkit==1.0.7
tooz>=1.55.0,<1.56.0
sseclient==0.0.18
python-editor==1.0.3
prompt-toolkit==1.0.14
tooz==1.57.4
zake==0.2.2
routes==2.3.1
flex==6.5.0
webob==1.6.0
prance==0.5.1
routes==2.4.1
flex==6.10.0
webob==1.7.3
prance==0.6.1
50 changes: 25 additions & 25 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,50 +2,50 @@
apscheduler==3.3.1
argcomplete
bcrypt
eventlet<0.19,>=0.18.4
flex==6.5.0
eventlet==0.19.0
flex==6.10.0
git+https://github.com/Kami/entrypoints.git@dont_use_backports#egg=entrypoints
git+https://github.com/Kami/logshipper.git@stackstorm_patched#egg=logshipper
git+https://github.com/StackStorm/python-mistralclient.git#egg=python-mistralclient
git+https://github.com/StackStorm/st2-auth-backend-flat-file.git@master#egg=st2-auth-backend-flat-file
gitpython==2.1.3
gunicorn==19.6.0
gitpython==2.1.5
gunicorn==19.7.1
ipaddr
jinja2
jsonpath-rw>=1.3.0
jsonschema<2.6,>=2.5.0
kombu==3.0.37
lockfile<0.11,>=0.10.2
jsonpath-rw==1.4.0
jsonschema==2.6.0
kombu==4.0.2
lockfile==0.12.2
mock
mongoengine==0.11.0
networkx==1.10
mongoengine==0.13.0
networkx==1.11
nose
oslo.config<1.13,>=1.12.1
oslo.utils<3.1.0
paramiko<2.2,>=2.1.2
passlib==1.6.5
prance==0.5.1
paramiko==2.2.1
passlib==1.7.1
prance==0.6.1
prettytable
prompt-toolkit==1.0.7
prompt-toolkit==1.0.14
psutil
pyinotify<=0.10,>=0.9.5
pyinotify==0.9.6
pymongo==3.4.0
python-dateutil
python-editor==1.0.1
python-gnupg==0.3.9
python-editor==1.0.3
python-gnupg==0.4.1
python-json-logger
python-keyczar==0.716
pytz
pyyaml<4.0,>=3.12
rednose
requests[security]<2.15,>=2.14.1
retrying<1.4,>=1.3
routes==2.3.1
semver==2.7.2
requests[security]<2.19,>=2.18.1
retrying==1.3.3
routes==2.4.1
semver==2.7.7
six==1.10.0
sseclient==0.0.12
stevedore<1.8,>=1.7.0
tooz<1.56.0,>=1.55.0
sseclient==0.0.18
stevedore==1.22.0
tooz==1.57.4
unittest2
webob==1.6.0
webob==1.7.3
zake==0.2.2
7 changes: 6 additions & 1 deletion st2client/st2client/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@

import six
import requests
import urllib3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Kami If we revert the requests back to previous version in #3597, this part probably should be reverted as well.


from urllib3.exceptions import InsecureRequestWarning

from st2client import models
from st2client.config_parser import CLIConfigParser
Expand Down Expand Up @@ -103,7 +106,9 @@ def get_client(self, args, debug=False):
# Silence SSL warnings
silence_ssl_warnings = rc_config.get('general', {}).get('silence_ssl_warnings', False)
if silence_ssl_warnings:
requests.packages.urllib3.disable_warnings()
# pylint: disable=no-member
urllib3.disable_warnings(InsecureRequestWarning)
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

# We skip automatic authentication for some commands such as auth
try:
Expand Down
11 changes: 8 additions & 3 deletions st2common/st2common/bootstrap/actionsregistrar.py
Original file line number Diff line number Diff line change
Expand Up @@ -133,11 +133,16 @@ def _register_action(self, pack, action):
# We throw a more user-friendly exception on invalid parameter name
msg = str(e)

is_invalid_parameter_name = 'Additional properties are not allowed' in msg
is_invalid_parameter_name &= 'in schema[\'properties\'][\'parameters\']' in msg
is_invalid_parameter_name = 'does not match any of the regexes: ' in msg

if is_invalid_parameter_name:
parameter_name = re.search('\'(.+?)\' was unexpected', msg).groups()[0]
match = re.search('\'(.+?)\' does not match any of the regexes', msg)

if match:
parameter_name = match.groups()[0]
else:
parameter_name = 'unknown'

new_msg = ('Parameter name "%s" is invalid. Valid characters for parameter name '
'are [a-zA-Z0-0_].' % (parameter_name))
new_msg += '\n\n' + msg
Expand Down
4 changes: 2 additions & 2 deletions st2common/st2common/services/coordination.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,8 @@ class NoOpDriver(coordination.CoordinationDriver):
This driver is used if coordination service is not configured.
"""

def __init__(self, member_id):
super(NoOpDriver, self).__init__(member_id)
def __init__(self, member_id, parsed_url=None, options=None):
super(NoOpDriver, self).__init__(member_id, parsed_url, options)

def watch_join_group(self, group_id, callback):
self._hooks_join_group[group_id].append(callback)
Expand Down
4 changes: 3 additions & 1 deletion st2common/st2common/util/shell.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,13 @@
import os
import shlex
import signal
import subprocess
from subprocess import list2cmdline
from ctypes import cdll

import six
# NOTE: eventlet 0.19.0 removed support for sellect.poll() so we not only provide green version of
# subprocess functionality and run_command
from eventlet.green import subprocess

from st2common import log as logging

Expand Down
15 changes: 13 additions & 2 deletions st2tests/st2tests/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -436,11 +436,22 @@ def assertProcessIsRunning(self, process):
Assert that a long running process provided Process object as returned by subprocess.Popen
has succesfuly started and is running.
"""
if not process:
raise ValueError('process is None')

return_code = process.poll()

if return_code is not None:
stdout = process.stdout.read()
stderr = process.stderr.read()
if process.stdout:
stdout = process.stdout.read()
else:
stdout = ''

if process.stderr:
stderr = process.stderr.read()
else:
stderr = ''

msg = ('Process exited with code=%s.\nStdout:\n%s\n\nStderr:\n%s' %
(return_code, stdout, stderr))
self.fail(msg)
Expand Down