Skip to content

Conversation

@Kami
Copy link
Member

@Kami Kami commented Jun 30, 2017

Just some house keeping and another attempt after #3189 didn't make it.

It updates the following dependencies to the latest stable version.

General dependencies

  • paramiko
  • appscheduler
  • pyyaml
  • semver
  • retrying
  • kombu
  • amqp
  • networkx
  • jsonschema
  • sseclient
  • jsonpath-rw
  • requests
  • git-python
  • stevedore
  • passlib
  • webob
  • tooz
  • routes
  • flex
  • prance
  • greenlet
  • eventlet
  • gunicorn

The plan is to include those changes in v2.4.0 after some more testing is done.

I did go over all the changelog entries and now I'm looking into unit tests to make sure that all of them pass (there are some changes needed, but mostly because some libraries doesn't expose good enough APIs so we need to write some less than ideal and robust code to make things happen - e.g. how we throw more user-friendly exceptions for some jsonschema validation failures).

TODO

@Kami Kami added this to the 2.4.0 milestone Jun 30, 2017
mongoengine==0.11.0
gitpython==2.1.5
jsonschema==2.6.0
mongoengine==0.13.0
Copy link
Member

@arm4b arm4b Jun 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the same change which we reverted before: #3402
It caused the regression for Xenial when 'st2ctl register' failed to register rules on a fresh DB.

If you could investigate and fix the root cause of StackStorm/st2-packages#445 without adding timeouts, - I'm good with that.
Otherwise let's please avoid the same mongoengine update we reverted before.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, plan is to merge it once more testing is done and that issue is resolved.

Copy link
Member Author

@Kami Kami Jun 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this change should do it - 60a9dde.

As mentioned in that ticket and discussed on Slack, it seems like there is some weird race and internal trigger types are not registered yet when the script runs.

Registering internal triggers is an idempotent operation we do at every service setup / init phase so doing it again here should have no negative consequences (perhaps just a small performance overhead, but should be fine).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, as we just discussed offline, you are correct, this script shouldn't rely on other services to start first and with this change this is now indeed the case.

In the past we didn't do that, because registering internal trigger types talks to the database and we thought register-content will also be able to run on other servers without st2 and talk to st2 over API, but that's not the case and other functionality of the script also requires to be run from the server where st2 components are running.

And if we ever move it to the API, that's fine as well because then script won't require DB access anymore.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We believe #3542 should resolve the actual underlying race issue so we should be good, but we will still of course do more testing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@Kami Kami changed the title [WIP] Upgrade various dependencies (v2.4.0_ [WIP] Upgrade various dependencies (v2.4.0) Jun 30, 2017
@Kami
Copy link
Member Author

Kami commented Jun 30, 2017

Sadly we still can't upgrade to the latest version of eventlet.

v0.20.0 removes select.poll() which affects us. I managed to fix some of the related issue, but it still breaks packs pack tests which use gitpython so it needs more work to make it work. To clarify, it only breaks tests because we don't do monkey patch there, but the actual python runner actions work fine.

Kami added 2 commits June 30, 2017 16:35
st2-register-content script.

This should resolve the weird race issue @Arma spotted on Ubuntu 16.04.
Copy link
Member

@arm4b arm4b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the #3542 fix mongoengine upgrade looks good to me 👍

Cant say about other dependencies.

@nmaludy
Copy link
Member

nmaludy commented Jul 13, 2017

+1 interested in this based on a conversation in slack about HA and coordination backends. It appears there is a bug in the current version of tooz that prevents the postgres driver from connecting properly.

@Kami Kami changed the title [WIP] Upgrade various dependencies (v2.4.0) Upgrade various dependencies (v2.4.0) Jul 19, 2017
@Kami
Copy link
Member Author

Kami commented Jul 20, 2017

Upgrading eventlet is quite challenging because of removal of select.poll() (we can't control 3rd party libraries which we use and rely on this functionality) so I will go ahead, merge this and then working on upgrading eventlet in a separate PR / branch.

@Kami
Copy link
Member Author

Kami commented Jul 20, 2017

@nmaludy also updated tooz to latest version, those changes should be included in v2.4.0 (should be out some time in the next couple of months).

If it's a small bug fix, we can potentially also pick that change in v2.3.2.


import six
import requests
import urllib3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Kami If we revert the requests back to previous version in #3597, this part probably should be reverted as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants