Register internal trigger types in the init / setup phase of st2-register-content script #3542

Kami · 2017-06-30T15:58:51Z

The should fix the race issue described in StackStorm/st2-packages#445.

Previously we didn't register internal trigger types inside this script so if user ran this script before any of the service which register them was started (e.g. st2api, st2stream, etc.) and rules were to be registered, the script would fail with "Failed to create trigger" error.

More context and explanation in StackStorm/st2-packages#445 and #3538 (comment).

@Arma

st2-register-content script. This should resolve the weird race issue @Arma spotted on Ubuntu 16.04.

arm4b · 2017-06-30T16:33:47Z

st2common/st2common/script_setup.py

    2. Establishes DB connection
    3. Suppress DEBUG log level if --verbose flag is not used
    4. Registers RabbitMQ exchanges
+    5. Registers internal trigger types (optional, disabled by default)


Maybe it worth to register trigger types only when --register-triggers is requested from the st2-register-content script?

Move this logic here: https://github.com/StackStorm/st2/blob/master/st2common/st2common/content/bootstrap.py#L128 instead of doing that every time in setup.

arm4b · 2017-06-30T18:14:05Z

CI failure from e2e test is real:

##### st2 components status #####
st2actionrunner PID: 11517
st2actionrunner PID: 11521
st2actionrunner PID: 11524
st2actionrunner PID: 11527
st2actionrunner PID: 11530
st2actionrunner PID: 11533
st2actionrunner PID: 11536
st2actionrunner PID: 11539
st2actionrunner PID: 11542
st2actionrunner PID: 11545
st2api PID: 11550
st2stream PID: 11554
st2auth PID: 11558
st2garbagecollector PID: 11562
st2notifier PID: 11567
st2resultstracker PID: 11571
st2rulesengine PID: 11576
st2sensorcontainer PID: 11580
st2chatops is not running.
mistral-server is not running.
mistral-api is not running.
Registering content...[flags = --config-file /etc/st2/st2.conf --register-all]
2017-06-30 17:27:31,696 INFO [-] Connecting to database "st2" @ "0.0.0.0:27017" as user "stackstorm".
2017-06-30 17:27:31,899 INFO [-] Registered trigger: st2.generic.actiontrigger.
2017-06-30 17:27:31,930 INFO [-] Registered trigger: st2.generic.notifytrigger.
2017-06-30 17:27:31,946 INFO [-] Registered trigger: st2.action.file_writen.
2017-06-30 17:27:31,964 ERROR [-] Conflict while trying to save in DB.
Traceback (most recent call last):
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/persistence/base.py", line 173, in add_or_update
    model_object = cls._get_impl().add_or_update(model_object)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/models/db/__init__.py", line 314, in add_or_update
    instance.save()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/mongoengine/document.py", line 408, in save
    raise NotUniqueError(message % six.text_type(err))
NotUniqueError: Tried to save duplicate unique keys (E11000 duplicate key error collection: st2.trigger_type_d_b index: uid_1 dup key: { : "trigger_type:core:st2.key_value_pair.create" })
2017-06-30 17:27:31,966 ERROR [-] Failed registering internal trigger: {'payload_schema': {'type': 'object', 'properties': {'object': {}}}, 'description': 'Trigger encapsulating datastore item creation.', 'name': 'st2.key_value_pair.create', 'pack': 'core'}.
Traceback (most recent call last):
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/triggers.py", line 83, in register_internal_trigger_types
    trigger_definition=trigger_definition)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/triggers.py", line 44, in _register_internal_trigger_type
    if not trigger_type_db.parameters_schema:
UnboundLocalError: local variable 'trigger_type_db' referenced before assignment
Traceback (most recent call last):
  File "/usr/bin/st2-register-content", line 22, in <module>
    sys.exit(content_loader.main(sys.argv[1:]))
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/content/bootstrap.py", line 387, in main
    setup(argv)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/content/bootstrap.py", line 379, in setup
    register_internal_trigger_types=True)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/script_setup.py", line 98, in setup
    triggers.register_internal_trigger_types()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/triggers.py", line 83, in register_internal_trigger_types
    trigger_definition=trigger_definition)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/triggers.py", line 44, in _register_internal_trigger_type
    if not trigger_type_db.parameters_schema:
UnboundLocalError: local variable 'trigger_type_db' referenced before assignment
##### st2 components status #####
st2actionrunner PID: 11517
st2actionrunner PID: 11521
st2actionrunner PID: 11524
st2actionrunner PID: 11527
st2actionrunner PID: 11530
st2actionrunner PID: 11533
st2actionrunner PID: 11536
st2actionrunner PID: 11539
st2actionrunner PID: 11542
st2actionrunner PID: 11545
st2api PID: 11550
st2api PID: 11689
st2stream PID: 11554
st2stream PID: 11702
st2auth PID: 11558
st2auth PID: 11705
st2garbagecollector PID: 11562
st2notifier PID: 11567
st2resultstracker PID: 11571
st2rulesengine PID: 11576
st2sensorcontainer PID: 11580
st2chatops is not running.
mistral-server is not running.
mistral-api is not running.
############### ERROR ###############
# Failed on step - Install st2 #
#####################################

…_types

arm4b · 2017-07-01T09:16:35Z

Confirming the fix.

Before

vagrant@ubuntu14:~$ st2ctl reload --register-all
Registering content...[flags = --config-file /etc/st2/st2.conf --register-all]
2017-07-01 09:12:35,655 INFO [-] Connecting to database "st2" @ "0.0.0.0:27017" as user "stackstorm".
2017-07-01 09:12:35,764 INFO [-] =========================================================
2017-07-01 09:12:35,764 INFO [-] ############## Registering triggers #####################
2017-07-01 09:12:35,764 INFO [-] =========================================================
2017-07-01 09:12:35,819 INFO [-] Registered 0 triggers.
2017-07-01 09:12:35,819 INFO [-] =========================================================
2017-07-01 09:12:35,819 INFO [-] ############## Registering sensors ######################
2017-07-01 09:12:35,819 INFO [-] =========================================================
2017-07-01 09:12:35,846 INFO [-] Registered 1 sensors.
2017-07-01 09:12:35,846 INFO [-] =========================================================
2017-07-01 09:12:35,847 INFO [-] ############## Registering runners ######################
2017-07-01 09:12:35,847 INFO [-] =========================================================
2017-07-01 09:12:36,067 INFO [-] Registered 13 runners.
2017-07-01 09:12:36,067 INFO [-] =========================================================
2017-07-01 09:12:36,067 INFO [-] ############## Registering actions ######################
2017-07-01 09:12:36,067 INFO [-] =========================================================
2017-07-01 09:12:37,077 INFO [-] Registered 65 actions.
2017-07-01 09:12:37,077 INFO [-] =========================================================
2017-07-01 09:12:37,077 INFO [-] ############## Registering rules ########################
2017-07-01 09:12:37,077 INFO [-] =========================================================
2017-07-01 09:12:37,092 WARNING [-] Failed to register rules: Failed to register rule "/opt/stackstorm/packs/chatops/rules/notify_hubot.yaml" from pack "chatops": A simple trigger should have been created when registering triggertype. Cannot create trigger: {'type': 'core.st2.generic.notifytrigger', 'parameters': {}, 'pack': 'core'}.
Traceback (most recent call last):
  File "/usr/bin/st2-register-content", line 22, in <module>
    sys.exit(content_loader.main(sys.argv[1:]))
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/content/bootstrap.py", line 387, in main
    register_content()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/content/bootstrap.py", line 339, in register_content
    register_rules()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/content/bootstrap.py", line 248, in register_rules
    raise e
ValueError: Failed to register rule "/opt/stackstorm/packs/chatops/rules/notify_hubot.yaml" from pack "chatops": A simple trigger should have been created when registering triggertype. Cannot create trigger: {'type': 'core.st2.generic.notifytrigger', 'parameters': {}, 'pack': 'core'}.

After

vagrant@ubuntu14:~$ st2ctl reload --register-all
Registering content...[flags = --config-file /etc/st2/st2.conf --register-all]
2017-07-01 09:13:41,787 INFO [-] Connecting to database "st2" @ "0.0.0.0:27017" as user "stackstorm".
2017-07-01 09:13:41,896 INFO [-] Registered trigger: st2.generic.actiontrigger.
2017-07-01 09:13:41,911 INFO [-] Registered trigger: st2.generic.notifytrigger.
2017-07-01 09:13:41,920 INFO [-] Registered trigger: st2.action.file_writen.
2017-07-01 09:13:41,929 INFO [-] Registered trigger: st2.key_value_pair.create.
2017-07-01 09:13:41,939 INFO [-] Registered trigger: st2.key_value_pair.update.
2017-07-01 09:13:41,948 INFO [-] Registered trigger: st2.key_value_pair.value_change.
2017-07-01 09:13:41,958 INFO [-] Registered trigger: st2.key_value_pair.delete.
2017-07-01 09:13:41,969 INFO [-] Registered trigger: st2.sensor.process_spawn.
2017-07-01 09:13:41,978 INFO [-] Registered trigger: st2.sensor.process_exit.
2017-07-01 09:13:41,985 INFO [-] =========================================================
2017-07-01 09:13:41,985 INFO [-] ############## Registering triggers #####################
2017-07-01 09:13:41,986 INFO [-] =========================================================
2017-07-01 09:13:42,047 INFO [-] Registered 0 triggers.
2017-07-01 09:13:42,047 INFO [-] =========================================================
2017-07-01 09:13:42,048 INFO [-] ############## Registering sensors ######################
2017-07-01 09:13:42,048 INFO [-] =========================================================
2017-07-01 09:13:42,065 INFO [-] Registered 1 sensors.
2017-07-01 09:13:42,065 INFO [-] =========================================================
2017-07-01 09:13:42,065 INFO [-] ############## Registering runners ######################
2017-07-01 09:13:42,066 INFO [-] =========================================================
2017-07-01 09:13:42,275 INFO [-] Registered 13 runners.
2017-07-01 09:13:42,275 INFO [-] =========================================================
2017-07-01 09:13:42,275 INFO [-] ############## Registering actions ######################
2017-07-01 09:13:42,276 INFO [-] =========================================================
2017-07-01 09:13:43,260 INFO [-] Registered 65 actions.
2017-07-01 09:13:43,261 INFO [-] =========================================================
2017-07-01 09:13:43,261 INFO [-] ############## Registering rules ########################
2017-07-01 09:13:43,261 INFO [-] =========================================================
2017-07-01 09:13:43,285 INFO [-] Registered 1 rules.
2017-07-01 09:13:43,285 INFO [-] =========================================================
2017-07-01 09:13:43,285 INFO [-] ############## Registering aliases ######################
2017-07-01 09:13:43,285 INFO [-] =========================================================
2017-07-01 09:13:43,354 INFO [-] Registered 10 aliases.
2017-07-01 09:13:43,355 INFO [-] =========================================================
2017-07-01 09:13:43,355 INFO [-] ############## Registering policy types #################
2017-07-01 09:13:43,355 INFO [-] =========================================================
2017-07-01 09:13:43,372 INFO [-] Registered 3 policy types.
2017-07-01 09:13:43,372 INFO [-] =========================================================
2017-07-01 09:13:43,372 INFO [-] ############## Registering policies #####################
2017-07-01 09:13:43,373 INFO [-] =========================================================
2017-07-01 09:13:43,373 INFO [-] Registered 0 policies.
2017-07-01 09:13:43,373 INFO [-] =========================================================
2017-07-01 09:13:43,374 INFO [-] ############## Registering configs ######################
2017-07-01 09:13:43,374 INFO [-] =========================================================
2017-07-01 09:13:43,375 INFO [-] Registered 0 configs.

arm4b · 2017-07-01T09:17:21Z

Minor thing: as said before #3542 (comment), I think we should register trigger types only when --register-triggers flag is provided.

Here is an example of incosistency:

vagrant@ubuntu14:~$ st2ctl reload --register-actions

Registering content...[flags = --config-file /etc/st2/st2.conf --register-actions]
2017-07-01 09:15:12,643 INFO [-] Connecting to database "st2" @ "0.0.0.0:27017" as user "stackstorm".
2017-07-01 09:15:12,759 INFO [-] Registered trigger: st2.generic.actiontrigger.
2017-07-01 09:15:12,777 INFO [-] Registered trigger: st2.generic.notifytrigger.
2017-07-01 09:15:12,787 INFO [-] Registered trigger: st2.action.file_writen.
2017-07-01 09:15:12,798 INFO [-] Registered trigger: st2.key_value_pair.create.
2017-07-01 09:15:12,808 INFO [-] Registered trigger: st2.key_value_pair.update.
2017-07-01 09:15:12,818 INFO [-] Registered trigger: st2.key_value_pair.value_change.
2017-07-01 09:15:12,829 INFO [-] Registered trigger: st2.key_value_pair.delete.
2017-07-01 09:15:12,839 INFO [-] Registered trigger: st2.sensor.process_spawn.
2017-07-01 09:15:12,849 INFO [-] Registered trigger: st2.sensor.process_exit.
2017-07-01 09:15:12,858 INFO [-] =========================================================
2017-07-01 09:15:12,858 INFO [-] ############## Registering runners ######################
2017-07-01 09:15:12,858 INFO [-] =========================================================
2017-07-01 09:15:13,081 INFO [-] Registered 13 runners.
2017-07-01 09:15:13,081 INFO [-] =========================================================
2017-07-01 09:15:13,081 INFO [-] ############## Registering actions ######################
2017-07-01 09:15:13,082 INFO [-] =========================================================
2017-07-01 09:15:14,164 INFO [-] Registered 65 actions.

Kami · 2017-07-01T09:18:35Z

@armab We could do that - only register it when rules are to be registered, but it also doesn't hurt to do in each case.

If it's too confusing for the user, we can simply remove the log messages in this scenario (or perhaps change level to debug).

trigger registration.

to the correct place.

…thub.com:StackStorm/st2 into register_content_register_internal_trigger_types

arm4b

Thanks for digging in and finding the fix! 👍

Also, we need a CHANGELOG (bugfix) update.

Bug #445 fixed by StackStorm/st2#3542

Kami added 2 commits June 30, 2017 17:56

Also register internal trigger types in bootstrap phase of

c7a1ff6

st2-register-content script. This should resolve the weird race issue @Arma spotted on Ubuntu 16.04.

Update docstring.

2d7e8c2

arm4b reviewed Jun 30, 2017

View reviewed changes

Kami added 2 commits June 30, 2017 22:14

Correctly ignore trigger dont exist errors and add missing error.

94f3d10

Merge branch 'master' into register_content_register_internal_trigger…

9f80dae

…_types

Kami added 3 commits July 1, 2017 11:24

Change log level from info to debug when logging successful internal

f0c0f91

trigger registration.

Add internal prefix to make it more clear and more "registered" log line

4b4ea3a

to the correct place.

Merge branch 'register_content_register_internal_trigger_types' of gi…

29d1559

…thub.com:StackStorm/st2 into register_content_register_internal_trigger_types

arm4b approved these changes Jul 1, 2017

View reviewed changes

arm4b pushed a commit to StackStorm/st2-packages that referenced this pull request Jul 1, 2017

Remove sleep timeout before register content

1321cb6

Bug #445 fixed by StackStorm/st2#3542

arm4b mentioned this pull request Jul 1, 2017

Remove sleep timeout before Register content StackStorm/st2-packages#466

Merged

Add a CHANGELOG note

2cad943

arm4b added this to the 2.3.1 milestone Jul 1, 2017

arm4b merged commit 6230973 into master Jul 1, 2017

arm4b deleted the register_content_register_internal_trigger_types branch July 1, 2017 14:17

Kami mentioned this pull request Jul 1, 2017

Upgrade various dependencies (v2.4.0) #3538

Merged

1 task

Kami pushed a commit to StackStorm/st2-packages that referenced this pull request Jul 27, 2017

Remove sleep timeout before register content

fd66a08

Bug #445 fixed by StackStorm/st2#3542

arm4b mentioned this pull request Jun 28, 2018

'st2ctl register' fails to register rules on a fresh DB StackStorm/st2-packages#445

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Register internal trigger types in the init / setup phase of st2-register-content script #3542

Register internal trigger types in the init / setup phase of st2-register-content script #3542

Uh oh!

Kami commented Jun 30, 2017 •

edited

Loading

Uh oh!

arm4b Jun 30, 2017 •

edited

Loading

Uh oh!

arm4b commented Jun 30, 2017

Uh oh!

arm4b commented Jul 1, 2017 •

edited

Loading

Uh oh!

arm4b commented Jul 1, 2017

Uh oh!

Kami commented Jul 1, 2017

Uh oh!

arm4b left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Register internal trigger types in the init / setup phase of st2-register-content script #3542

Register internal trigger types in the init / setup phase of st2-register-content script #3542

Uh oh!

Conversation

Kami commented Jun 30, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

arm4b Jun 30, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arm4b commented Jun 30, 2017

Uh oh!

arm4b commented Jul 1, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Before

After

Uh oh!

arm4b commented Jul 1, 2017

Uh oh!

Kami commented Jul 1, 2017

Uh oh!

arm4b left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Kami commented Jun 30, 2017 •

edited

Loading

arm4b Jun 30, 2017 •

edited

Loading

arm4b commented Jul 1, 2017 •

edited

Loading