ST2 API Exiting on Start

[bigmstone]

ST2 API is failing to start. I've logged into the container and retrieved the following traceback while executing st2api directly.

2018-03-12 15:40:31,628 INFO [-] Creating st2api: StackStorm v2.5.1 as OpenAPI app.
2018-03-12 15:40:32,228 INFO [-] Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
2018-03-12 15:40:32,247 INFO [-] Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
2018-03-12 15:40:32,355 WARNING [-] You must either provide a valid token or a password (api_key) and a user.
2018-03-12 15:40:32,499 INFO [-] Connected to amqp://admin:**@rabbitmq:5672//
2018-03-12 15:40:32,501 INFO [-] Started timer core.9c8418b4-7cc9-47c4-b20f-35593f36bc8c with parameters {u'unit': u'seconds', u'delta': 5}
2018-03-12 15:40:32,505 ERROR [-] (PID=2437) ST2 API quit due to exception.
Traceback (most recent call last):
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2api/cmd/api.py", line 74, in main
    return _run_server()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2api/cmd/api.py", line 63, in _run_server
    wsgi.server(sock, app.setup_app(), custom_pool=worker_pool, log=LOG, log_output=False)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2api/app.py", line 67, in setup_app
    router.add_spec(spec, transforms=transforms)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/router.py", line 186, in add_spec
    __import__(module_name)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2api/controllers/v1/keyvalue.py", line 306, in <module>
    key_value_pair_controller = KeyValuePairController()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2api/controllers/v1/keyvalue.py", line 62, in __init__
    self._coordinator = coordination.get_coordinator()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/services/coordination.py", line 177, in get_coordinator
    COORDINATOR = coordinator_setup()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/services/coordination.py", line 161, in coordinator_setup
    coordinator.start()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/tooz/coordination.py", line 687, in start
    super(CoordinationDriverWithExecutor, self).start(start_heart)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/tooz/coordination.py", line 423, in start
    self._start()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/tooz/drivers/redis.py", line 447, in _start
    self._server_info = self._client.info()
  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/tooz/drivers/redis.py", line 55, in _translate_failures
    cause=e)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/tooz/utils.py", line 225, in raise_with_cause
    excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 143, in raise_with_cause
    six.raise_from(exc_cls(message, *args, **kwargs), kwargs.get('cause'))
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/six.py", line 718, in raise_from
    raise value
ToozError: NOAUTH Authentication required.

[bigmstone]

The core of the issue is sourced from #80
This introduced fixes to redis configuration as well as introducing password for it. The problem is that now the latest container does not contain these fixes and the old entrypoint does not properly apply the : to the beginning of the redis config. This results in st2api crashing on start.

The fix is spinning new container and uploading to the registry.

Additionally I have two items:

1. CI should have caught this.
2. We should consider adding CD to this so that we never have those images out of sync again.

[warrenvw]

I've discovered an issue with the scripts executed during st2-docker CI that affects latest builds. I'll submit a PR shortly.