Asynchronous management of events

[christian-pinto]

When the sunfish core receives an event it handles it synchronously and the event sender waits for the entire time it gets processed. Events should return immediately for the client and be processed by a thread (or similar).

[christian-pinto]

@mjaguil

[mjaguil]

I will be working on threads and thread locking

[christian-pinto]

While making events asynchronous we should also look into multi-threading in general.

The two places where multithreading is needed are:

1. The REST API interface
   • Clients use the Sunfish RESTful API, with potentially multiple clients doing it in parallel. Flask restful is already supporting multi threading at the API interface level.
2. Events handling
   • Clients/agents send an event to sunfish (POST) and are released immediately without waiting for the event to be processed.

For both the above we need to make sure that both the sunfish core library and the event interface are thread safe. This includes that the storage backends are also thread safe.

[christian-pinto]

A way of working towards thread safe-ness is to use a database for storing the redfish tree.

We should evaluate a couple of alternatives:

• Redis
• Cassandra
• ETCD
• MongoDB
• others?

[christian-pinto]

I have been evaluating the usage of python threads vs adopting the newer asyncio library. I personally prefer using asyncio because it is better integrated with python and allows us to catch exceptions from asynchronous co-routines. Unlike threads where exceptions can be caught only the a join is executed on the main thread.

Implementation with threads is already available but lacks catching the above mentioned exceptions. On the other hand switching to an asyncio based implementation would require a substantial overhaul of the entire library as most of the plugins and functionalities would have to adopt the asynchronous pattern.