Autogenerate openapi.yaml.j2 from model code

[nmaludy]

Currently st2common/st2common/openapi.yaml.j2 needs to be updated manually when a change is made to the API. This makes the update process error prone and potentially the openapi schema may not match the actual API implementation (see: #3522, #3783).

I'm proposing to autogenerate the openapi.yaml.j2 from source.

It would actually be great if the CLI, API and API documentation all were generated from a single "source of truth" to avoid errors and inconsistencies in the future.

[Kami]

I agree, openapi entity definitions should be generated from actual API models.

We had a huge discussion about that in the past - I was always against current approach (copy and paste) since it's very error prone and hard to maintain (even now we are missing definitions for a bunch of models inside openapi file, some are incomplete / out of date, etc.).

Initially, some people were for dropping API models all together, but I think that's a bad idea because we would lose a lot of "static like typing" functionality which we get for free with linters, etc. once we have model class definitions. And in Python in general, working with dicts instead of models (class instances) with static / fixed list of attributes makes it very painful and error prone for the developer.

Also, I believe that's a duplicate of #3575. Please confirm, so I can close and we can continue there if that's indeed the case.

[nmaludy]

@Kami Yes this is a duplicate of #3575 except i'm also proposing auto-generating the CLI based on the API definitions. This could be considered an extra feature, however i think it would fall under the same "update in one place" umbrella.

[emptywee]

I second that. Moreover, we need to allow end users override some defaults, e.g. number of items fetched by st2 client from the datastore and so forth.

[Kami]

I would consider that as a separate issue / task since it's quite a big one. I also assume that with CLI you mean http client / API SDK.

But yeah, long term, low-level api client should be auto-generated for sure. I think "low-level" part is important. Auto-generated stuff usually only provides basic low level client functionality (simple 1 to 1 mapping to each API endpoint) and if you want to make it more language specific and user-friendly you need to add some wrappers around low-level auto-generated client.

[nmaludy]

By CLI i was referring to the st2 command that essentially is a command line wrapper for the API.

[Kami]

We could try that, at the very least we should re-use low level auto-generated API client library, but for some other things it might be harder since things can't be simply auto-generated (some commands can, but many can't because they aren't simple wrappers around API operations).

Having said that, in terms of priority (what we work on first), it should probably go something like that:

1. Definitions are generated from API models or vice versa
2. (Python) API client library
3. CLI - since it actually depends on 2)