How to build models, repositories and controllers dynamically at runtime

[bajtos]

This is a follow-up for the spike #4235

So far, our documentation and tooling was focused on the use case where most of application artifacts are static, defined in source files. In the Epic From model definition to REST API #2036, we are working on a more declarative approach, where we have a model source file and a configuration file, and the bootstrapper creates the repository & controller classes during the boot process.

In this story, I would like to show our users even more flexible and dynamic approach, where they have full control of all aspects at runtime - from defining the model class (e.g. based on schema discovery) to exposing the REST API (or decide not to).

Examples

See the demo application presented in #4235:

1. Accept a MySQL connection string (host+port+database+credentials) and a list of table names.
2. Connect to the specified MySQL database.
3. Discover model schemas from the given tables
4. Define LB4 models for the given tables
5. Expose these models via REST API
6. Return a list of URL paths of the newly added models (e.g. /Products, /CoffeeShops)

Acceptance criteria

A new documentation page with the following sections & content:

MUST HAVE

• Define a model class from a ModelDefinition object
• Define a repository class for the given model, how to configure dependencies (the datasource to use)
• Define a controller class for the given model & repository, how to configure dependencies (the repository to use) and how to build the REST API path (e.g. /products for Product model).
• Discover model definition from a database, build ModelDefinition object from the discovered schema in juggler/LB3 format.

The page can use code snippets from the demo controller (see #4235) to illustrate the approach in practice.

SHOULD HAVE

These items are not strictly required for the initial version. If you decide to leave them out, then create follow-up stories and/or move them to acceptance criteria of and existing story like #2740.

• How to use a custom Model base class (e.g. UserBase provided by the app) instead of Model or Entity.
• How to use a custom Repository base class (e.g. MyCustomRepository provided by the app) instead of DefaultCrudRepository. We need to explain how to create a custom repository class (I think this part belongs to Repositories part of our docs?) and how to build a function for creating named repository classes from the repository class. For the second part, we can tell users to copy the code from defineCrudRepositoryClass function. See also Docs for exposing REST API of a Model with no custom classes #2740.
• How to use a custom Controller to create a custom REST API. The current answer is to copy the code from defineCrudRestController function. See also Docs for exposing REST API of a Model with no custom classes #2740.

[dkrantsberg]

I came across the need to dynamically define controllers at runtime.
I was able to create and add a controller using this (somewhat hacky) approach:

    const defineNewController = new Function(`return class TestController {
        async create() {};
        async get() { return 'Hello World'; };
      }`);
     const testControllerClass = defineNewController();
    MetadataInspector.defineMetadata(
      OAI3Keys.CONTROLLER_SPEC_KEY.key,
      testControllerSpec,
      testControllerClass,
    );
    app.controller(testControllerClass);

Now is there a way to pass Request context (or request/response objects) to this controller? I need methods of this controller to be able to access request and response objects.

[dkrantsberg]

Actually I was able inject Request and Response object into a dynamically created controller by also using MetadataInspector.defineMetadata and programmatically adding injection metadata to the controller class. Like this:

const METHODS_KEY = MetadataAccessor.create<Injection, MethodDecorator>('inject:methods');

const injectionSpecs = getControllerInjectionSpecs(controllerClass);
MetadataInspector.defineMetadata<MetadataMap<Readonly<Injection>[]>>(
  METHODS_KEY,
  injectionSpecs,
  controllerClass,
);

function getControllerInjectionSpecs(target: Object): MetadataMap<Readonly<Injection>[]> {
  return {
    '': [
      {
        target,
        methodDescriptorOrParameterIndex: 0,
        bindingSelector: RestBindings.Http.REQUEST,
        metadata: {
          decorator: '@inject'
        }
      },
      {
        target,
        methodDescriptorOrParameterIndex: 1,
        bindingSelector: RestBindings.Http.RESPONSE,
        metadata: {
          decorator: '@inject'
        }
      }
    ]
  }
}  

[emonddr]

For someone that doesn't want to discover a datastructure, have an endpoint that allows the user to post a custom model definition, and continue from there.

Have a discover endpoint to which you send a connection url and it returns the discovered models.
Then POST these models into the endpoint above.

Some things to think about.

[bajtos]

Important: until #433 is implemented, controllers registered after app startup WILL NOT be picked up by our REST layer. I think we should include a note (a warning?) in the new documentation to explain the problem to readers.