Epic: Validation at Model/ORM level

[David-Mulder]

Feel a bit guilty about asking this like this, but considering this is currently "released", how are we supposed to do validation?

As far as I can tell the model doesn't support any validation (per #1624) and frankly I am at a loss at how to use this without validation? Of course you could manually do the validation in the controller, but that destroys the point of an ORM 😅 . What am I missing?

Or is this not yet supposed to be used for CRUD API's?

[bajtos]

@David-Mulder could you be more specific please? What kind of validations do you have in mind?

I acknowledge that the current set of validations offered by the data-access layer of LB4 is way too small. In LoopBack 3.x, it was possible to use one of the few built-in validation rules like "validatesPresenceOf", "validatesLengthOf", etc. In our experience, these rules were not sufficient enough and were difficult to use & extend anyways.

Let's discuss the use cases and requirements for a better validation in LB4.

[bajtos]

(FWIW, the discussion in #1624 has some interesting bits to be carried over to this epic.)

[bajtos]

I was thinking about ORM-level validations in the past few days and want to post few ideas and opinions.

• I feel it's critically important do design these validations in such way that extensions can easily contribute custom validations.
• I think it's also very important to make validations robust and free of race conditions. For certain validations like uniqueness constraint and foreign keys, only the database can enforce them in a reliable way and LB4 should make it possible to write such validations.
• The design of our validation framework should allow us to perform partial validations too. For example, when PATCH /customers/123 wants to change the phone field only, then we should be able to run only validations for the phone property and skip other validations (e.g. email is required and must be a valid email address).

I am thinking about the following high-level design:

1. A model-level mixin to add validation-related APIs:

   • Register a validation step. Few examples of a validation steps: check that property email is a valid email address; check that endDate is always greater than startDate.
   • Run all validation steps. (Optionally, perform a partial validation.)
   • Describe the validation steps as constraints that connectors can use to define database schema. For example, a string property with the constraint maxLength: 50 can be mapped to NVARCHAR(50). A validation rule "end date must be greater or equal than start date" can be mapped into a CHECK constraint CHECK (endDate >= startDate).
   • Describe the validation steps as JSON schema constraints.

2. Each validation rule is implemented as a decorator, under the hood this decorator calls validation APIs provided by the mixin above to define a new validation step. By using decorators, we put app developers in control of how to discover validation rules and which version of validation rule to use. There is no need for a central registry of all validation rules. (We may need to introduce such registry later, to support declarative declaration of validation steps in JSON files - see Declarative Support #565 - but let's not worry about that right now.)

   Example usage:

   @check(
     // which properties are required to perform this check
     // this allows the validation framework to skip this rule
     // when doing a partial update that's not touching these properties
     ['startDate', 'endDate'],
   
     // the validation check
     data => data.endDate >= data.startDate,
   )
   class Event {
     @unique()
     @minLength(5)
     @maxLength(20)
     @property()
     urlSlug: string;
   }

3. Repository implementations are expected to call validation APIs as part of the data-access operations. The repositories provided by LoopBack (DefaultCrudRepository, DefaultKeyValueRepository) should implement validation out of the box. This is based on the following assumption:

4. All Entity classes should come with the validation mixin applied. For example, if the mixin is called Validated, then the Entity class should be defined as class Entity extends Validated(Model). At the same time, the underlying Model class should not come with validation baked in, to allow app developers to define light-weight model classes used as a type of nested properties and also to describe input parameters and return values of remote (controller) methods.

Additional ideas/requirements:

• All validation checks should be asynchronous (returning a promise) by default. To improve performance, we can introduce support for synchronous validation checks too: if a validation check returns a thenable instance, then we await it, otherwise we treat the returned value as the validation outcome. (We already follow this approach in @loopback/context when resolving dependencies.)

• When a model instance (data) is not valid, the validation error should return all validation errors and include machine-readable details that will allow the clients to understand which property values are invalid (to render validation errors next to appropriate UI input elements) and offer the user localized error messages in their language.

• For validations that can be provided by the database only, e.g. uniqueness constraint or foreign keys, the framework should check at app startup and/or before the first database command is executed whether the database schema is configured to enforce these constraints.

• To be investigated: how to support auto-discovery of constraints, i.e. map database-specific constraints like NVARCHAR(50) or CHECK(start <= end) into LB4 validations.

@raymondfeng @strongloop/loopback-maintainers what's your opinion? Do you agree with my proposal? Is there anything you would like to add?