[CLI] Add tests verifying our templates pass `npm test`

[bajtos]

Overview

Right now, we have zero automated tests verifying that commands like lb4 app, lb4 extension, etc. produce a project that can be compiled and that pass its own tests. As a result, it's easy to accidentally break our templates or forget to update them after making a breaking change in runtime parts.

We should add acceptance (end-to-end) tests verifying that each CLI command leaves the target project in a state where npm test passes.

See end-to-end tests in loopback-workspace for inspiration

Acceptance Criteria

Each of the following commands has an acceptance test

• lb4 app
• lb4 controller
• lb4 example (one test for each packages/example-*)
• lb4 extension

Project Template Tests

(These are the various lb4 example functions)

• Create an empty directory using testlab's sandbox functionality
  • Possible idea: Add .sandbox to packages/ and then cleanup / bootstrap the monorepo. For design on local dependency install reach out to @raymondfeng
• Run the selected command
• Fork/spawn/whatever to run npm start inside the template project instance
• Assert that
  • npm start was successful
  • make requests (using the request lib, not supertest!) against the target project/example to validate that it is functional
    • these do not need to be exhaustive examinations of all of the routes (the example templates have their own test suites!)
• Spin down the instance as a part of cleanup (process kill)

Individual Template Tests

• Create a target project in a temporary directory using testlab's sandbox functionality using lb4 app
• Call the appropriate function to generate a template file
• Shell out a command to trigger the typescript compiler
• Assert that no errors were thrown by the compiler when building the template instance

Original Discussion

These test must create the target project in a directory outside of our monorepo, otherwise we may not detect the situation where the template is missing a dependency, because the test will resolve that missing dependency from monorepo dependencies.

In order to speed up installation times, especially when running locally multiple times, we should consider using https://github.com/strongloop/strong-cached-install. However, note that the current version of strong-cached-install enforces legacy node_modules format used by npm in Node.js 0.x days. We may want to rework strong-cached-install to support the current layout, where deep dependencies are hoisted and shared in the top-level node_modules folder.

[kjdelisle]

+1

As much as we need to lock down the MVP scope, the truth is that this should've been done to begin with.

[kjdelisle]

After our estimation session, @dhmlau @raymondfeng and myself have decided to pull this from MVP.

[raymondfeng]

There are two factors for the justification to pull it out of MVP even it's very valuable:

1. The cli module is fairly stable and we don't see more changes for the MVP. We expect smoke tests to be performed whenever new PRs are being created/reviewed.

2. The amount of work for this issue will be pretty big.