Fix test suite

[bajtos]

Our test suite has several problems.

1. bluemix generator is calling process.exit() on failure, which aborts Mocha test suite in the middle. Because the process exits with exit code 0, it looks like all tests passed.

   generator-loopback/bluemix/index.js

   Lines 74 to 75 in 060bdd7

   	this.log(chalk.red('\n Invalid LoopBack directory\n'));
   	process.exit();

   I am proposing to change this block and throw an Error instead.

   throw new Error('Invalid LoopBack directory');

2. The test suite for bluemix generator works when executed on its own (mocha test/bluemix.test.js) passes, but fails when executed together with other tests. We need to find and fix the reason.

3. Setting the bluemix test suite aside, several other tests are failing. These tests are not executed now because bluemix exits the process prematurely. We need to fix these tests to pass again.

[dhmlau]

I think the description contains pretty clear acceptance criteria. Moving for estimation.

[bajtos]

At my machine, there are 17 failing tests.

1. loopback:bluemix generator should generate datasources.bluemix.js:
2. loopback:bluemix generator should generate .bluemix/datasources-config.json:
3. loopback:bluemix generator should generate all basic Bluemix files:
4. loopback:bluemix generator should generate only basic Bluemix files:
5. loopback:bluemix generator should generate only Docker files:
6. loopback:bluemix generator should generate only toolchain files:
7. loopback:bluemix generator should generate manifest file:
8. loopback:export-api-def generator produces JSON format:
9. loopback:export-api-def generator produces YAML format:
10. loopback:export-api-def generator running on project with onging async operations
11. loopback:export-api-def generator running on project with onging async operations eventually exits the process:
12. loopback generator help prints right help message for each generator prints right help message for generator app:
13. loopback:middleware generator adds a new phase to server/middleware.json:
14. loopback:middleware generator adds a new phase next to a selected one:
15. loopback:middleware generator adds a new entry to an existing phase in server/middleware.json:
16. loopback:middleware generator supports sub-phase:
17. loopback:soap tests periodic table wsdl periodic table wsdl:
    Uncaught Error: Invalid WSDL URL: http://www.webservicex.net/periodictable.asmx?WSDL

By looking at the detailed logs (see below), I suspect most test are failing because of the same reason, which is that somehow loopback-workspace is creating a new project in test/sandbox/test-app instead of test/sandbox, where the tests expect the project to be created.

12. loopback generator help prints right help message for each generator prints right help message for generator app:

This test should be easy to fix, just edit expected help text to accommodate recent additions.

17. loopback:soap tests periodic table wsdl periodic table wsdl:
    Uncaught Error: Invalid WSDL URL: http://www.webservicex.net/periodictable.asmx?WSDL

Looks like the periodictable API is gone. I think we will need to rework the test to use a different api, see http://www.webservicex.net/api

Expand to see full failure details

1) loopback:bluemix generator should generate datasources.bluemix.js: Uncaught Error: Invalid LoopBack directory at constructor.validateLoopBackDir (bluemix/index.js:74:15) at Object. (node_modules/yeoman-generator/lib/base.js:439:23) at node_modules/run-async/index.js:25:25 at new Promise () at node_modules/run-async/index.js:24:19 at node_modules/yeoman-generator/lib/base.js:440:9 at process.topLevelDomainCallback (domain.js:121:23)

2. loopback:bluemix generator should generate .bluemix/datasources-config.json:
   Uncaught Error: Invalid LoopBack directory
   at constructor.validateLoopBackDir (bluemix/index.js:74:15)
   at Object. (node_modules/yeoman-generator/lib/base.js:439:23)
   at node_modules/run-async/index.js:25:25
   at new Promise ()
   at node_modules/run-async/index.js:24:19
   at node_modules/yeoman-generator/lib/base.js:440:9
   at process.topLevelDomainCallback (domain.js:121:23)

3. loopback:bluemix generator should generate all basic Bluemix files:
   Uncaught Error: Invalid LoopBack directory
   at constructor.validateLoopBackDir (bluemix/index.js:74:15)
   at Object. (node_modules/yeoman-generator/lib/base.js:439:23)
   at node_modules/run-async/index.js:25:25
   at new Promise ()
   at node_modules/run-async/index.js:24:19
   at node_modules/yeoman-generator/lib/base.js:440:9
   at process.topLevelDomainCallback (domain.js:121:23)

4. loopback:bluemix generator should generate only basic Bluemix files:
   Uncaught Error: Invalid LoopBack directory
   at constructor.validateLoopBackDir (bluemix/index.js:74:15)
   at Object. (node_modules/yeoman-generator/lib/base.js:439:23)
   at node_modules/run-async/index.js:25:25
   at new Promise ()
   at node_modules/run-async/index.js:24:19
   at node_modules/yeoman-generator/lib/base.js:440:9
   at process.topLevelDomainCallback (domain.js:121:23)

5. loopback:bluemix generator should generate only Docker files:
   Uncaught Error: Invalid LoopBack directory
   at constructor.validateLoopBackDir (bluemix/index.js:74:15)
   at Object. (node_modules/yeoman-generator/lib/base.js:439:23)
   at node_modules/run-async/index.js:25:25
   at new Promise ()
   at node_modules/run-async/index.js:24:19
   at node_modules/yeoman-generator/lib/base.js:440:9
   at process.topLevelDomainCallback (domain.js:121:23)

6. loopback:bluemix generator should generate only toolchain files:
   Uncaught Error: Invalid LoopBack directory
   at constructor.validateLoopBackDir (bluemix/index.js:74:15)
   at Object. (node_modules/yeoman-generator/lib/base.js:439:23)
   at node_modules/run-async/index.js:25:25
   at new Promise ()
   at node_modules/run-async/index.js:24:19
   at node_modules/yeoman-generator/lib/base.js:440:9
   at process.topLevelDomainCallback (domain.js:121:23)

7. loopback:bluemix generator should generate manifest file:
   Uncaught Error: Invalid LoopBack directory
   at constructor.validateLoopBackDir (bluemix/index.js:74:15)
   at Object. (node_modules/yeoman-generator/lib/base.js:439:23)
   at node_modules/run-async/index.js:25:25
   at new Promise ()
   at node_modules/run-async/index.js:24:19
   at node_modules/yeoman-generator/lib/base.js:440:9
   at process.topLevelDomainCallback (domain.js:121:23)

8. loopback:export-api-def generator produces JSON format:
   Uncaught Error: Cannot find module '/Users/bajtos/src/loopback/generator-loopback/test/sandbox'
   at Function.Module._resolveFilename (internal/modules/cjs/loader.js:594:15)
   at Function.Module._load (internal/modules/cjs/loader.js:520:25)
   at Module.require (internal/modules/cjs/loader.js:650:17)
   at require (internal/modules/cjs/helpers.js:20:18)
   at constructor.generate (export-api-def/index.js:38:15)
   at Object. (node_modules/yeoman-generator/lib/base.js:439:23)
   at node_modules/run-async/index.js:25:25
   at new Promise ()
   at node_modules/run-async/index.js:24:19
   at node_modules/yeoman-generator/lib/base.js:440:9
   at process.topLevelDomainCallback (domain.js:121:23)

9. loopback:export-api-def generator produces YAML format:
   Uncaught Error: Cannot find module '/Users/bajtos/src/loopback/generator-loopback/test/sandbox'
   at Function.Module._resolveFilename (internal/modules/cjs/loader.js:594:15)
   at Function.Module._load (internal/modules/cjs/loader.js:520:25)
   at Module.require (internal/modules/cjs/loader.js:650:17)
   at require (internal/modules/cjs/helpers.js:20:18)
   at constructor.generate (export-api-def/index.js:38:15)
   at Object. (node_modules/yeoman-generator/lib/base.js:439:23)
   at node_modules/run-async/index.js:25:25
   at new Promise ()
   at node_modules/run-async/index.js:24:19
   at node_modules/yeoman-generator/lib/base.js:440:9
   at process.topLevelDomainCallback (domain.js:121:23)

10. loopback:export-api-def generator running on project with onging async operations immediately does not exit the process:
    Uncaught Error: Cannot find module '/Users/bajtos/src/loopback/generator-loopback/test/sandbox'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:594:15)
    at Function.Module._load (internal/modules/cjs/loader.js:520:25)
    at Module.require (internal/modules/cjs/loader.js:650:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at constructor.generate (export-api-def/index.js:38:15)
    at Object. (node_modules/yeoman-generator/lib/base.js:439:23)
    at node_modules/run-async/index.js:25:25
    at new Promise ()
    at node_modules/run-async/index.js:24:19
    at node_modules/yeoman-generator/lib/base.js:440:9
    at process.topLevelDomainCallback (domain.js:121:23)

11. loopback:export-api-def generator running on project with onging async operations eventually exits the process:
    Uncaught Error: Cannot find module '/Users/bajtos/src/loopback/generator-loopback/test/sandbox'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:594:15)
    at Function.Module._load (internal/modules/cjs/loader.js:520:25)
    at Module.require (internal/modules/cjs/loader.js:650:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at constructor.generate (export-api-def/index.js:38:15)
    at Object. (node_modules/yeoman-generator/lib/base.js:439:23)
    at node_modules/run-async/index.js:25:25
    at new Promise ()
    at node_modules/run-async/index.js:24:19
    at node_modules/yeoman-generator/lib/base.js:440:9
    at process.topLevelDomainCallback (domain.js:121:23)

12. loopback generator help prints right help message for each generator prints right help message for generator app:

    AssertionError [ERR_ASSERTION]: 'Usage:\n slc loopback:app [options] []\n\nOptions:\n -h, --help # Print the command's options and usa... == 'Usage:\n slc loopback:app [options] []\n\nOptions:\n -h, --help # Print the command's options and usa...

    • expected - actual

         --skip-cache       # Do not remember prompt answers                          Default: false
         --skip-install     # Do not install npm dependencies                         Default: false
         --skip-next-steps  # Do not print "next steps" info
         --explorer         # Add Loopback Explorer to the project (true by default)
      

    •    --loopbackVersion  # Select the LoopBack version
      

    •    --template         # Set up the LoopBack application template
         --bluemix          # Set up as a Bluemix app
      

    Arguments:
    name # Name of the application to scaffold. Type: String Required: false

    at Context. (test/help.test.js:83:16)
    at process.topLevelDomainCallback (domain.js:121:23)

13. loopback:middleware generator adds a new phase to server/middleware.json:
    Error: ENOENT: no such file or directory, open '/Users/bajtos/src/loopback/generator-loopback/test/sandbox/server/middleware.json'
    at Object.openSync (fs.js:443:3)
    at Object.readFileSync (fs.js:348:35)
    at readMiddlewaresJsonSync (test/middleware.test.js:113:22)
    at Context. (test/middleware.test.js:34:38)
    at node_modules/loopback-workspace/node_modules/async/lib/async.js:726:13
    at node_modules/loopback-workspace/node_modules/async/lib/async.js:52:16
    at node_modules/loopback-workspace/node_modules/async/lib/async.js:269:32
    at node_modules/loopback-workspace/node_modules/async/lib/async.js:44:16
    at node_modules/loopback-workspace/node_modules/async/lib/async.js:723:17
    at node_modules/loopback-workspace/node_modules/async/lib/async.js:167:37
    at process._tickCallback (internal/process/next_tick.js:61:11)

14. loopback:middleware generator adds a new phase next to a selected one:

    Uncaught AssertionError: expected 0 to equal -2

    • expected - actual

    -0
    +-2

    at test/middleware.test.js:58:21
    at constructor. (node_modules/yeoman-generator/lib/base.js:419:5)
    at Queue.EventEmitter.emit (domain.js:442:20)
    at Queue. (node_modules/grouped-queue/lib/queue.js:68:12)
    at Queue.next (node_modules/grouped-queue/lib/queue.js:79:43)
    at SubQueue.run (node_modules/grouped-queue/lib/subqueue.js:42:45)
    at Queue.next (node_modules/grouped-queue/lib/queue.js:80:39)
    at SubQueue.run (node_modules/grouped-queue/lib/subqueue.js:42:45)
    at Queue.next (node_modules/grouped-queue/lib/queue.js:80:39)
    at SubQueue.run (node_modules/grouped-queue/lib/subqueue.js:42:45)
    at Queue.next (node_modules/grouped-queue/lib/queue.js:80:39)
    at SubQueue.run (node_modules/grouped-queue/lib/subqueue.js:42:45)
    at Queue.next (node_modules/grouped-queue/lib/queue.js:80:39)
    at SubQueue.run (node_modules/grouped-queue/lib/subqueue.js:42:45)
    at Queue.next (node_modules/grouped-queue/lib/queue.js:80:39)
    at SubQueue.run (node_modules/grouped-queue/lib/subqueue.js:42:45)
    at Queue.next (node_modules/grouped-queue/lib/queue.js:80:39)
    at SubQueue.run (node_modules/grouped-queue/lib/subqueue.js:42:45)
    at Queue.next (node_modules/grouped-queue/lib/queue.js:80:39)
    at SubQueue.run (node_modules/grouped-queue/lib/subqueue.js:42:45)
    at Queue.next (node_modules/grouped-queue/lib/queue.js:80:39)
    at Queue._exec (node_modules/grouped-queue/lib/queue.js:83:3)
    at constructor. (node_modules/yeoman-generator/lib/base.js:495:7)
    at node_modules/async/dist/async.js:3888:9
    at node_modules/async/dist/async.js:473:16
    at replenish (node_modules/async/dist/async.js:1006:25)
    at node_modules/async/dist/async.js:1016:9
    at eachOfLimit (node_modules/async/dist/async.js:1041:24)
    at node_modules/async/dist/async.js:1046:16
    at _parallel (node_modules/async/dist/async.js:3879:5)
    at Object.series (node_modules/async/dist/async.js:4735:5)
    at Conflicter.resolve (node_modules/yeoman-generator/lib/util/conflicter.js:91:9)
    at constructor. (node_modules/yeoman-generator/lib/base.js:490:21)
    at process.topLevelDomainCallback (domain.js:121:23)

15. loopback:middleware generator adds a new entry to an existing phase in server/middleware.json:
    Error: ENOENT: no such file or directory, open '/Users/bajtos/src/loopback/generator-loopback/test/sandbox/server/middleware.json'
    at Object.openSync (fs.js:443:3)
    at Object.readFileSync (fs.js:348:35)
    at readMiddlewaresJsonSync (test/middleware.test.js:113:22)
    at Context. (test/middleware.test.js:74:9)
    at node_modules/loopback-workspace/node_modules/async/lib/async.js:726:13
    at node_modules/loopback-workspace/node_modules/async/lib/async.js:52:16
    at node_modules/loopback-workspace/node_modules/async/lib/async.js:269:32
    at node_modules/loopback-workspace/node_modules/async/lib/async.js:44:16
    at node_modules/loopback-workspace/node_modules/async/lib/async.js:723:17
    at node_modules/loopback-workspace/node_modules/async/lib/async.js:167:37
    at process._tickCallback (internal/process/next_tick.js:61:11)

16. loopback:middleware generator supports sub-phase:
    Error: ENOENT: no such file or directory, open '/Users/bajtos/src/loopback/generator-loopback/test/sandbox/server/middleware.json'
    at Object.openSync (fs.js:443:3)
    at Object.readFileSync (fs.js:348:35)
    at readMiddlewaresJsonSync (test/middleware.test.js:113:22)
    at Context. (test/middleware.test.js:94:7)
    at node_modules/loopback-workspace/node_modules/async/lib/async.js:726:13
    at node_modules/loopback-workspace/node_modules/async/lib/async.js:52:16
    at node_modules/loopback-workspace/node_modules/async/lib/async.js:269:32
    at node_modules/loopback-workspace/node_modules/async/lib/async.js:44:16
    at node_modules/loopback-workspace/node_modules/async/lib/async.js:723:17
    at node_modules/loopback-workspace/node_modules/async/lib/async.js:167:37
    at process._tickCallback (internal/process/next_tick.js:61:11)

17. loopback:soap tests periodic table wsdl periodic table wsdl:
    Uncaught Error: Invalid WSDL URL: http://www.webservicex.net/periodictable.asmx?WSDL

Code: 404

Response Body: "{"code":"PAGE_NOT_FOUND","message":"Page not found"}\n"
at node_modules/strong-soap/src/parser/wsdl.js:419:22
at Request._callback (node_modules/strong-soap/src/http.js:136:9)
at Request.self.callback (node_modules/request/request.js:185:22)
at Request.EventEmitter.emit (domain.js:442:20)
at Request. (node_modules/request/request.js:1157:10)
at Request.EventEmitter.emit (domain.js:442:20)
at IncomingMessage. (node_modules/request/request.js:1079:12)
at IncomingMessage.EventEmitter.emit (domain.js:442:20)
at endReadableNT (_stream_readable.js:1081:12)
at process._tickCallback (internal/process/next_tick.js:63:19)