[WIP] newly proposed commands

[choheekim]

Newly Proposed Commands

This pr contains changes in command lines in ember-exam.

With the current existing/proposed commands,

• --split splits the full list of test modules into a number of partitions set by the input value, but --split feels lacking in meaning when looking at the command by itself.
• parallel is a boolean to run tests on multiple browsers and the number of browsers to invoke depending on the combination of commands: split or partition. When parallel is used with load-balance parallel takes a numeric value and parallel is to decide a number of browsers.

This confusion raises problems of that the 'parallel' command is convoluted the naming of some of the commands are not straight forward.

Newly proposed commands will replace split with partition-count and a combination of load-balance and parallel with browser-count. partition remains the same as before to specify the partition to execute.

The usage of browser-count defaults running tests with load-balancing mode, while keeping the existing commands to run tests as it does today. The new commands tries to remove the convoluted usage, provide more meaningful names and simple usages.

Example

1. With no parallel child processes (only one browser running a whole test suite)

current proposed CLI	new CLI
ember exam	ember exam
ember exam --random	ember exam --random
ember exam --split=2 --partition=1	Ember exam --partition-count=2 --partition=1
Ember exam --split=4 --partition=1..3	Ember exam --partition-count=4 --partition=1..3

2. With parallel child processes and no TLB ( number of browsers varied)

Old cli	New cli	Notes
ember exam --split=3 --partition1,2 --parallel	ember exam --partition-count=3 --partition1,2 --parallel	Split tests into 3 partitions and run partition 1 and 2 with 2 browsers
ember exam --split=3 --parallel	ember exam --partition-count=3 --parallel	Split tests into 3 partitions and run partition 1 and 2 with 2 browsers

3. With parallel child processes and TLB

Old cli	New cli	Notes
ember exam --load-balance --parallel=2	ember exam --browser-count=2	Execute test suites with two browsers with TLB
ember exam --split=2 --partition=1 --load-balance --parallel=3	ember exam --partition-count=2 --partition=1 --browser-count=3	Split tests into 2 partition and run the first partition with 1 browsers with TLB
ember exam --replay-execution=[file_name.json] --replay-browser=[number]	The same as before	/

*** The pr should come after #136 lands

[stefanpenner]

Is it fair to then describe --parallel as the short-hand or alias for something like --browser-count=n --no-load-balance ?

[stefanpenner]

Although this is more accurate, is the cost of split sufficiently high to require a rename? I worry the churn is more costly then the benefit. Thoughts?

[scalvert]

I personally have to do mental gymnastics to disambiguate the options in their current state. I always convert split to partition count in my mind. I'm in favor of naming it what it is.

I think we could keep both for backwards compatibility, but I'd just opt for a full conversion over with a major bump.

Am open to other thoughts.

[stefanpenner]

browser-count does not, for me anyways, imply load balancing. I wonder if we can do anything about that?