Running tests on multiple different browsers

[MehdiK]

We currently allow the user to pick which browser they want to run the tests on using IAppConfigurator's WithRemoteWebDriver method; but this only allows the user to pick one driver for a test. How can a user run a test on multiple browsers?

I was thinking perhaps we could allow the user to specify more than one driver and then run the test on all selected drivers; but then how do we report back which driver passed and which failed. So I am thinking perhaps it's the responsibility of the user to run the test over multiple SelenoHosts each one targeting a different driver. It's relatively simple to setup and they get proper reports back.

Thoughts?

[JakeGinnivan]

White has a similar problem..

xUnit actually can deal with the problem quite nicely. I might spike up how I do it in the White UITestBase..

I agree with it is the consumers responsibility, but this would be a good place for guidance :)

[MehdiK]

Yeah, definitely if we choose to leave it to devs we have to provide some
how-tos for it.

Looking forward to your spike. I really need to take a close look at White.

On Sun, Aug 4, 2013 at 2:40 PM, Jake Ginnivan notifications@github.comwrote:

White has a similar problem..

xUnit actually can deal with the problem quite nicely. I might spike up
how I do it in the White UITestBase..

I agree with it is the consumers responsibility, but this would be a good
place for guidance :)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/76#issuecomment-22069411
.

[robdmoore]

I've been thinking about this.

I suspect it might be possible to run it on a bunch of browsers by returning a proxy object for all calls that delegates to all the browsers. It could check each thing on all browsers and if any of them or multiple ones have a problem then an exception can be thrown that indicates which of them had a problem...

Not sure how practical it would be though. If its not then I think its important to make creating multiple hosts a first class citizen and easy to reference between them.

[JakeGinnivan]

What I don't like about that is the test reporting.

I think having a test attribute would be better, for example xUnit:

[SelenoTest(Browser.Firefox, Browser.Chrome)]
public void MyTest(Browser browser) {}

[SelenoTest(Browser.All)]
public void MyTest(Browser browser) {}

Using xUnit this would essentially be similar to the Theory attribute, which means a single test can produce multiple test runs.

[robdmoore]

That would be really cool, but we'd need to figure out how to either make it work across all test frameworks or provide a few implementations (and I guess separate nuget packages)...

Also, we need to be careful to allow people to use custom web drivers - if we use an enum that might prevent it (unless we just try and support all current web drivers and try and keep up with new ones?)

[MehdiK]

I was thinking along the same lines as Jake.

With regards to enum for web drivers that could be based on RemoteWebDriver
subclasses from Selenium; so not so much an enum but a string.

nUnit supports TestCases too which allows for this and IIRC you can achieve
the same thing using MSTest. Oh I am getting rusty on .Net :(

On Sun, Aug 4, 2013 at 3:17 PM, Robert Moore notifications@github.comwrote:

That would be really cool, but we'd need to figure out how to either make
it work across all test frameworks or provide a few implementations (and I
guess separate nuget packages)...

Also, we need to be careful to allow people to use custom web drivers - if
we use an enum that might prevent it (unless we just try and support all
current web drivers and try and keep up with new ones?)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/76#issuecomment-22069796
.

[mwhelan]

Yeah, I quite like Jake's suggestion (though I don't love attributes and
I'm not discounting Rob's suggestion as I haven't really engaged with it
yet). So would we look to provide solutions using each framework's multiple
test case functionality (i.e. nunit, xunit and mstest)?

Another option might be to not deal with multiple browsers in Seleno at
all. Add multiple test case functionality as a first class citizen in BDDfy
that is framework agnostic and write a How To on using the two together.....

Just an idea, haven't thought it through as my brain is in web api land
right now.... :-)

On Sun, Aug 4, 2013 at 11:52 AM, Mehdi Khalili notifications@github.comwrote:

I was thinking along the same lines as Jake.

With regards to enum for web drivers that could be based on
RemoteWebDriver
subclasses from Selenium; so not so much an enum but a string.

nUnit supports TestCases too which allows for this and IIRC you can
achieve
the same thing using MSTest. Oh I am getting rusty on .Net :(

On Sun, Aug 4, 2013 at 3:17 PM, Robert Moore notifications@github.comwrote:

That would be really cool, but we'd need to figure out how to either
make
it work across all test frameworks or provide a few implementations (and
I
guess separate nuget packages)...

Also, we need to be careful to allow people to use custom web drivers -
if
we use an enum that might prevent it (unless we just try and support all
current web drivers and try and keep up with new ones?)

—
Reply to this email directly or view it on GitHub<
https://github.com/TestStack/TestStack.Seleno/issues/76#issuecomment-22069796>

.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/76#issuecomment-22069851
.

[robdmoore]

Relates to #35

[MehdiK]

Thanks for the link @robdmoore and sorry for creating a duplicate entry. I guess one could be closed. Since this has more discussions on it perhaps we can close the other issue and keep this one open. Thoughts?

[robdmoore]

I think that this makes sense as examples of how to do it rather than something we need to build in. The fact you can take control over the SelenoHost now and instantiate and keep track of multiple instances means that it would be fairly easy to pull something like this off.

[circa1741]

For what its worth, I came across this:
http://blog.dmbcllc.com/running-selenium-in-parallel-with-any-net-unit-testing-tool/
Thanks

[huan086]

@robdmoore did you managed to solve the issue? When I run multiple SelenoHost, I'm getting SelenoException : You have already created a Seleno application; Seleno currently only supports one application at a time per app domain