From 9e018415a0718daa68263f6b93bd798689244fdc Mon Sep 17 00:00:00 2001 From: Jia Hao Goh Date: Tue, 1 Sep 2020 11:37:12 +0800 Subject: [PATCH 1/4] [integration_test] Recommend tests to be in `integration_test/`, fix example In this way, there is a clear distinction between integration tests that run on a device (in `integration_test/`, and widget tests that run with the flutter tester in `test/`. flutter/flutter#64690 --- .cirrus.yml | 2 +- packages/integration_test/README.md | 89 +++++++++++-------- packages/integration_test/example/README.md | 23 +++-- .../example_test.dart} | 4 +- .../example_test_io.dart} | 0 .../example_test_web.dart} | 0 .../extended_test.dart} | 7 +- .../extended_test_io.dart} | 0 .../extended_test_web.dart} | 0 .../failure_test.dart} | 0 .../test_driver/example_integration_test.dart | 5 -- .../test_driver/integration_test_driver.dart | 3 + ... => integration_test_extended_driver.dart} | 2 - ...t => integration_test_failure_driver.dart} | 4 +- 14 files changed, 77 insertions(+), 62 deletions(-) rename packages/integration_test/example/{test_driver/example_integration.dart => integration_test/example_test.dart} (82%) rename packages/integration_test/example/{test_driver/example_integration_io.dart => integration_test/example_test_io.dart} (100%) rename packages/integration_test/example/{test_driver/example_integration_web.dart => integration_test/example_test_web.dart} (100%) rename packages/integration_test/example/{test_driver/example_integration_extended.dart => integration_test/extended_test.dart} (67%) rename packages/integration_test/example/{test_driver/example_integration_io_extended.dart => integration_test/extended_test_io.dart} (100%) rename packages/integration_test/example/{test_driver/example_integration_web_extended.dart => integration_test/extended_test_web.dart} (100%) rename packages/integration_test/example/{test_driver/failure.dart => integration_test/failure_test.dart} (100%) delete mode 100644 packages/integration_test/example/test_driver/example_integration_test.dart create mode 100644 packages/integration_test/example/test_driver/integration_test_driver.dart rename packages/integration_test/example/test_driver/{example_integration_extended_test.dart => integration_test_extended_driver.dart} (94%) rename packages/integration_test/example/test_driver/{failure_test.dart => integration_test_failure_driver.dart} (96%) diff --git a/.cirrus.yml b/.cirrus.yml index d5a1ac82f6d5..25396ea3faf4 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -59,7 +59,7 @@ task: - ./chromedriver/chromedriver --port=4444 & test_script: - cd $INTEGRATION_TEST_PATH/example/ - - flutter drive -v --target=test_driver/example_integration.dart -d web-server --release --browser-name=chrome + - flutter drive -v --driver=test_driver/integration_test_driver.dart --target=integration_test/example_test.dart -d web-server --release --browser-name=chrome - name: build-apks+java-test+firebase-test-lab env: matrix: diff --git a/packages/integration_test/README.md b/packages/integration_test/README.md index a48210beacf8..4708ed66a2df 100644 --- a/packages/integration_test/README.md +++ b/packages/integration_test/README.md @@ -7,11 +7,12 @@ and native Android instrumentation testing. ## Usage Add a dependency on the `integration_test` and `flutter_test` package in the -`dev_dependencies` section of pubspec.yaml. For plugins, do this in the -pubspec.yaml of the example app. +`dev_dependencies` section of `pubspec.yaml`. For plugins, do this in the +`pubspec.yaml` of the example app. -Invoke `IntegrationTestWidgetsFlutterBinding.ensureInitialized()` at the start -of a test file, e.g. +Create a `integration_test/` directory for your package. In this directory, +create a `_test.dart`, using the following as a starting point to make +assertions. ```dart import 'package:flutter_test/flutter_test.dart'; @@ -19,66 +20,78 @@ import 'package:integration_test/integration_test.dart'; void main() { IntegrationTestWidgetsFlutterBinding.ensureInitialized(); + testWidgets("failing test example", (WidgetTester tester) async { expect(2 + 2, equals(5)); }); } ``` -## Test locations - -It is recommended to put integration_test tests in the `test/` folder of the app -or package. For example apps, if the integration_test test references example -app code, it should go in `example/test/`. It is also acceptable to put -integration_test tests in `test_driver/` folder so that they're alongside the -runner app (see below). - -## Using Flutter Driver to Run Tests - -`IntegrationTestWidgetsTestBinding` supports launching the on-device tests with -`flutter drive`. Note that the tests don't use the `FlutterDriver` API, they -use `testWidgets` instead. +### Driver Entrypoint -Put the a file named `_integration_test.dart` in the app' -`test_driver` directory: +An accompanying driver script will be needed that can be shared across all +integration tests. Create a `integration_test_driver.dart` in the `test_driver/` +directory with the following contents: ```dart -import 'dart:async'; - import 'package:integration_test/integration_test_driver.dart'; -Future main() async => integrationDriver(); +Future main() => integrationDriver(); ``` -To run a example app test with Flutter driver: +### Package Structure + +Your package should have a structure that looks like this: -```sh -cd example -flutter drive test/_integration.dart +``` +lib/ + ... +integration_test/ + foo_test.dart + bar_test.dart +test/ + # Other unit tests go here. +test_driver/ + integration_test_driver.dart ``` -To test plugin APIs using Flutter driver: +[Example](https://github.com/flutter/plugins/tree/master/packages/integration_test/example) + +## Using Flutter Driver to Run Tests + +These tests can be launched with the `flutter drive` command. + +To run `integration_test/foo_test.dart`: ```sh -cd example -flutter drive --driver=test_driver/_test.dart test/_integration_test.dart +flutter drive \ + --driver=test_driver/integration_test_driver.dart \ + --target=integration_test/foo_test.dart ``` -You can run tests on web in release or profile mode. +### Web + +Make sure you have [enabled web support](https://flutter.dev/docs/get-started/web#set-up) +then [download and run](https://flutter.dev/docs/cookbook/testing/integration/introduction#6b-web) +the web driver in another process. -First you need to make sure you have downloaded the driver for the browser. +Use following command to execute the tests, passing either `--release` or +`--profile` as debug mode is not supported. ```sh -cd example -flutter drive -v --target=test_driver/dart -d web-server --release --browser-name=chrome +flutter drive \ + --driver=test_driver/integration_test_driver.dart \ + --target=integration_test/foo_test.dart \ + -d web-server \ + --release ``` -## Android device testing +## Android Device Testing Create an instrumentation test file in your application's **android/app/src/androidTest/java/com/example/myapp/** directory (replacing com, example, and myapp with values from your app's package name). You can name -this test file MainActivityTest.java or another name of your choice. +this test file `MainActivityTest.java` or another name of your choice. ```java package com.example.myapp; @@ -96,7 +109,7 @@ public class MainActivityTest { ``` Update your application's **myapp/android/app/build.gradle** to make sure it -uses androidx's version of AndroidJUnitRunner and has androidx libraries as a +uses androidx's version of `AndroidJUnitRunner` and has androidx libraries as a dependency. ```gradle @@ -160,7 +173,7 @@ You can pass additional parameters on the command line, such as the devices you want to test on. See [gcloud firebase test android run](https://cloud.google.com/sdk/gcloud/reference/firebase/test/android/run). -## iOS device testing +## iOS Device Testing You need to change `iOS/Podfile` to avoid test target statically linking to the plugins. One way is to link all of the plugins dynamically: @@ -189,4 +202,4 @@ change the code. You can change `RunnerTests.m` to the name of your choice. INTEGRATION_TEST_IOS_RUNNER(RunnerTests) ``` -Now you can start RunnerTests to kick out integration tests! +Now you can start RunnerTests to kick-off integration tests! diff --git a/packages/integration_test/example/README.md b/packages/integration_test/example/README.md index 7c61fbf5c8de..8cda23216d6c 100644 --- a/packages/integration_test/example/README.md +++ b/packages/integration_test/example/README.md @@ -2,15 +2,22 @@ Demonstrates how to use the `package:integration_test`. -## Getting Started +To run `integration_test/example_test.dart`, -This project is a starting point for a Flutter application. +Android / iOS: -A few resources to get you started if this is your first Flutter project: +```sh +flutter drive \ + --driver=test_driver/integration_test_driver.dart \ + --target=integration_test/example_test.dart +``` -- [Lab: Write your first Flutter app](https://flutter.dev/docs/get-started/codelab) -- [Cookbook: Useful Flutter samples](https://flutter.dev/docs/cookbook) +Web: -For help getting started with Flutter, view our -[online documentation](https://flutter.dev/docs), which offers tutorials, -samples, guidance on mobile development, and a full API reference. +```sh +flutter drive \ + --driver=test_driver/integration_test_driver.dart \ + --target=integration_test/example_test.dart \ + -d web-server \ + --release +``` diff --git a/packages/integration_test/example/test_driver/example_integration.dart b/packages/integration_test/example/integration_test/example_test.dart similarity index 82% rename from packages/integration_test/example/test_driver/example_integration.dart rename to packages/integration_test/example/integration_test/example_test.dart index 3ee993b911b9..ca00cf594d81 100644 --- a/packages/integration_test/example/test_driver/example_integration.dart +++ b/packages/integration_test/example/integration_test/example_test.dart @@ -7,8 +7,8 @@ import 'package:integration_test/integration_test.dart'; -import 'example_integration_io.dart' - if (dart.library.html) 'example_integration_web.dart' as tests; +import 'example_test_io.dart' if (dart.library.html) 'example_test_web.dart' + as tests; void main() { IntegrationTestWidgetsFlutterBinding.ensureInitialized(); diff --git a/packages/integration_test/example/test_driver/example_integration_io.dart b/packages/integration_test/example/integration_test/example_test_io.dart similarity index 100% rename from packages/integration_test/example/test_driver/example_integration_io.dart rename to packages/integration_test/example/integration_test/example_test_io.dart diff --git a/packages/integration_test/example/test_driver/example_integration_web.dart b/packages/integration_test/example/integration_test/example_test_web.dart similarity index 100% rename from packages/integration_test/example/test_driver/example_integration_web.dart rename to packages/integration_test/example/integration_test/example_test_web.dart diff --git a/packages/integration_test/example/test_driver/example_integration_extended.dart b/packages/integration_test/example/integration_test/extended_test.dart similarity index 67% rename from packages/integration_test/example/test_driver/example_integration_extended.dart rename to packages/integration_test/example/integration_test/extended_test.dart index 79ed2762165e..7796fc0c97bc 100644 --- a/packages/integration_test/example/test_driver/example_integration_extended.dart +++ b/packages/integration_test/example/integration_test/extended_test.dart @@ -1,6 +1,7 @@ // This is a Flutter widget test can take a screenshot. // -// NOTE: Screenshots are only supported on Web for now. +// NOTE: Screenshots are only supported on Web for now. For Web, this needs to +// be executed with the `test_driver/integration_test_extended_driver.dart`. // // To perform an interaction with a widget in your test, use the WidgetTester // utility that Flutter provides. For example, you can send tap and scroll @@ -9,8 +10,8 @@ import 'package:integration_test/integration_test.dart'; -import 'example_integration_io_extended.dart' - if (dart.library.html) 'example_integration_web_extended.dart' as tests; +import 'extended_test_io.dart' if (dart.library.html) 'extended_test_web.dart' + as tests; void main() { IntegrationTestWidgetsFlutterBinding.ensureInitialized(); diff --git a/packages/integration_test/example/test_driver/example_integration_io_extended.dart b/packages/integration_test/example/integration_test/extended_test_io.dart similarity index 100% rename from packages/integration_test/example/test_driver/example_integration_io_extended.dart rename to packages/integration_test/example/integration_test/extended_test_io.dart diff --git a/packages/integration_test/example/test_driver/example_integration_web_extended.dart b/packages/integration_test/example/integration_test/extended_test_web.dart similarity index 100% rename from packages/integration_test/example/test_driver/example_integration_web_extended.dart rename to packages/integration_test/example/integration_test/extended_test_web.dart diff --git a/packages/integration_test/example/test_driver/failure.dart b/packages/integration_test/example/integration_test/failure_test.dart similarity index 100% rename from packages/integration_test/example/test_driver/failure.dart rename to packages/integration_test/example/integration_test/failure_test.dart diff --git a/packages/integration_test/example/test_driver/example_integration_test.dart b/packages/integration_test/example/test_driver/example_integration_test.dart deleted file mode 100644 index 10acaa69a42e..000000000000 --- a/packages/integration_test/example/test_driver/example_integration_test.dart +++ /dev/null @@ -1,5 +0,0 @@ -import 'dart:async'; - -import 'package:integration_test/integration_test_driver.dart'; - -Future main() async => integrationDriver(); diff --git a/packages/integration_test/example/test_driver/integration_test_driver.dart b/packages/integration_test/example/test_driver/integration_test_driver.dart new file mode 100644 index 000000000000..b38629cca97b --- /dev/null +++ b/packages/integration_test/example/test_driver/integration_test_driver.dart @@ -0,0 +1,3 @@ +import 'package:integration_test/integration_test_driver.dart'; + +Future main() => integrationDriver(); diff --git a/packages/integration_test/example/test_driver/example_integration_extended_test.dart b/packages/integration_test/example/test_driver/integration_test_extended_driver.dart similarity index 94% rename from packages/integration_test/example/test_driver/example_integration_extended_test.dart rename to packages/integration_test/example/test_driver/integration_test_extended_driver.dart index 1428a5092a78..056ba4bad722 100644 --- a/packages/integration_test/example/test_driver/example_integration_extended_test.dart +++ b/packages/integration_test/example/test_driver/integration_test_extended_driver.dart @@ -1,5 +1,3 @@ -import 'dart:async'; - import 'package:flutter_driver/flutter_driver.dart'; import 'package:integration_test/integration_test_driver_extended.dart'; diff --git a/packages/integration_test/example/test_driver/failure_test.dart b/packages/integration_test/example/test_driver/integration_test_failure_driver.dart similarity index 96% rename from packages/integration_test/example/test_driver/failure_test.dart rename to packages/integration_test/example/test_driver/integration_test_failure_driver.dart index bd9302f39409..0b60e471d711 100644 --- a/packages/integration_test/example/test_driver/failure_test.dart +++ b/packages/integration_test/example/test_driver/integration_test_failure_driver.dart @@ -1,7 +1,5 @@ -import 'dart:async'; - -import 'package:integration_test/common.dart' as common; import 'package:flutter_driver/flutter_driver.dart'; +import 'package:integration_test/common.dart' as common; import 'package:test/test.dart'; Future main() async { From e53c2d0ec8a0cfcc1d56870c92dce249bc10a163 Mon Sep 17 00:00:00 2001 From: Jia Hao Goh Date: Tue, 1 Sep 2020 19:19:04 +0800 Subject: [PATCH 2/4] Use underscore prefix for conditional imports --- .../{example_test_io.dart => _example_test_io.dart} | 0 .../{example_test_web.dart => _example_test_web.dart} | 0 .../{extended_test_io.dart => _extended_test_io.dart} | 0 .../{extended_test_web.dart => _extended_test_web.dart} | 0 .../integration_test/example/integration_test/example_test.dart | 2 +- .../example/integration_test/extended_test.dart | 2 +- 6 files changed, 2 insertions(+), 2 deletions(-) rename packages/integration_test/example/integration_test/{example_test_io.dart => _example_test_io.dart} (100%) rename packages/integration_test/example/integration_test/{example_test_web.dart => _example_test_web.dart} (100%) rename packages/integration_test/example/integration_test/{extended_test_io.dart => _extended_test_io.dart} (100%) rename packages/integration_test/example/integration_test/{extended_test_web.dart => _extended_test_web.dart} (100%) diff --git a/packages/integration_test/example/integration_test/example_test_io.dart b/packages/integration_test/example/integration_test/_example_test_io.dart similarity index 100% rename from packages/integration_test/example/integration_test/example_test_io.dart rename to packages/integration_test/example/integration_test/_example_test_io.dart diff --git a/packages/integration_test/example/integration_test/example_test_web.dart b/packages/integration_test/example/integration_test/_example_test_web.dart similarity index 100% rename from packages/integration_test/example/integration_test/example_test_web.dart rename to packages/integration_test/example/integration_test/_example_test_web.dart diff --git a/packages/integration_test/example/integration_test/extended_test_io.dart b/packages/integration_test/example/integration_test/_extended_test_io.dart similarity index 100% rename from packages/integration_test/example/integration_test/extended_test_io.dart rename to packages/integration_test/example/integration_test/_extended_test_io.dart diff --git a/packages/integration_test/example/integration_test/extended_test_web.dart b/packages/integration_test/example/integration_test/_extended_test_web.dart similarity index 100% rename from packages/integration_test/example/integration_test/extended_test_web.dart rename to packages/integration_test/example/integration_test/_extended_test_web.dart diff --git a/packages/integration_test/example/integration_test/example_test.dart b/packages/integration_test/example/integration_test/example_test.dart index ca00cf594d81..918aec8777de 100644 --- a/packages/integration_test/example/integration_test/example_test.dart +++ b/packages/integration_test/example/integration_test/example_test.dart @@ -7,7 +7,7 @@ import 'package:integration_test/integration_test.dart'; -import 'example_test_io.dart' if (dart.library.html) 'example_test_web.dart' +import '_example_test_io.dart' if (dart.library.html) '_example_test_web.dart' as tests; void main() { diff --git a/packages/integration_test/example/integration_test/extended_test.dart b/packages/integration_test/example/integration_test/extended_test.dart index 7796fc0c97bc..23d69a8f9438 100644 --- a/packages/integration_test/example/integration_test/extended_test.dart +++ b/packages/integration_test/example/integration_test/extended_test.dart @@ -10,7 +10,7 @@ import 'package:integration_test/integration_test.dart'; -import 'extended_test_io.dart' if (dart.library.html) 'extended_test_web.dart' +import '_extended_test_io.dart' if (dart.library.html) '_extended_test_web.dart' as tests; void main() { From 696427a524dc7e69b9cfbb250ec980cbd8675af7 Mon Sep 17 00:00:00 2001 From: Jia Hao Goh Date: Wed, 2 Sep 2020 10:10:57 +0800 Subject: [PATCH 3/4] Run web tests in debug mode --- packages/integration_test/README.md | 6 ++---- packages/integration_test/example/README.md | 3 +-- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/packages/integration_test/README.md b/packages/integration_test/README.md index 4708ed66a2df..5399d527ac9a 100644 --- a/packages/integration_test/README.md +++ b/packages/integration_test/README.md @@ -75,15 +75,13 @@ Make sure you have [enabled web support](https://flutter.dev/docs/get-started/we then [download and run](https://flutter.dev/docs/cookbook/testing/integration/introduction#6b-web) the web driver in another process. -Use following command to execute the tests, passing either `--release` or -`--profile` as debug mode is not supported. +Use following command to execute the tests: ```sh flutter drive \ --driver=test_driver/integration_test_driver.dart \ --target=integration_test/foo_test.dart \ - -d web-server \ - --release + -d web-server ``` ## Android Device Testing diff --git a/packages/integration_test/example/README.md b/packages/integration_test/example/README.md index 8cda23216d6c..7e6317fcf96f 100644 --- a/packages/integration_test/example/README.md +++ b/packages/integration_test/example/README.md @@ -18,6 +18,5 @@ Web: flutter drive \ --driver=test_driver/integration_test_driver.dart \ --target=integration_test/example_test.dart \ - -d web-server \ - --release + -d web-server ``` From e3a3b9b1e50cfa91561cda1f69f1a57ae10373f3 Mon Sep 17 00:00:00 2001 From: Jia Hao Goh Date: Thu, 10 Sep 2020 15:31:21 +0800 Subject: [PATCH 4/4] Add advanced usage of driver --- packages/integration_test/README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/packages/integration_test/README.md b/packages/integration_test/README.md index 5399d527ac9a..a6b1726475e2 100644 --- a/packages/integration_test/README.md +++ b/packages/integration_test/README.md @@ -39,6 +39,11 @@ import 'package:integration_test/integration_test_driver.dart'; Future main() => integrationDriver(); ``` +You can also use different driver scripts to customize the behavior of the app +under test. For example, `FlutterDriver` can also be parameterized with +different [options](https://api.flutter.dev/flutter/flutter_driver/FlutterDriver/connect.html). +See the [extended driver](https://github.com/flutter/plugins/tree/master/packages/integration_test/example/test_driver/integration_test_extended_driver.dart) for an example. + ### Package Structure Your package should have a structure that looks like this: @@ -61,7 +66,8 @@ test_driver/ These tests can be launched with the `flutter drive` command. -To run `integration_test/foo_test.dart`: +To run the `integration_test/foo_test.dart` test with the +`test_driver/integration_test_driver.dart` driver, use the following command: ```sh flutter drive \