diff --git a/docs/dev/api/rdc.md b/docs/dev/api/rdc.md index 07eee28585..dd81c52714 100644 --- a/docs/dev/api/rdc.md +++ b/docs/dev/api/rdc.md @@ -1014,6 +1014,156 @@ curl -u "$SAUCE_USERNAME:$SAUCE_ACCESS_KEY" --location \ --- +### Start a XCTest, XCUITest or Espresso Job + +
+POST /v1/rdc/native-composer/tests +

+ +Start a XCTest. + +#### Parameters + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
test_framework

| BODY | REQUIRED | STRING |

The type of 'native' test you want to run, either 'XCUITEST', 'XCTEST', or 'ANDROID_INSTRUMENTATION' (Espresso).

app_id

| BODY | REQUIRED | STRING |

The id of the app you want to test. The app must be uploaded to Sauce Labs AppStorage. Format: `storage://filename=AnApp.ipa`, or `storage://{file_id}`, or just `{file_id}`.

xc_test_run_file

| BODY | REQUIRED | STRING |

The id of the `xctestrun` file (the xctest config). The file must be uploaded to Sauce Labs AppStorage. Format: `storage://filename=Runner.xctestrun`, or `storage://{file_id}`, or just `{file_id}`.

device_query

| BODY | REQUIRED | OBJECT |

Defines on which device you want to run you test. The available attributes are:

  • type - String of 'DynamicDeviceQuery'
  • device_name - String - regular expression to select a device.
  • os_version - String - regular expression to select a device version.

The type parameter is required..

tunnel_name

| BODY | OPTIONAL | STRING |

Assign a sauce connect tunnel to the job.

tunnel_owner

| BODY | OPTIONAL | STRING |

Define the name of the tunnel owner of the sauce connect tunnel you want to use.

settings_overwrite

| BODY | OPTIONAL | STRING |

Define instrumentations settings you want to apply to your app.

test_name

| BODY | OPTIONAL | STRING |

Assign a name to the job, to be displayed in the UI.

build

| BODY | OPTIONAL | STRING |

Assign the job to a build. You can specify an existing build name or create a new one.

tags

| BODY | OPTIONAL | ARRAY |

The set of tags to apply to the job.

+ + + + + +```jsx title="Sample Request" +curl -u "$SAUCE_USERNAME:$SAUCE_ACCESS_KEY" --location \ +--request POST 'https://api.us-west-1.saucelabs.com/v1/rdc/native-composer/tests' \ +--header 'Content-Type: application/json' +--data-raw '{ + "test_framework": "XCTEST", + "test_name": "Your XCTest POC", + "app_id": "9349d683-5d26-46eb-a943-267cbe8b4deb", + "xc_test_run_file": "37eef454-a7f8-4a6d-a340-d39865ad6db3", + "device_query": { + "type": "DynamicDeviceQuery" + } +}' +``` + + + + + +```jsx title="Sample Request" +curl -u "$SAUCE_USERNAME:$SAUCE_ACCESS_KEY" --location \ +--request POST 'https://api.eu-central-1.saucelabs.com/v1/rdc/native-composer/tests' \ +--header 'Content-Type: application/json' +--data-raw '{ + "test_framework": "XCTEST", + "test_name": "Your XCTest POC", + "app_id": "9349d683-5d26-46eb-a943-267cbe8b4deb", + "xc_test_run_file": "37eef454-a7f8-4a6d-a340-d39865ad6db3", + "device_query": { + "type": "DynamicDeviceQuery" + } +}' +``` + + + + +#### Responses + + + + + + + + + + + + + + +
200Success.
400Bad Request.
+ +```jsx title="Sample Response" +{ + "test_report": { + "id": "ef6058735d8d4dfa9e0077d250757aac", + "url": "https://api.eu-central-1.saucelabs.com/tests/ef6058735d8d4dfa9e0077d250757aac" + } +} +``` + +

+ +--- + ### Stop a Job
diff --git a/docs/mobile-apps/automated-testing/flutter/flutter-integration-testing-android.md b/docs/mobile-apps/automated-testing/flutter/flutter-integration-testing-android.md index 2946b6810c..a974669fce 100644 --- a/docs/mobile-apps/automated-testing/flutter/flutter-integration-testing-android.md +++ b/docs/mobile-apps/automated-testing/flutter/flutter-integration-testing-android.md @@ -72,7 +72,7 @@ Follow this guide to run [integration tests](https://docs.flutter.dev/cookbook/t 5. Create a directory called `integration_test` in the root of your Flutter project. 6. Create a file called `flutter_integration_test.dart` in the `integration_test` directory. 7. Update your testing dart file `flutter_integration_test.dart` to include the ***tearDownAll***, - The purpose for this is to make sure we close the connection to the driver after the tests have completed. + The purpose for this is to make sure we close the connection to the device after the tests have completed. ```dart import 'package:flutter/material.dart'; import 'package:flutter_test/flutter_test.dart'; @@ -118,7 +118,7 @@ Follow this guide to run [integration tests](https://docs.flutter.dev/cookbook/t 9. Configure `saucectl` to run the test. * Create a folder `saucectl` in your project root directory. - * Inside this folder create a `flutter_integration_test.yaml` with the following content: + * Inside this folder create a `flutter_integration_test_android.yaml` with the following content: ```yaml apiVersion: v1alpha kind: espresso @@ -207,12 +207,4 @@ Follow this guide to run [integration tests](https://docs.flutter.dev/cookbook/t ## Example Implementation For a practical example of how to set up and run integration tests for Flutter apps, you can refer to the [Sauce Labs Flutter demo application](https://github.com/saucelabs/my-demo-app-flutter) repository. -The steps outlined in this guide have already been implemented in that repository. You can follow along with the demo app to see how everything is configured and run your tests accordingly. - -## What's Next - -:::info Next step - -We're excited to share that Sauce Labs is actively working on expanding support for Flutter integration tests on iOS. -Stay tuned for updates as we continue to develop this capability! -::: \ No newline at end of file +The steps outlined in this guide have already been implemented in that repository. You can follow along with the demo app to see how everything is configured and run your tests accordingly. \ No newline at end of file diff --git a/docs/mobile-apps/automated-testing/flutter/flutter-integration-testing-ios.md b/docs/mobile-apps/automated-testing/flutter/flutter-integration-testing-ios.md new file mode 100644 index 0000000000..ed406b2d0a --- /dev/null +++ b/docs/mobile-apps/automated-testing/flutter/flutter-integration-testing-ios.md @@ -0,0 +1,105 @@ +--- +id: flutter-integration-testing-ios +title: Flutter iOS +sidebar_label: Flutter iOS +description: Run your Flutter integration tests for iOS +--- + +import useBaseUrl from '@docusaurus/useBaseUrl'; +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +Flutter compiles iOS [integration tests](https://docs.flutter.dev/cookbook/testing/integration/introduction) into [XCTests](https://developer.apple.com/documentation/xctest) so that they can be executed on Apple devices. The following will explain how to run your XCTests on Sauce Labs infrastructure. + +**To run an XCTest (or 'Flutter test') on Sauce Labs, you need to provide two test artifacts:** +1. Your flutter-iOS app compiled as an `.app` or `.ipa` file. +2. The `.xctestrun` file for that app. The [.xctestrun file](https://keith.github.io/xcode-man-pages/xcodebuild.xctestrun.5.html) is the config for your test, this is the same config that Xcode uses when it runs your tests on your development machine. + + +## Contents +1. [How to build the '.app' and '.xctestrun' files for your Flutter app.](#1-how-to-build-the-app-and-xctestrun-files-for-your-flutter-app) +2. [How to run the flutter-iOS integration test on Sauce Labs infrastructure.](#2-how-to-run-flutter-ios-integration-tests-on-sauce-labs-infrastructure) +3. [Sample Implementation](#example-implementation) + + +:::info What You'll Need + +- A Sauce Labs account ([Log in](https://accounts.saucelabs.com/am/XUI/#login/) or sign up for + a [free trial license](https://saucelabs.com/sign-up)) +- Your Sauce Labs [Username and Access Key](https://app.saucelabs.com/user-settings) +- Access to Sauce Labs Real Devices. Sauce Labs only supports XCTests on Real Devices, not virtual. +- Flutter mobile app. If you don't have one, you could use our Flutter Demo App: + - [Sauce Labs Flutter Demo App](https://github.com/saucelabs/my-demo-app-flutter) +- `xcodebuild` tools +- `zip` and/or `saucectl` +::: + + +## 1. How to build the '.app' and '.xctestrun' files for your Flutter app. + +:::note You need to setup your Flutter app for integration tests. + +Before you build your app, you must ensure that you correctly set up the `integration_tests` for your flutter-ios app. You can follow the [flutter documentation](https://github.com/flutter/flutter/tree/main/packages/integration_test#integration_test) to do so. The most relevant section is the part on [iOS Device Testing](https://github.com/flutter/flutter/tree/main/packages/integration_test#ios-device-testing). You can stop following Flutter's guide after you have executed the `xcodebuild build-for-testing` command. This command will generate the `.app` and `.xctestrun` files. +::: + +To execute your xctest, we require your app (which must be packaged together with your XCTests) in '.app' or '.ipa' format. Additionally we need your your `.xctestrun` file, which is the config for your test. + +By default, Xcode will not persist the `.xctestrun` file if you kick off an XCTest on your development machine. To persist the `.xctestrun` file we need to use the `xcodebuild build-for-testing` command. Make sure you are using the correct `scheme` so it includes your integration tests. + +```shell +# Example of the xcodebuild command to build the application. +# You will need to adjust the args according to your app. +output="../build/ios_integ" +xcodebuild build-for-testing \ + -workspace Runner.xcworkspace \ + -scheme Runner \ + -xcconfig Flutter/Release.xcconfig \ + -configuration Release \ + -derivedDataPath \ + $output -sdk iphoneos + +# The .app and .xctestrun files will now be present in your output directory. In this case: `build/ios_integ/Products/Release-iphoneos` +``` + + +## 2. How to run flutter-iOS integration tests on Sauce Labs infrastructure + +To run your Flutter XCTest on Sauce Labs, you have two options: use `saucectl` or integrate with our APIs yourself. If you are unfamiliar with our APIs, we recommend using `saucectl` for ease of use and getting you started quickly. + + +### Run XCTests via saucectl + +First install [saucectl](/docs/dev/cli/saucectl.md#installing-saucectl). **Note: minimum version is `0.192.0`.** Then you can use `saucectl` command to configure and run your test on Sauce Labs infrastructure. + +```shell +# If it's the first time you're using saucectl, run: +saucectl configure + +# follow the steps to configure your XCTest, with your `.app`/`.ipa` file and the `.xctestrun` test config. Use `Real Device` not `Virtual Device` +saucectl init xctest + +# run the newly created XCTest config. +saucectl run +``` + +For further configuration options and info on how to use `saucectl` visit [/docs/mobile-apps/automated-testing/espresso-xcuitest/xcuitest.md](/docs/mobile-apps/automated-testing/espresso-xcuitest/xcuitest.md) + +### Run XCTests without saucectl + +If you prefer not to use saucectl, you can directly integrate with our APIs. + +**First**, compile your `.app` as an `.ipa` file as described [above](/docs/mobile-apps/automated-testing/ipa-files.md#building-an-ipa-from-an-app-bundle). + +**Second**, upload your `.ipa` and `.xctestrun` files to our AppStorage backend, see [AppStorage APIs](/docs/mobile-apps/app-storage.md#upload-apps-via-rest-api). + +**Third**, call our native testing API with the AppStorage IDs of the two files you just uploaded. See [RDC native /test API](/docs/dev/api/rdc.md#start-a-xctest-xcuitest-or-espresso-job). + +**Fourth**, poll the state of the job and wait until the `status` is `passed|failed|error|complete`. You can do this through the [Jobs API](/docs/dev/api/rdc.md#get-a-specific-real-device-job). + + +## Example Implementation + +For a practical example of how to set up and run integration tests for Flutter apps, you can refer to +the [Sauce Labs Flutter demo application](https://github.com/saucelabs/my-demo-app-flutter) repository. +The steps outlined in this guide have already been implemented in that repository. You can follow along with the demo app to see how +everything is configured and run your tests accordingly. diff --git a/docs/mobile-apps/automated-testing/ipa-files.md b/docs/mobile-apps/automated-testing/ipa-files.md index 02c4b734c3..27bf8f5c48 100644 --- a/docs/mobile-apps/automated-testing/ipa-files.md +++ b/docs/mobile-apps/automated-testing/ipa-files.md @@ -35,6 +35,13 @@ In order to disable the resigning process, you must buy your own private devices 3. Compress the `Payload` directory into an archive (.zip file) and give it a new name with .ipa appended to the end of the file name. 4. Your `.ipa` file is now ready for upload to Sauce Labs. +```shell +# example for building an '.ipa' file out of an '.app' +mkdir Payload +cp -r PATH_TO_BUILD_FOLDER/Runner.app Payload +zip -r Runner.ipa Payload +``` + ### Building an .ipa File You can use any of the existing methods of distribution for your iOS app, except for the **App Store** type. This means that you can choose any of the three other export methods: **Ad Hoc**, **Enterprise**, or **Development**. diff --git a/sidebars.js b/sidebars.js index 27648e4443..6165a5afe3 100644 --- a/sidebars.js +++ b/sidebars.js @@ -1438,6 +1438,7 @@ module.exports = { items: [ 'mobile-apps/automated-testing/flutter', 'mobile-apps/automated-testing/flutter/flutter-integration-testing-android', + 'mobile-apps/automated-testing/flutter/flutter-integration-testing-ios', ], }, 'mobile-apps/automated-testing/ipa-files', @@ -1728,7 +1729,7 @@ module.exports = { 'visual-testing/integrations/python', 'visual-testing/integrations/python-robot-framework', 'visual-testing/integrations/playwright', - 'visual-testing/integrations/espresso' + 'visual-testing/integrations/espresso', ], }, 'visual-testing/cli',