Automate Xamarin.Forms testing with Xamarin.UITest and Test Cloud

PDF for offline use:
Sample Code:
Related Links:

Let us know how you feel about this.


0/250
Thanks for the feedback!

last updated: 2016-12

Xamarin Test Cloud's UITest component can be used with Xamarin.Forms to write UI tests to run in the cloud on hundreds of devices.

Overview

Xamarin.UITest allows developers to write automated user interface tests for iOS and Android apps. With some minor tweaks, Xamarin.Forms apps can be tested using Xamarin.UITest, including sharing the same test code. This article introduces specific tips to get Xamarin.UITest working with Xamarin.Forms.

This guide does assume that familiarity with Xamarin.UITest. The following guides are recommended for gaining familiarity with Xamarin.UITest:

Once a UITest project has been added to a Xamarin.Forms solution, the steps for writing and and running the tests for a Xamarin.Forms application are the same as for a Xamarin.Android or Xamarin.iOS application.

Requirements

Xamarin Test Cloud and Xamarin.UITest only support Android and iOS applications. The Windows Phone and Windows versions of Xamarin.Forms apps cannot be tested in Test Cloud.

It is important to understand the concepts described in the Introduction to Xamarin Test Cloud guide.

It is assumed that the most recent stable version of the Xamarin.UITest NuGet Package is installed in the UITest project. It is assume that iOS projects have the most recent version of the Xamarin Test Cloud Agent installed.

In order to run Xamarin.UITests with Xamarin Studio, the following dependencies must be met:
  • NUnit 2.6.x – Xamarin.UITest is not compatible with NUnit 3.x.
  • Android SDK – Only if testing Android apps.
  • Java Developers Kit – Only if testing Android apps.
  • Xcode Command Line Tools – Only for testing iOS apps.
In order to run Xamarin.UITests with Visual Studio, the following dependencies must be met:
  • NUnit 2.6.x – Xamarin.UITest is not compatible with NUnit 3.x.
  • A Test Runner for Visual Studio – A 3rd party test runner, such as the NUnit Test Adapter or Resharper from Jetbrains, is required for Visual Studio to be able to run the NUnit tests.
  • Android SDK – Only if testing Android apps. Windows requires that the ANDROID_HOME environment variable is set with the path to the Android SDK.
  • Java Developers Kit – Only if testing Android apps.

It is not possible to run UITests for iOS apps in Visual Studio or on Windows.

The iOS version of Xamarin.Forms apps cannot be be tested from Windows or Visual Studio.

Adding UITest Support to Xamarin.Forms Apps

UITest automates the user interface by activating controls on the screen and performing input anywhere a user would normally interact with the application. To enable tests that can press a button or enter text in a box the test code will need a way to identify the controls on the screen.

To enable the UITest code to reference controls, each control needs a unique identifier. In Xamarin.Forms, the recommended way to set this identifier is by using the AutomationId property as shown below:

var b = new Button {
    Text = "Click me",
    AutomationId = "MyButton"
};
var l = new Label {
    Text = "Hello, Xamarin.Forms!",
    AutomationId = "MyLabel"
};

The AutomationId property can also be set in XAML:

<Button x:Name="b" AutomationId="MyButton" Text="Click me"/>
<Label x:Name="l" AutomationId="MyLabel" Text="Hello, Xamarin.Forms!" />

A unique AutomationId should be added to all controls that are required for testing (including buttons, text entries, and labels whose value might need to be queried).

iOS Application Project

To run tests on iOS ,the Xamarin Test Cloud Agent NuGet package must be added to the project. Once it has been added, copy the following code into the AppDelegate.FinishedLaunching method:

#if ENABLE_TEST_CLOUD
// requires Xamarin Test Cloud Agent
Xamarin.Calabash.Start();
#endif

The Calabash assembly makes uses of non-public Apple API's which will cause apps to be rejected by the App Store. However, the Xamarin.iOS linker will remove the Calabash assembly from the final IPA if it isn't explicitly referenced from code.

ℹ️

Release builds do not have the ENABLE_TEST_CLOUD compiler variable, which will cause the Calabash assembly to be removed from app bundle. However, debug builds do have the compiler directive defined, preventing the linker from removing the assembly.

The following screenshot shows the ENABLE_TEST_CLOUD compiler variable set for Debug builds:

Android Application Project

Unlike iOS, Android projects do not need any special startup code.

Writing UITests

For information about writing UITests, see Test Cloud UITest documentation. The steps below are a summary, specifically describing how the Xamarin.Forms demo UsingUITest is built.

Use AutomationId in the Xamarin.Forms UI

Before any UITests can be written, the Xamarin.Forms application user interface must be scriptable. Ensure that all controls in the user interface have a AutomationId so that they can be referenced in test code.

Adding a UITest Project to a New Solution

When creating a new Xamarin.Forms project using Xamarin Studio, a new UITest project can be added to the solution by selecting Xamarin Test Cloud: Add an automated UI test project:

The new solution will automatically be configured to run Xamarin.UITests against the Xamarin.Forms application.

Referring to the AutomationId in UITests

When writing UITests, the AutomationId value is exposed differently on each platform:

  • iOS uses the id field.
  • Android uses the label field.

To write cross-platform UITests that will find the AutomationId on both iOS and Android, use the Marked test query:

app.Query(c=>c.Marked("MyButton"))

The shorter form app.Query("MyButton") also works.

Adding a UITest Project to an Existing Solution

It is possible to add a new Xamarin.UITest project to an existing solution manually:

  1. Start by adding a new project by selecting the solution, and clicking File > Add New Project. In the New Project dialog, select Cross-platform > Tests > Xamarin Test Cloud > UI Test App:

    This will add a new project that already has the NUnit and Xamarin.UITest NuGet packages in the solution:

    The new project also has two classes in it. AppInitializer contains code to help initialize and setup tests. The other class, Tests, contains boilerplate code to help start the UITests.

  2. Select View > Pads > Unit Tests to display the Unit Test pad. Expand UsingUITest > UsingUITest.UITests > Test Apps:

  3. Right click on Test Apps, click on Add App Project, and select iOS and Android projects in the dialog that appears:

    The Unit Test pad should now have a reference to the iOS and Android projects. This will allow the Xamarin Studio test runner to execute UITests locally against the two Xamarin.Forms projects.

Adding UITest to the iOS App

There are some additional changes that need to be performed to the iOS application before Xamarin.UITest will work:

  1. Add the Xamarin Test Cloud Agent NuGet package. Right click on Packages, select Add Packages, search NuGet for the Xamarin Test Cloud Agent and add it to the Xamarin.iOS project:

  2. Edit the FinishedLaunching method of the AppDelegate class to initialize the Xamarin Test Cloud Agent when the iOS application starts, and to set the AutomationId property of the views. The FinishedLaunching method should resemble the following code example:

    public override bool FinishedLaunching(UIApplication app, NSDictionary options)
            {
                    #if ENABLE_TEST_CLOUD
                    Xamarin.Calabash.Start();
                    #endif
    
                    global::Xamarin.Forms.Forms.Init();
    
                    LoadApplication(new App());
    
                    return base.FinishedLaunching(app, options);
            }

Visual Studio has a template to help add a Xamarin.UITest project to an existing Xamarin.Forms solution:

  1. Right click on the solution, and select File > New Project.
  2. From the Visual C# Templates, select the Test category. Select the UI Test App > Cross-Platform template:

    This will add a new project with the NUnit, Xamarin.UITest, and NUnitTestAdapter NuGet packages to the solution:

    The NUnitTestAdapter is a third party test runner that allows Visual Studio to run NUnit tests from Visual Studio.

    The new project also has two classes in it. AppInitializer contains code to help initialize and setup tests. The other class, Tests, contains boilerplate code to help start the UITests.

  3. Add a project reference from the UITest project to the Xamarin.Android project:

    This will allow the NUnitTestAdapter to run the UITests for the Android app from Visual Studio.

After adding Xamarin.UITest to the Xamarin.Forms solution, it's possible to create UITests, run them locally, and submit them to Xamarin Test Cloud.

Summary

Xamarin.Forms applications can be easily tested with Xamarin.UITest using a simple mechanism to expose the AutomationId as a unique view identifier for test automation. Once a UITest project has been added to a Xamarin.Forms solution, the steps for writing and and running the tests for a Xamarin.Forms application are the same as for a Xamarin.Android or Xamarin.iOS application.

For information about how to submit tests to Xamarin Test Cloud, see Submitting UITests to Xamarin Test Cloud. For more information about UITest, see Test Cloud documentation.

Xamarin Workbook

If it's not already installed, install the Xamarin Workbooks app first. The workbook file should download automatically, but if it doesn't, just click to start the workbook download manually.