Xamarin.UITest Cheat Sheet

PDF for offline use

Let us know how you feel about this


0/250

last updated: 2016-10

This document is a cheat sheet that condenses UITest information for quick reference, it contains the following topics:

Requirements

Xamarin Test Cloud and Xamarin.UITest only support Android and iOS applications. The Windows Phone and Windows versions of Xamarin.Forms apps cannot be tested in Test Cloud.

It is important to understand the concepts described in the Introduction to Xamarin Test Cloud guide.

It is assumed that the most recent stable version of the Xamarin.UITest NuGet Package is installed in the UITest project. It is assume that iOS projects have the most recent version of the Xamarin Test Cloud Agent installed.

In order to run Xamarin.UITests with Xamarin Studio, the following dependencies must be met:
  • NUnit 2.6.x – Xamarin.UITest is not compatible with NUnit 3.x.
  • Android SDK – Only if testing Android apps.
  • Java Developers Kit – Only if testing Android apps.
  • Xcode Command Line Tools – Only for testing iOS apps.
In order to run Xamarin.UITests with Visual Studio, the following dependencies must be met:
  • NUnit 2.6.x – Xamarin.UITest is not compatible with NUnit 3.x.
  • A Test Runner for Visual Studio – A 3rd party test runner, such as the NUnit Test Adapter or Resharper from Jetbrains, is required for Visual Studio to be able to run the NUnit tests.
  • Android SDK – Only if testing Android apps. Windows requires that the ANDROID_HOME environment variable is set with the path to the Android SDK.
  • Java Developers Kit – Only if testing Android apps.

It is not possible to run UITests for iOS apps in Visual Studio or on Windows.

Preparing the Solution for UITest

Preparing a Solution for Xamarin UITest consists of the following two steps:

  1. Adding a new C# project for the UITests.
  2. Add the Xamarin Test Cloud Agent Nuget package to the Xamarin.iOS projects.

Adding a New Xamarin.UITest Project

The templates for Xamarin.Android, Xamarin.iOS, and Xamarin.Forms projects will add a Xamarin.UITest project to the solution if the Add an automated UI test project option is checked:

In Visual Studio, a Xamarin.UITest project must be added to the solution manually: 1. Right click on the solution, and select File > New Project. 2. From the Visual C# Templates, select the Test category. Select the UI Test App template that is suitable for your application:

This section is provided as a reference for older projects that need to have a project added to them. Xamarin.Forms solutions should consult the Xamarin.Forms and Test Cloud guide for more information about adding UITests.

Xamarin Studio Visual Studio
  1. Add a new **UITest Project** to the solution.
  2. Ensure that the NuGet packages for the new project are up to date.
  1. Add a new **UITest** project to the solution.
  2. Ensure that the NuGet packages for the new project are up to date.
⚠️

UITest requires NUnit 2.6.x; it is not compatible with NUnit 3.0

Initializing Xamarin.UITest on iOS

Add the following snippet to the FinishedLaunching method of the AppDelegate class:

#region Code for starting up the Xamarin Test Cloud Agent

// Newer version of Xamarin Studio and Visual Studio provide the
// ENABLE_TEST_CLOUD compiler directive to prevent the Calabash DLL from
// being included in the released version of the application.
#if ENABLE_TEST_CLOUD
Xamarin.Calabash.Start();
#endif
#endregion

The Calabash DLL makes uses of non public Apple API's which will cause your app to be rejected by the App Store. The Xamarin.iOS linker will remove the Calabash DLL from the final IPA if it not explicitly referenced anywhere by the code. Release builds do not have the ENABLE_TEST_CLOUD compiler variable, which will cause the Calabash DLL to be removed from app bundle. Debug builds will have the compiler directive defined, preventing the linker from removing the Calabash DLL.

Writing Tests

This snippet is a boilerplate TestFixture for a test class on a single platform:

using System;
using NUnit.Framework;
using Xamarin.UITest;
using Xamarin.UITest.Android;
using Xamarin.UITest.Queries;

namespace MyApp.MyUITests
{
    [TestFixture]
    public class Tests
    {
        IApp app;

        [SetUp]
        public void BeforeEachTest()
        {
            // Uncomment the line that is appropriate for the platform:
            // app = ConfigureApp.Android.StartApp();
            // app = ConfigureApp.iOS.StartApp();
        }

        // Test cases here
    }
}

For solutions involving both Android and iOS, the following code will help with writing cross platform UITests.

using System;
using NUnit.Framework;
using Xamarin.UITest;
using Xamarin.UITest.Queries;

namespace MyApp.MyCrossPlatformUITests
{

    public class AppInitializer
    {
        public static IApp StartApp(Platform platform)
        {
            if(platform == Platform.Android)
            {
                return ConfigureApp.Android.StartApp();
            }
            return ConfigureApp.iOS.StartApp();
        }
    }

    [TestFixture(Platform.Android)]
    [TestFixture(Platform.iOS)]
    public class Tests
    {
        IApp app;
        Platform platform;

        public Tests(Platform platform)
        {
            this.platform = platform;
        }

        [SetUp]
        public void BeforeEachTest()
        {
            app = AppInitializer.StartApp(platform);
        }
    }
}

Xamarin.Forms solutions should follow the instructions described in the guide Automating Xamarin.Forms testing with Xamarin.UITest and Test Cloud.

⚠️

Xamarin.UITest requires NUnit 2.6.3 or higher to run tests. Xamarin.UITest is not compatible with NUnit 3.0

Determine the Device ID for an iOS Simulator

You can determine the UUID for the iOS simulators on a computer, use the instruments command as shown below:

$ xcrun instruments -s devices
Known Devices:
bushmaster [5A4B28A1-392A-59FB-81C5-137E881D61E9]
Resizable iPad (8.1 Simulator) [B3BF8A06-2938-4B74-BF87-16C223F8690C]
Resizable iPhone (8.1 Simulator) [E712409B-CFCC-409A-8162-627B6254EB3C]
iPad 2 (7.1 Simulator) [E8572F8F-227B-4DB0-8C92-590DC770360D]
iPad 2 (8.1 Simulator) [1F425263-3F96-4DAB-B843-0D041C3C71EA]
iPad Air (7.1 Simulator) [2863AFF6-D9FC-45E8-8385-E2A548F19002]
iPad Air (8.1 Simulator) [BBCF5CF2-20A4-4C47-9FA5-EBFF7311B071]
iPad Retina (7.1 Simulator) [B7CBB024-E1D3-4B24-8C20-3E9F7B54CF61]
iPad Retina (8.1 Simulator) [3E21ECD3-397A-4251-AEB6-2ADCF29AEE89]
iPhone 4s (7.1 Simulator) [D36354DD-D6A3-4E08-A25B-276620D844B8]
iPhone 4s (8.1 Simulator) [5C8FE602-8BA7-494D-A113-66C8B9AB3CB7]
iPhone 5 (7.1 Simulator) [C696E83D-F9FE-4DBC-8C67-FA0FC533246E]
iPhone 5 (8.1 Simulator) [9A8A5D92-A7D9-4A3C-81AA-97A9924F7D09]
iPhone 5s (7.1 Simulator) [6CDF5B5C-A315-4A8C-9D38-29437FE59C6D]
iPhone 5s (8.1 Simulator) [3F1C286F-3D5D-47B2-92B8-66B673BD0236]
iPhone 6 (8.1 Simulator) [995FF713-9DE4-460B-800E-F5A20FD93AA7]
iPhone 6 Plus (8.1 Simulator) [AB1C20F6-BFFC-4C80-879C-F19A7E3F0B5C]

The UUID for the iOS simulators changes from computer to computer.

Code Snippets

This section will provide some code snippets that can be helpful for writing UITests.

Querying Elements Based on Property Value

//Finds all elements that have a "hint" property with a value of "Search"
app.Query(e => e.All().Property("hint", "Search"));

Enable Screenshots Locally

app = ConfigureApp.Android
    .EnableLocalScreenshots()
    .StartApp();

This will put the screenshots in the directory where the test assembly where the test assemblie resides. The image will be named screenshot-X-png.

Invoke a Method on an AppResult or UI Element

It is possible to execute native methods on underlying views with the AppQuery.Invoke method. The method being invoked must mach the native method name, not the C# method name. For example, to invoke the setGravity method on an Android TextView:

app.Query(e => e.Id("userName").Invoke("setGravity", 1)); //center text

Notice that the Invoke uses Java TextView.setGravity method and not the C# TextView.Gravity property.

Starting a Particular iOS Instance

It is possible to run UITests on a specific iOS version and simulator by using the device ID.

const string simId = "3F1C286F-3D5D-47B2-92B8-66B673BD0236"; //iPhone 5s (8.1 Simulator)
app = ConfigureApp.iOS.DeviceIdentifier(simId).StartApp();

Reset an iOS Simulator to Factory Defaults

This code snippet can be used to stop a given iOS simulator and reset it back to factory defaults:

static void ResetSimulator(string deviceId)
{
    var shutdownProcess = Process.Start("xcrun", string.Format("simctl shutdown {0}", deviceId));
    shutdownProcess.WaitForExit();
    var eraseProcess = Process.Start("xcrun", string.Format("simctl erase {0}", deviceId));
    eraseProcess.WaitForExit();
}

Running Tests

Xamarin.UITests will only run on local devices if the API key is provided. If the API key is not available then tests may only run on a simulator (iOS) or emulator (Android).

ℹ️

UITest can only run iOS tests on OS X.

Running Android 6.0 Applications from the IDE

When the IDE installs an Android 6.0 app for the first time, it does not grant all permissions required by the application. The workaround for this is to use UITest to install and start the application:

ConfigureApp.Android.Debug().ApkFile(apkpath).StartApp ()

Do use ApkFile() and do not use PreferIdeSettings(). This will force UITest to install instead of the IDE.

Running UITests Using the Command Line

It is possible to run UITests at the command line using any NUnit test-runner that is compatible with NUnit 2.6.3 or higher.

⚠️

Mono current ships with and older version of NUnit that does not support all of the NUnit features that UITests may rely on. You are strongly recommend to use the test runner from Nunit 2.6.3 or higher when running UITests at the command line. Note that Xamarin.UITest is not compatible with NUnit 3.0.

This command line shows how to run tests using the nunit-console.exe runner on OS X:

$ mono <path-to>/Nunit-2.6.3/bin/nunit-console.exe <path/to/uitest-assembly.dll>

Submitting Tests To Xamarin Test Cloud

Submitting tests can be performed at the command line. The test-cloud.exe can be found in the directory packages/Xamarin.UITest./tools. It is necessary to have a Test Cloud team API key.

The following snippet shows how to use test-cloud.exe:

$ mono test-cloud.exe submit <YOUR_IPA_OR_APK> <TEST_CLOUD_TEAM_API_KEY> --devices=<DEVICE_ID> --assembly-dir=<PATH_TO_UITEST_ASSEMBLIES> --user=<USER_EMAIL>

Because of the repetitive nature of this, it can be easier to script this.

ℹ️

Remember to recompile the iOS or Android application to ensure that the most recent code is being tested.

Obtain the Xamarin Test Cloud API Key

Log in to Xamarin Test Cloud and create a test run, one for iOS and one for Android. Record the Test Cloud API key and the device IDs for each test run. It is okay to share device ID's but make sure to protect your Xamarin Test Cloud API key.

Sample Bash Script for Submitting Tests

To simplify submitting to Test Cloud, script the steps of compiling the application and uploading to Test Cloud:

#!/bin/sh
# This is a sample build script in Bash.

### This will have to be updated when Xamarin.UITest is updated via NuGet.
export TESTCLOUD=./packages/Xamarin.UITest.0.8.0/tools/test-cloud.exe

### You will have to update these variables for your environment
export XTC_API_KEY=YOUR_TEAM_API_KEY_HERE
export IOS_DEVICE_ID=YOUR_IOS_DEVICE_ID_HERE
export ANDROID_DEVICE_ID=YOUR_ANDROID_DEVICE_ID_HERE
export USER_EMAIL=YOUR_EMAIL_ADDRESS_HERE

### You shouldn't have to update these variables.
export TEST_ASSEMBLIES=PATH_TO_DIRECTORY_HOLDING_THE_UITEST_ASSEMBLY
export APK=PATH_TO_THE_APK_TO_SUBMIT
export IPA=PATH_TO_THE_IPA_TO_SUBMIT

### Uploading the dSYM files is optional - but it can help with troubleshooting
export DSYM=PATH_TO_THE_DSYM

### iOS : build and submit the iOS app for testing
/Applications/Xamarin\ Studio.app/Contents/MacOS/mdtool -v build "--configuration:Debug|iPhone" ./CreditCardValidation.sln
# Submit the IPA to Xamarin Test Cloud
/usr/bin/mono $TESTCLOUD submit $IPA $XTC_API_KEY --devices=$IOS_DEVICE_ID --assembly-dir=$TEST_ASSEMBLIES --dsym=$DSYM --user=$USER_MAIL

### Android: Build and submit the Android app for testing using the default keystore
/Library/Frameworks/Mono.framework/Commands/xbuild /t:Package /p:Configuration=Debug /p:AndroidUseSharedRuntime=false /p:EmbedAssembliesIntoApk=true ./CreditCardValidation.Droid/CreditCardValidation.Droid.csproj
# Submit the APK to Xamarin Test Cloud
/usr/bin/mono $TESTCLOUD submit $APK $XTC_API_KEY --devices=$ANDROID_DEVICE_ID --assembly-dir=$TEST_ASSEMBLIES --user=$USER_EMAIL

Xamarin Workbook

If it's not already installed, install the Xamarin Workbooks app first. The workbook file should download automatically, but if it doesn't, just click to start the workbook download manually.