About these ads

Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs – Part 3

This series was originally posted on Sauce Labs Blog. This is a reblog with revisions.

Here is last installment of  Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs series, which takes the mobile tests running on a local environment and running them on Sauce with Appium. If you missed the earlier parts from this series here are the links

Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs Part 1

Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs Part 2

Running on Sauce Labs

So far we executed features in a local environment. Now this is just one feature, there could be 100s of such and you might want to run these features frequently. Setting up dedicated machine may not be always possible and involves costs. Sauce labs, the company behind Appium provides you an ability to run Appium tests in a virtual environment without needing you to setup everything from scratch. This also saves you costs for setting up your own infrastructure.

But at times you may also want to run these on a local environment, how do we support both running in local environment and sauce?

Let’s use Tags and Hook feature of Cucumber. We already have a setup method which connects to a local Appium sever using the RemoteWebDriver. Let’s add a tag to this method

@Before("local")
public void setUpLocal()  throws Throwable {
	...
}

And add another setup method setUpiOS() and add “ios-sauce” tag to it

@Before("@ios-sauce")
public void setUpiOS()  throws Throwable
{
	DesiredCapabilities capabilities = new DesiredCapabilities();
	capabilities.setCapability(CapabilityType.VERSION, "6.1");
	capabilities.setCapability(CapabilityType.PLATFORM, "Mac");
	capabilities.setCapability("app", "sauce-storage:my_ios_app.zip");
	capabilities.setCapability("device", "iPhone Simulator");
	String sauceUserName = System.getenv("SAUCE_USER_NAME");
	String sauceAccessKey = System.getenv("SAUCE_ACCESS_KEY");

	//Create an instance of RemoteWebDriver and connect to the Appium server.
	//Appium will launch the BmiCalc App in iPhone Simulator using the configurations specified in Desired Capabilities
	driver = new RemoteWebDriver(new URL(MessageFormat.format("http://{0}:{1}@ondemand.saucelabs.com:80/wd/hub", sauceUserName, sauceAccessKey)), capabilities);
}

This allows selectively run features on desired environment. For example if you wish to run scenario on Sauce Labs, then you need to specify the “ios-sauce” tag on the scenario definition in feature file

@ios-sauce
Scenario Outline: Calculate Body Mass Index
…

This will run the feature on Sauce Labs environment. If you specify “local” then it will run the feature on local Appium setup. Here is complete code example

package bmicalculator.test;

import java.net.URL;
import java.text.MessageFormat;

import cucumber.annotation.*;
import cucumber.annotation.en.*;
import static org.junit.Assert.assertEquals;

import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.By;

public class Bmi_Calculator_Step_Defs {

    private WebDriver driver;

    @Before("@local")
    public void setUpLocal()  throws Throwable
    {
        //Setup for running Appium test in local environment
        DesiredCapabilities capabilities = new DesiredCapabilities();
        capabilities.setCapability(CapabilityType.BROWSER_NAME, "iOS");
        capabilities.setCapability(CapabilityType.VERSION, "6.1");
        capabilities.setCapability(CapabilityType.PLATFORM, "Mac");
        capabilities.setCapability("app", "/Users/upgundecha/Desktop/AppExamples/BmiCalculator/build/Release-iphonesimulator/BmiCalculator.app");

        //Create an instance of RemoteWebDriver and connect to the Appium server.
        //Appium will launch the BmiCalc App in iPhone Simulator using the configurations specified in Desired Capabilities
        driver = new RemoteWebDriver(new URL("http://0.0.0.0:4723/wd/hub"), capabilities);
    }

	@Before("@ios-sauce")
	public void setUpiOS()  throws Throwable
	{
		DesiredCapabilities capabilities = new DesiredCapabilities();
		capabilities.setCapability(CapabilityType.VERSION, "6.1");
		capabilities.setCapability(CapabilityType.PLATFORM, "Mac");
		capabilities.setCapability("app", "sauce-storage:my_ios_app.zip");
		capabilities.setCapability("device", "iPhone Simulator");
		String sauceUserName = System.getenv("SAUCE_USER_NAME");
		String sauceAccessKey = System.getenv("SAUCE_ACCESS_KEY");

		//Create an instance of RemoteWebDriver and connect to the Appium server.
		//Appium will launch the BmiCalc App in iPhone Simulator using the configurations specified in Desired Capabilities
		driver = new RemoteWebDriver(new URL(MessageFormat.format("http://{0}:{1}@ondemand.saucelabs.com:80/wd/hub", sauceUserName, sauceAccessKey)), capabilities);
	}

    @Given("^I enter \"([^\"]*)\" as height$")
    public void I_enter_as_height(String height) throws Throwable {
        WebElement heightTextField = driver.findElement(By.name("Height"));
        heightTextField.sendKeys(height);

    }

    @Then("^I enter \"([^\"]*)\" as weight$")
    public void I_enter_as_weight(String weight) throws Throwable {
        WebElement weightTextField = driver.findElement(By.name("Weight"));
        weightTextField.sendKeys(weight);
    }

    @Then("^I press the Calculate button$")
    public void I_press_the_Calculate_button() throws Throwable {
        WebElement calculateButton =  driver.findElement(By.name("Calculate"));
        calculateButton.click();
    }

    @Then("^I should see \"([^\"]*)\" as bmi and \"([^\"]*)\" as category$")
    public void I_should_see_as_bmi_and_Normal_as_category(String bmi, String category) throws Throwable {

        WebElement bmilabel = driver.findElement(By.name("bmi"));
        WebElement bmiCategorylabel = driver.findElement(By.name("category"));

        //Check the calculated Bmi and Category displayed
        assertEquals(bmi,bmilabel.getText());
        assertEquals(category,bmiCategorylabel.getText());
    }

    @After
    public void tearDown()
    {
        driver.quit();
    }
}
Sauce Labs Dashboard

Sauce Labs Dashboard

The complete source code for the sample App and test is available at https://github.com/upgundecha/BmiCalculator

Using Continues Integration

Continuous Integration (CI) is becoming a widely accepted practice in agile projects for early feedback. Each integration is verified by an automated build to detect errors as quickly as possible. CI also gives great visibility to all stakeholders about the overall health of the project and its progress.

Jenkins is one of the popular CI tool used by the development teams.  Let’s setup a build and test jobs on Jenkins for this project.

Building the App with Jenkins

For building the App with Jenkins, create a new Build a free-style software project Job. The sample App source is hosted on GitHub. Configure the Git repository and point it to the GitHub repo as show in below screen shot

Jenkins Project Configuration

Jenkins Project Configuration

Next, configure the build trigger, so every time a change is submitted to Git, this job will be trigger a new build of the App. Once the App is built it will be uploaded to the Sauce Labs temporary storage area so the test job can run acceptance tests on the build

Jenkins Build Trigger

Jenkins Build Trigger

Running Acceptance tests with Jenkins

For running acceptance test after a successful build, create a new Build a maven2/3 project job in Jenkins with following parameters

Jenkins Test Project

Jenkins Test Project

In the Build Trigger section configure Build after other project are built and specify name of the earlier job. This will automatically run acceptance test (i.e. current job) when the App build is successful

Jenkins Test Project Build Configuration

Jenkins Test Project Build Configuration

Cucumber provides a report plugin for Jenkins which create a nicely formatted interactive web reports as shown in below screenshot:

Cucumber Report

Cucumber Report

Summary

We can write automated acceptance tests for native mobile Apps with Cucumber, Selenium WebDriver API and Appium. These can be run on local, remote or Sauce Labs environment. We can also get benefits of whole team communication and early feedback by applying these methods in mobile application development.


Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs – Part 2

This series was originally posted on Sauce Labs Blog. This is a reblog with revisions.

Welcome to the second part of Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs. In this post we will define steps for the feature we identified in Part 1. We will run this feature with Maven and look at Cucumber results.

Running features with Maven

Before we move forward, we need to configure a support/helper class which tells Cucumber options for running  features that we have added to the project. Add a new Java class RunCukesTest to scr/test/java/bmicalculator.test and copy following code


package bmicalculator.test;

import cucumber.junit.Cucumber;
import org.junit.runner.RunWith;

@RunWith(Cucumber.class)
@Cucumber.Options(format = {"pretty", "html:target/cucumber-html-report", "json-pretty:target/cucumber-report.json"})
public class RunCukesTest {

}

Now let’s run this feature file, in terminal navigate to the project folder and run following command


mvn test

After executing above command, Cucumber will suggest some step definitions for missing steps as below:


@Given("^I enter \"([^\"]*)\" as height$")
public void I_enter_as_height(String arg1) throws Throwable {
 // Express the Regexp above with the code you wish you had
 throw new PendingException();
}

@Given("^I enter \"([^\"]*)\" as weight$")
public void I_enter_as_weight(String arg1) throws Throwable {
 // Express the Regexp above with the code you wish you had
 throw new PendingException();
}

@Given("^I press the Calculate button$")
public void I_press_the_Calculate_button() throws Throwable {
 // Express the Regexp above with the code you wish you had
 throw new PendingException();
}

@Then("^I should see \"([^\"]*)\" as bmi and \"([^\"]*)\" as category$")
public void I_should_see_as_bmi_and_as_category(String arg1, String arg2) throws Throwable {
 // Express the Regexp above with the code you wish you had
 throw new PendingException();
}

Undefined step: Given I enter "170" as height
Undefined step: And I enter "50" as weight
Undefined step: And I press the Calculate button
Undefined step: Then I should see "17.30" as bmi and "Underweight" as category

Undefined step: Given I enter "181" as height
Undefined step: And I enter "80" as weight
Undefined step: And I press the Calculate button
Undefined step: Then I should see "24.42" as bmi and "Normal" as category

Undefined step: Given I enter "180" as height
Undefined step: And I enter "90" as weight
Undefined step: And I press the Calculate button
Undefined step: Then I should see "27.78" as bmi and "Overweight" as category
Undefined step: Given I enter "175" as height

Undefined step: And I enter "100" as weight
Undefined step: And I press the Calculate button
Undefined step: Then I should see "32.65" as bmi and "Obese" as category

1 scenario (0 passed)
16 steps (0 passed)

Implementing Step Definitions

Cucumber drives features & scenarios with automated steps using step definitions. Step definitions are glue between the steps described in a Scenario to the actual automation code which drive the App either through API or User Interface. The App user interface many not be ready at this time.

While development team is busy in developing the App we can create place holders for the steps from the sample scenario as suggested by Cucumber.

Add new Java class Bmi_Calculator_Step_Defs to src/test/java/bmicalculator.test package and implement steps for our example feature. We will mark them as pending with Cucumber PendingExeception() as below:

package bmicalculator.test;

import cucumber.annotation.en.*;
import cucumber.runtime.PendingException;

public class Bmi_Calculator_Step_Defs {

@Given("^I enter \"([^\"]*)\" as height$")
public void I_enter_as_height(String arg1) throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}

@Given("^I enter \"([^\"]*)\" as weight$")
public void I_enter_as_weight(String arg1) throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}

@Given("^I press the Calculate button$")
public void I_press_the_Calculate_button() throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}

@Then("^I should see \"([^\"]*)\" as bmi and \"([^\"]*)\" as category$")
public void I_should_see_as_bmi_and_as_category(String arg1, String arg2) throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}
}

When this feature is executed by Cucumber, it will show these steps as pending.

Once developers are done with the initial version of the App and it’s ready for testing we will refactor these steps and add code to drive the user interface of the App using Appium and Selenium WebDriver API to check the expected behavior of the App.

We need to declare an instance of WebDriver class from Selenium WebDriver API. We will add a setup method setupLocal() which will initialize the driver instance and connect to Appium running on a local machine

@Before
public void setUpLocal()  throws Throwable {
	//Setup for running Appium test in local environment
	DesiredCapabilities capabilities = new DesiredCapabilities();
	capabilities.setCapability(CapabilityType.BROWSER_NAME, "iOS");
	capabilities.setCapability(CapabilityType.VERSION, "6.1");
	capabilities.setCapability(CapabilityType.PLATFORM, "Mac");
	capabilities.setCapability("app", "/Users/upgundecha/Desktop/AppExamples/BmiCalculator/build/Release-iphonesimulator/BmiCalculator.app");

	//Create an instance of RemoteWebDriver and connect to the Appium server.
	//Appium will launch the BmiCalc App in iPhone Simulator using the configurations specified in Desired Capabilities
	driver = new RemoteWebDriver(new URL("http://0.0.0.0:4723/wd/hub"), capabilities);
}

Also implement the steps with Selenium WebDriver API to interact with the App elements in following way

public void I_enter_as_height(String height) throws Throwable {
	WebElement heightTextField = driver.findElement(By.name("Height"));
	heightTextField.sendKeys(height);

}

@Then("^I enter \"([^\"]*)\" as weight$")
public void I_enter_as_weight(String weight) throws Throwable {
	WebElement weightTextField = driver.findElement(By.name("Weight"));
	weightTextField.sendKeys(weight);
}

@Then("^I press the Calculate button$")
public void I_press_the_Calculate_button() throws Throwable {
	WebElement calculateButton =  driver.findElement(By.name("Calculate"));
	calculateButton.click();
}

@Then("^I should see \"([^\"]*)\" as bmi and \"([^\"]*)\" as category$")
public void I_should_see_as_bmi_and_Normal_as_category(String bmi, String category) throws Throwable {

	WebElement bmilabel = driver.findElement(By.name("bmi"));
	WebElement bmiCategorylabel = driver.findElement(By.name("category"));

	//Check the calculated Bmi and Category displayed
	assertEquals(bmi,bmilabel.getText());
	assertEquals(category,bmiCategorylabel.getText());
}

At the end add a tearDown() method to close the App

public void tearDown() {
	driver.quit();
}

This makes the step definitions complete. When this feature is run, Cucumber will call the Selenium WebDriver API which in turn calls the Appium and steps are performed on the App’s user interface in iOS Simulator. At the end of execution Cucumber will display results as below


1 scenario (1 passed)
16 steps (16 passed)

In the next post we will explore how to run Cucumber features with Sauce Labs cloud and Jenkins.


Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs – Part 1

This series was originally posted on Sauce Labs Blog This is a reblog with revisions.

Overview

Agile development projects are adopting practices such as automated acceptance testing, BDD, continuous integration, and others. There is a lot written about these approaches for developing software, their benefits and limitations etc. However, one of the fundamental benefits these practices offers is enhanced communication between project stakeholders including users, product owner, and the development team, a shorter feedback loop and agility. These practices requires all the project participants to come together, discuss, and elicit the behavior of the application in agreed upon format of features, user stories and acceptance criteria and shared definition of “Done!”. Read more about these in Wikipedia

ATDD

Mobile application development project can leverage on these advantages in building mobile apps in shorter cycles and feedback loops to build a successful applications. In this series of posts I will explain how you can apply automated acceptance testing to iOS native applications using Appium and Cucumber-JVM. You will also see how these applications can be tested in cloud using Sauce Labs platform.

Cucumber-JVM

Cucumber-JVM is a pure Java implementation of original Cucumber BDD/ATDD framework. It supports the most popular programming languages on the JVM. It’s already been used by various teams along with Selenium WebDriver for testing Web applications. Cucumber support creating features files which are written in a ubiquitous language understood by the whole team. These feature file describe the expected behavior of the application and are used as tests to run against the application. Cucumber can be used for API, integration and functional testing of the application.

In this example we will see functional testing of a sample native iOS App using Cucumber-JVM and Appium.

The Sample App

This example is based on a sample BMI Calculator application which is used by Health & Nutrition specialists to calculate the Body Mass Index of patients by submitting Height and Weight values to the App.

Bmi Calculator App

Bmi Calculator App

Let’s work on one of the main feature of this App as described below

Feature: Calculating Body Mass Index
 As a health specialist
 I want a BMI Calculator
 So that I can calculate patient's Body Mass Index

Setting up Test Project

Let’s setup a new project using with IntelliJ IDEA using Maven & Cucumber-JVM with following steps:

  1. Create a new Project as Maven module, provide appropriate values for <bGroupId and ArtifactId. In this example GroupId is set with org.bmicalc.test value and ArtifactId as bmicalculator.test
  2. Once Intellij IDEA creates project with appropriate folder structure, locate and modify pom.xml file. Add below highlighted dependencies to pom.xml

&amp;lt;?xml version=&amp;quot;1.0&amp;quot; encoding=&amp;quot;UTF-8&amp;quot;?&amp;gt;
&amp;lt;project xmlns=&amp;quot;http://maven.apache.org/POM/4.0.0&amp;quot;
xmlns:xsi=&amp;quot;http://www.w3.org/2001/XMLSchema-instance&amp;quot;
xsi:schemaLocation=&amp;quot;http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd&amp;quot;&amp;gt;
&amp;lt;modelVersion&amp;gt;4.0.0&amp;lt;/modelVersion&amp;gt;
&amp;lt;groupId&amp;gt;org.bmicalc.test&amp;lt;/groupId&amp;gt;
&amp;lt;artifactId&amp;gt;bmicalculator.test&amp;lt;/artifactId&amp;gt;
&amp;lt;version&amp;gt;1.0-SNAPSHOT&amp;lt;/version&amp;gt;
&amp;lt;dependencies&amp;gt;
&amp;lt;dependency&amp;gt;
&amp;lt;groupId&amp;gt;junit&amp;lt;/groupId&amp;gt;
&amp;lt;artifactId&amp;gt;junit&amp;lt;/artifactId&amp;gt;
&amp;lt;version&amp;gt;4.11&amp;lt;/version&amp;gt;
&amp;lt;scope&amp;gt;test&amp;lt;/scope&amp;gt;
&amp;lt;/dependency&amp;gt;
&amp;lt;dependency&amp;gt;
&amp;lt;groupId&amp;gt;org.seleniumhq.selenium&amp;lt;/groupId&amp;gt;
&amp;lt;artifactId&amp;gt;selenium-java&amp;lt;/artifactId&amp;gt;
&amp;lt;version&amp;gt;LATEST&amp;lt;/version&amp;gt;
&amp;lt;scope&amp;gt;test&amp;lt;/scope&amp;gt;
&amp;lt;/dependency&amp;gt;
&amp;lt;dependency&amp;gt;
&amp;lt;groupId&amp;gt;info.cukes&amp;lt;/groupId&amp;gt;
&amp;lt;artifactId&amp;gt;cucumber-java&amp;lt;/artifactId&amp;gt;
&amp;lt;version&amp;gt;1.0.14&amp;lt;/version&amp;gt;
&amp;lt;scope&amp;gt;test&amp;lt;/scope&amp;gt;
&amp;lt;/dependency&amp;gt;
&amp;lt;dependency&amp;gt;
&amp;lt;groupId&amp;gt;info.cukes&amp;lt;/groupId&amp;gt;
&amp;lt;artifactId&amp;gt;cucumber-junit&amp;lt;/artifactId&amp;gt;
&amp;lt;version&amp;gt;1.0.14&amp;lt;/version&amp;gt;
&amp;lt;scope&amp;gt;test&amp;lt;/scope&amp;gt;
&amp;lt;/dependency&amp;gt;
&amp;lt;/dependencies&amp;gt;
&amp;lt;/project&amp;gt;

This will add Cucumber-JVM and Selenium dependencies to the project.

Writing Feature File

In Cucumber-JVM specifications or requirements are expressed in a plain text, Given/When/Then kind of syntax known as Gherkin language (https://github.com/cucumber/cucumber/wiki/Gherkin) which is understood by the whole team. So let’s create a feature file for above feature in the project.

Add a new package bmicalculator.test under src/test/resources as shown in below screenshot and add a new file with name bmi_calculator.feature

Project Folder

Project Folder

Copy following feature text to bmi_calculator.feature file


Feature: Calculating Body Mass Index
  As a health specialist
  I want a BMI Calculator
  So that I can calculate patient's Body Mass Index
  BmiCalculator

  @ios-sauce
  Scenario Outline: Calculate Body Mass Index
    Given I enter &amp;quot;&amp;lt;Height&amp;gt;&amp;quot; as height
    And I enter &amp;quot;&amp;lt;Weight&amp;gt;&amp;quot; as weight
    And I press the Calculate button
    Then I should see &amp;quot;&amp;lt;BMI&amp;gt;&amp;quot; as bmi and &amp;quot;&amp;lt;Category&amp;gt;&amp;quot; as category

  Examples:
    |Height |Weight |BMI  |Category   |
    |170    |50     |17.30|Underweight|
    |181    |80     |24.42|Normal     |
    |180    |90     |27.78|Overweight |
    |175    |100    |32.65|Obese      |

Every feature file contains a single feature. A feature usually contains a list of scenarios.  Every scenario consists of a list of steps, which must start with one of the keywords Given, When, Then, But or And. Scenarios express expected behavior of the system under given conditions.

In addition to a scenario, a feature may contain a background, scenario outline and examples. Our example scenario contains Scenario background and examples for number of BMI calculations representing each category.

This scenario outlines allow us to more concisely express these examples through use of a template with placeholders. In this example Calculate Body Mass Index is run once for each row in the Examples section beneath it (not counting the first row which is a header). This is similar to data driven testing.

In next part, we’ll see how to run features with Maven, and enter step definitions.


PageObject Generator Utility for Selenium WebDriver

Today I saw an interesting tweet lined up in my twitter stream about a Page Recorder utility developed by Dmitry Zhariy which aids in generating PageObjects for Selenium WebDriver tests. I could not resist to get hands-on with this tool and write this post.

I was playing with an idea to build such a utility and someone already done such a good work developing this cool utility. You can read the original blog post about SWD Page Recorder utility here (translated in English)

This project is hosted on GitHub https://github.com/dzhariy/swd-recorder and licensed under The MIT License.

First Impressions

The SWD Page Recorder utility helps users in finding and locating elements as well as creating page objects through a nicely built user interface. You don’t need to juggle around Browsers and tools like Firebug/Developer tools in Google Chrome or IE to find/create locator strategies. This tool allows you to launch various types of Browser and spy on elements, look at their attributes, create & test locators. You can then use this information to generate page objects in various programming languages.

SWD Recorder can be used to test locators just like Selenium IDE on browsers like IE, Chrome and Safari.

This is still in beta phase and have some areas for improvements. Read on the original blog for more details. I played with the utility to create a page object for http://demo.magentocommerce.com/customer/account/login/ with following steps:

Launch the SwdPageRecorder application. On the main Window you need to select & configure Browser that you want to use from Browser Settings tab. It also allows option to connect to RemoteWebDriver instance.

Select the desired Browser and hit Start button to start the Browser instance. By default utility points to http://www.yahoo.com. You can change this by entering desired URL in Browser textbox above Browser Settings tab and click on Go button.

It will navigate to the URL as shown in below screenshot:

Browser Settings

Browser Settings

Switch to Locators tab and click on Start button in In-Browser Web Element Explorer section. Now switch to the Browser instance opened by SWD Page Recorder.

Focus on a desired element in the Browser window and press Ctrl + Right click. This will open a popup window as shown in below screenshot:

Element Information

Element Information

Add a desired element by specifying a logical/descriptive name in Code Identifier textbox and click on Add element button. In this example I will specify emailTextBox in Code identifier textbox

Go on adding elements that are needed for test with above steps.  You can see the elements from the page added to the tree in below screenshot:

Login Page Elements

Login Page Elements

You can also add elements manually or edit elements that are already added by using WebElement section. Elements can be highlighted using Highlight button to test that locator information is sufficient or debug the locator values.

Generating PageObject Code

Once you capture all the elements needed for your PageObject, switch to Source Code tab. The source code tab provides templates for generating PageObject code in various languages (C#, Java, Perl, Python, Ruby etc.). Select a desired template and click on Generate button to generate the code. SWD Page Recorder generated following code the elements added from Login page.

PageObject Code

PageObject Code

You can either copy the code back to the editor or save this in a file and done!

Conclusion

Overall this utility worked pretty good. There are few glitches which I hope should be gone after beta is over. There is a scope for improvement in overall usability of the tool. Along with PageObjects I also want to see utility generating a sort of XML/Properties file based UI-Map.


Testing native Mobile Apps with TestComplete and Appium – An Alternative

[Note] This post was written well before SmartBear released TestComplete 10 with support for testing iOS and Android mobile apps. Please visit the SmartBear TestComplete product page for more details. This post is no longer valid and kept here as a reference.

My day job needs me to work on TestComplete a popular commercial testing tool from SmartBear Software. We do a lot of serious testing using TestComplete for our Desktop applications.

Recently I saw discussions on TestComplete User Forums about possibility of testing mobile apps using TestComplete.  While TestComplete supports testing on wide range of technologies and application platforms including Web and Desktop applications developed on Windows, .NET, WPF, Delphi, C++, Java and so on. However while writing this post it does not have in-built mobile testing capabilities. SmartBear is working on adding mobile testing features which are in beta phase. There are third party tool/plug-in(s) available which use Image Recognition based approach to test the applications. But there is no native support to test iOS or Android Apps.

With these thoughts in my mind, I started to explore if I could mix Appium with TestComplete which will allow testing mobile Apps without leaving the TestComplete environment? With couple hours of hacking, here are the results!

About Appium

Appium is an open source test automation framework for automating native and hybrid mobile apps.
It drives iOS and Android Apps using the WebDriver JSON wire protocol. Appium tests are built on Selenium WebDriver API.  The WebDriver API is used for driving user actions in Browsers, but now it can be used to drive the user interaction on native and hybrid mobile Apps. WebDriver API provides various language bindings which allow you to use them with your choice of programming language and platform, very cool!

How does this works?

TestComplete is a versatile tool and provides supports for more than dozen technologies and extensibility features. TestComplete provides a .NET Bridge which allows access to individual objects of .NET assemblies, their internal properties and methods. This is done by a special dotNET object that provides access to .NET assemblies, types and type members. This feature is available as part of .NET Classes Support plug-in in TestComplete.

Since Appium is driven by WebDriver API, we can connect the WebDriver .NET APIs with TestComplete using the .NET bridge and drive the mobile Apps using Appium. Following diagram provides an overview of this interaction

How to do it?

Setting up the WebDriver .NET API with TestComplete is relatively easy task. Following section describe the detailed steps for driving Appium through TestComplete for testing a sample iOS native App.

Step 1 – Download the WebDriver .NET API Binaries

Download the latest binaries from http://docs.seleniumhq.org/download/. You need to click on Download link in front of C# option in Selenium Client & WebDriver Language Bindings section.

Step 2 – Add WebDriver assemblies to TestComplete CLR Bridge

In next step make the WebDriver assemblies available to TestComplete with following steps. These assemblies will be added to the list in the project’s CLR Bridge settings group. This group contains the list of .NET assemblies whose functions will be available to your scripts through the dotNET object:

  1. Go to TestComplete Project Setting by selecting Current Project Settings from Tools Menu
  2. Select CLR Bridge option on left side of Project Properties Pane
  3. Click Browse Files button to select the WebDriver files. This will display Select Assembly dialog box
  4. On Select Assembly dialog navigate to the net40 folder in WebDriver .NET API files downloaded in Step 1 and select the WebDriver.dll and WebDriver.Support.dll files
  5. Also make sure System.dll is added to CLR Bridge. If it’s not added already, you can add this assembly by Browser GAC button. You will see screen similar to following screen shot after you add all the assemblies to the CLR Bridge
TestComplete CLR Bridge

TestComplete CLR Bridge

This will add WebDriver API to the TestComplete project and provide access to the internal classes and functions using the dotNET object. Assemblies that are added to the CLR Bridge become available in scripts as child objects of the dotNET object. Types defined in these assemblies are available as child of dotNET object. All available assemblies, types, and type members are displayed in the Code Completion window of TestComplete. For more information on dotNET object refer Calling .NET Routines via dotNET Object section in TestComplete Help documentation (also available at http://support.smartbear.com/viewarticle/28697/)

Step 3 – Setting up Appium

Before we start testing iOS or Android apps, we need to setup Appium in a local environment. If you wish to run tests for iOS, then you need to setup Appium on a Mac OSX machine. For testing Android applications you can setup environment on a Windows or Linux machine. Setting up Appium is fairly easy with new Appium App for Mac OSX.

In this example we will test a sample BMI Calculator application for iOS. This is a pretty simple App with bunch of UI elements on a single view. You can get the App source from https://github.com/upgundecha/BmiCalculator.

Let’s setup Appium on a Mac and TestComplete on a Windows machine

  1. On Mac machine download and install the Appium from http://appium.io
  2. Before we launch Appium, download the sample App from https://github.com/upgundecha/BmiCalculator/blob/master/Sample%20App/BmiCalculator.zip,
  3. Unzip BmiCalculator.zip file to get the BmiCalculator.app file
  4. Next, launch Appium on Mac. Select the App Path checkbox and choose the location of the App. Click on Launch button, Appium will launch the application in iOS Simulator as shown in below screenshot, now it’s ready for testing the App
Appium.app running on Mac

Appium.app running on Mac

Step 4 – Writing a test

TestComplete offers multiple language options to write your tests. You can write tests in JavaScript, VBScript, Delphi or C++/C# scripting languages. This example uses JavaScript.

Our test need to tell Appium about what platform is needed for executing the tests. This is done by creating an instance of DesiredCapabilities class of WebDriver API and setting up configuration information in key/value  pair.


//Set up the capabilities to run test on iOS application on local Appium server
Capabilities.setCapability(CapabilityType.BrowserName, "iOS");
Capabilities.setCapability(CapabilityType.Version, "6.1");
Capabilities.setCapability(CapabilityType.Platform, "Mac");

Create an instance of  RemoteWebDriver instance and point it to http://mymacmachine:4723/wd/hub in the test by using Uri class from .NET System namespace and desired capabilities defined earlier. Here mymacmachine could be different in your environment. You can specify Hostname or IP of the Mac machine where Appium is running.


//Set up the RemoteWebDriver
var uri = dotNET.System.Uri.zctor("http://mymacmachine:4723/wd/hub");
var driver = dotNET.OpenQA_Selenium_Remote.RemoteWebDriver.zctor_3(uri,Capabilities);

Here is complete script which tests the BMI calculator functionality (also available as a Gist @ https://gist.github.com/upgundecha/5894126)

function BmiCalculatorTest()
{
  try
  {
    	//Create an instance of DesirecCapabilities, CapabilityType and By Class using CLR Bridge & dotNET Object
    	var Capabilities = dotNET.OpenQA_Selenium_Remote.DesiredCapabilities.zctor();
    	var CapabilityType = dotNET.OpenQA_Selenium_Remote.CapabilityType;
    	var By = dotNET.OpenQA_Selenium.By;

    	//Set up the capabilities to run test on iOS application on local Appium server
    	Capabilities.setCapability(CapabilityType.BrowserName, "iOS");
    	Capabilities.setCapability(CapabilityType.Version, "6.1");
    	Capabilities.setCapability(CapabilityType.Platform, "Mac");

    	//Set up the RemoteWebDriver
    	var uri = dotNET.System.Uri.zctor("http://mymacmachine:4723/wd/hub");

    	var driver = dotNET.OpenQA_Selenium_Remote.RemoteWebDriver.zctor_3(uri,Capabilities);

    	//Get the Hight text field and enter value
    	var HeightField = driver.FindElement(By.Name("Height"));
    	HeightField.SendKeys("181");

    	//Get the Weight text field and enter value
    	var WeightField = driver.FindElement(By.Name("Weight"));
    	WeightField.SendKeys("80");

    	//Get the Calculate button and tap/click on it
    	var CalculateButton = driver.FindElement(By.Name("Calculate"));

    	CalculateButton.Click();

    	//Get the Bmi lables
    	var BmiLabel = driver.FindElement(By.Name("bmi"));
    	var BmiCategoryLabel = driver.FindElement(By.Name("category"));

    	//Check labels for expected values
    	if(aqObject.CompareProperty(BmiLabel.Text, cmpEqual, "24.42")) {
      	  Log.Checkpoint ("Calculated BMI is correct!");
    	}
    	else {
      	  Log.Error( "Calculated BMI does not match expected value");
    	}

    	if(aqObject.CompareProperty(BmiCategoryLabel.Text, cmpEqual, "Normal")) {
      	  Log.Checkpoint( "BMI Category is correct!");
    	}
    	else {
      	  Log.Error( "BMI Category does not match with expected value");
    	}
    	driver.Quit();
  }
  catch(err){
    Log.Error(err.Message);
  }
}

If you prefer TestComplete with VBScript, you can get the VBScript version @ https://gist.github.com/upgundecha/5894141

How this works?

When this test is executed, TestComplete calls the WebDriver API and connects to the Appium server using the RemoteWebDriver. Then it locates the UI elements using FindElement() method of WebDriver API. Test can locate the elements using various locator strategies provided by the By class of WebDriver API. In following example, test is locating Height text field from Bmi Calculator App using its name property. It then enters the specified text using SendKeys() method


//Get the Hight text field and enter value
var HeightField = driver.FindElement(By.Name("Height"));
HeightField.SendKeys("181");

This is pretty much similar to how TestComplete works with UI objects while testing Web or Desktop applications. After inputting the data, TestComplete verifies the expected outcome using its inbuilt verification methods. At the end of execution TestComplete generate a result log as shown in following screenshot

Test Results

Test Results

Running test with Sauce Labs

You may not always get a dedicated hardware to test these Apps. Sauce labs, the company behind Appium, provides you an ability to run Appium tests in a virtual environment without needing you to setup everything from scratch. This also saves you costs for setting up the infrastructure on your own.

So let’s setup and run this test with Sauce Labs. You need a free Sauce Labs account to begin with. Register for a free account on Sauce Labs and get the user name and access key. Sauce Labs provides all the needed hardware and software infrastructure to run your tests in the cloud.

For running tests in Sauce Labs environment, you need to upload your App to Sauce Labs temporary storage using Temporary Storage REST API. You can get the App compiled and built by developers and upload through curl command line utility (Windows users can get curl from http://curl.haxx.se/download.html)


curl -u <sauce_user_name>:<sauce_access_key> -X POST "http://saucelabs.com/rest/v1/storage/$SAUCE_USERNAME/my_ios_app.zip?overwrite=true" -H "Content-Type: application/octet-stream" --data-binary @my_ios_app.zip

We need to modify our test a bit to run on Sauce Labs. Apart from changing the Capabilities and RemoteWebDriver Uri, we need to pass Sauce Labs user name and access key. You can save these into TestComplete project variables. With following changes to earlier test, we can now run tests with Sauce Labs:


var USER_NAME = Project.Variables.SAUCE_USER_NAME
var ACCESS_KEY = Project.Variables.SAUCE_ACCESS_KEY

//Set up the capabilities to run test on iOS application on Sauce Labs
Capabilities.SetCapability(CapabilityType.BrowserName, "iOS 6.0");
Capabilities.SetCapability(CapabilityType.Platform, "Mac 10.8");
Capabilities.SetCapability("device", "iPhone Simulator");
Capabilities.SetCapability("app", "sauce-storage:my_ios_app.zip");
Capabilities.SetCapability("username",USER_NAME);
Capabilities.SetCapability("accessKey",ACCESS_KEY);

//Set up the RemoteWebDriver
var uri = dotNET.System.Uri.zctor("http://" + USER_NAME + ":" + ACCESS_KEY +  "@ondemand.saucelabs.com:80/wd/hub");
var driver = dotNET.OpenQA_Selenium_Remote.RemoteWebDriver.zctor_3(uri,Capabilities);

When you run the test you can login to your Sauce Labs account and see the progress in a Browser window. Based on desired capabilities, Appium provisions an instance of Mac in Sauce Labs cloud and drives the App.

Test Running on Sauce

Test Running on Sauce

You can also view test run in Sauce Labs dashboard with details of each step along with screenshot and video of entire run.

Sauce Dashboard

Sauce Dashboard

Summary

Appium bring exciting possibilities to TestComplete users for testing mobile applications in a platform independent way. These tests can be executed in a local environment or in the cloud or real devices in very a short time, with minimal efforts and costs. This brings more test coverage and faster feedback.


[Book Review] Present Yourself: Using SlideShare to Grow Your Business by Kit Seeborg and Andrea Meyer; O’Reilly Media

Read the rest of this entry »


Four Picks #4

After a long pause I am restarting the “Four Picks” series back again and hope you find it useful. Here is 4th edition of “Four Picks”

  1. Josh Glade has written an excellent article Five Sins of Mobile Testing on Stickyminds. Josh has given some very important tips for building an effective mobile testing strategy and avoid some common mistakes. His point on testing mobile apps on Emulator Software is valid, however emulators also provide first line of defence before apps are tesed on real devices. If you’re interested in learning more about mobile testing, I also recommend reading Jonathan Kohl’s book Tap into Mobile Application Testing I’m currently reading this book. It is packed with some real practical advice on testing mobile apps, I’ll post a review soon.
  2. Testability and Cost of Change I found this post while watching a video lesson. A well researched post with examples from Google. It provides very useful tips on testability and associated costs. There is an interesting example on cost savings with a bug found during TDD (again a solid reason to adopt TDD)
  3. From One Expert to Another: Dale Emery  Dale Emery talks about his recent talk on Test Automation Zombie Apocalypse I loved this presentation, though I do not agree to all points. This provides great stuff to reflect on your test automation efforts and assess the good and bad things. I also recommend reading Tales of Test Automation Gone Wrong by Elisabeth Hendrickson, in fact we built a assessment out of this presentation to check health of our test automation projects.
  4. Mindmaps! Here is a beautiful mind map on testing tools I found on Twitter

 


Follow

Get every new post delivered to your Inbox.

Join 46 other followers