iOS Automation with Appium & Selenium

Note: This post is not up to date with latest release of Appium. An update coming soon…

Yesterday I saw a tweet on Appium release from Sauce Labs and immediately started exploring it. This post summarizes my initial experience with Appium.

Appium (http://appium.io/) is an open source tool/framework for automating iOS Native and Hybrid Apps. It uses the WebDriver JSON wire protocol to drive iOS apps.

Appium server is written in Node.js and talks to iOS using UIAutomation via Instruments. You can use the Selenium WebDriver API for writing tests which talk to Appium via JSON wire protocol for running the Selenium commands. This also gives you advantage of writing tests in your language of preference.

Installation

I found installing Appium quite easy on a local machine. You need Node.js installed before using Appium.

1. Install Node.js from http://nodejs.org/

2. Install WebDriver package for Node.js with the following command

sudo npm install wd

3. Install Appium with the following command

sudo npm install appium -g

4. Start the Appium server with the following command

appium &

Appium server will start at http://localhost:4723

Implementing test using Selenium WebDriver

I am using a sample BMI Calculator App developed with native iOS SDK for this example

Bmi Calculator App
Bmi Calculator App

Build the app using xcodebuild command (In this example the BmiCalc app)

xcodebuild -sdk iphonesimulator6.1

I am using Maven to setup a Java project for this test and here is pom.xml with the following dependencies added. For this example I have used IntelliJ IDEA. For more information on using Maven for Selenium script development refer bonus Chapter Integration with other Tools from my Selenium Testing Tools Cookbook

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.bmicalc.test</groupId>
    <artifactId>bmi-ios-test</artifactId>
    <version>1.0-SNAPSHOT</version>
    <dependencies>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.11</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-java</artifactId>
        <version>LATEST</version>
        <scope>test</scope>
    </dependency>
    </dependencies>
</project>

And here is BmiCalcTest class

import org.junit.After;
import org.junit.Before;
import org.junit.Test;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.By;

import java.io.File;
import java.net.URL;
import java.util.List;

import static org.junit.Assert.assertEquals;

public class BmiCalcTest {

    private WebDriver driver;

    @Before
    public void setUp() throws Exception {

        //Appium needs the path of app build
        //Set up the desired capabilities and pass the iOS SDK version and app path to Appium
        File app = new File("/Users/upgundecha/Desktop/AppExamples/BmiCalculator/build/Release-iphonesimulator/BmiCalculator.app");
        DesiredCapabilities capabilities = new DesiredCapabilities();
        capabilities.setCapability(CapabilityType.BROWSER_NAME, "iOS");
        capabilities.setCapability(CapabilityType.VERSION, "6.1");
        capabilities.setCapability(CapabilityType.PLATFORM, "Mac");
        capabilities.setCapability("app", app.getAbsolutePath());

        //Create an instance of RemoteWebDriver and connect to the Appium server.
        //Appium will launch the BmiCalc App in iPhone Simulator using the configurations specified in Desired Capabilities
        driver = new RemoteWebDriver(new URL("http://localhost:4723/wd/hub"), capabilities);
    }

    @Test
    public void testBmiCalc() throws Exception {

        //iOS controls are accessed through WebElement class
        //Locate the Height & Weight textField by their accessibility labels using By.name()
        WebElement heightTextField = driver.findElement(By.name("Height"));
        heightTextField.sendKeys("181");

        WebElement weightTextField = driver.findElement(By.name("Weight"));
        heightTextField.sendKeys("80");

        //Locate and tap on Calculate button using the click() method
        WebElement calculateButton =  driver.findElement(By.name("Calculate"));
        calculateButton.click();

        //Locate all the label elements using By.tagName()
        List<WebElement> labels = driver.findElements(By.tagName("staticText"));

        //Check the calculated Bmi and Category displayed on labels
        //Label with index 8 has value of the Bmi and index 9 has the value for category
        assertEquals("24.42",labels.get(8).getText());
        assertEquals("Normal",labels.get(9).getText());
    }

    @After
    public void tearDown() throws Exception {
        //Close the app and simulator
        driver.quit();
    }
}

I really liked using Selenium WebDriver API for writing iOS tests with Appium. I can add iOS support to my existing Selenium Framework with minimal changes. Appium presently supports locating elements using the tag name (i.e type of iOS control) and accessibility labels.

Running tests in Cloud

You can also run Appium with Sauce Labs Cloud, for more details read http://sauceio.com/index.php/2013/02/announcing-appium-on-sauce-native-hybrid-ios-testing-in-the-cloud/

Overall Appium is a great tool to start with.

References:
Getting Started – http://appium.io/getting-started.html
Appium on GitHub – http://sauceio.com/index.php/2013/02/announcing-appium-on-sauce-native-hybrid-ios-testing-in-the-cloud/
Samples – https://github.com/appium/appium/tree/master/sample-code
Wiki – https://github.com/appium/appium/wiki
Google Group – https://groups.google.com/forum/?hl=en&fromgroups=#!forum/appium-discuss

Using Google Charts API for Test Results Dashboard

[This post was originally written in May 2008]

Image Charts

I am using Google Charts API to generate charts to visualize test results in Dashboard. Image charts are generated on the fly based on dynamic data and then embedded in to test results generated in HTML format. This feature can be used with any tool/framework where test results are generated in HTML format.  This does not require any software/dependency installed on user’s desktop and comes free from Google.

There are different types of charts you can generate with variety of options. Just hit http://code.google.com/apis/chart/ for API documentation.

Here is an example of Pie and Google-O-Meter types:

<a href="https://unmesh.files.wordpress.com/2012/05/google-charts2.png"><img class="aligncenter size-medium wp-image-333" title="Google Charts" src="https://unmesh.files.wordpress.com/2012/05/google-charts2.png?w=300" alt="" width="300" height="119" /></a>

You can try Pie chart example by simply copying following HTTP request in your Browser’s address bar. A chart will appear in your Browser window once you hit Enter key.

<a href="http://chart.apis.google.com/chart?chtt=Test+Results&amp;chts=000000,12&amp;chs=450x150&amp;chf=bg,s,ffffff&amp;cht=p3&amp;chd=t:66.66,33.33&amp;chl=Tests+Passed(2)|Tests+Failed(1)&amp;chco=006600,990000">http://chart.apis.google.com/chart?chtt=Test+Results&amp;chts=000000,12&amp;chs=450x150&amp;chf=bg,s,ffffff&amp;cht=p3&amp;chd=t:66.66,33.33&amp;chl=Tests+Passed(2)|Tests+Failed(1)&amp;chco=006600,990000</a>

Now, how do we implement this in QTP/VBScript or any other tool of your choice? Here is an example of VBScript function which accepts Number of Test Executed, Passed, and Failed and then generates a nicely formatted HTML report:

Public Function GenerateTestSummaryReport(ByVal intTestsExecuted, ByVal intTestsPassed, ByVal intTestsFailed)

	Set objFSO = CreateObject("Scripting.FileSystemObject")
	Set objFile = objFSO.CreateTextFile("C:\TestSummary.html",True)

	objFile.WriteLine "<!DOCTYPE html PUBLIC '-//W3C//DTD XHTML 1.0 Transitional//EN' 'http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'>"
	objFile.WriteLine "<html xmlns='http://www.w3.org/1999/xhtml' >"
	objFile.WriteLine "<head><title>Test Execution Summary Report</title></head>"
	objFile.WriteLine "<body style='font-weight: bold; font-size: 10pt; color: black; font-family: Tahoma'>"
	objFile.WriteLine "<br />Test Execution Summary"
	objFile.WriteLine "<br /><br />Test Suite Summary<table><tr><td style='width: 200px; height: 56px; background-color: #669918'>Total Tests Executed</td><td style='height: 56px; background-color: #669918'>Passed</td><td style='height: 56px; background-color: #669918'>Failed</td></tr>"
	objFile.WriteLine "<tr><td style='width: 200px; background-color: #D0F0A2'>" & intTestsExecuted & "</td><td style='background-color: #D0F0A2'>" & intTestsPassed & "</td><td style='background-color: #D0F0A2'>" & intTestsFailed & "</td></tr>"
	objFile.WriteLine "<tr><td><img src='http://chart.apis.google.com/chart?chtt=Test+Results&amp;chts=000000,12&amp;chs=450x150&amp;chf=bg,s,ffffff&amp;cht=p3&amp;chd=t:" & intTestsPassed & "," & intTestsFailed & "&amp;chl=Tests+Passed(" & intTestsPassed &")|Tests+Failed(" & intTestsFailed & ")&amp;chco=006600,990000' alt='Test Results Chart'/></td></tr>"
	objFile.WriteLine"</table>"

	Set objFile = Nothing
	Set objFSO = Nothing

End Function

'Call Function
Call GenerateTestSummaryReport(5,3,2)

In above code, highlighted IMG tag will make a request to Google Chart API and will show a nicely built chart on the fly when user opens this file in a Web Browser or email client.

If you find API documentation not working for you, there is a very interesting service available at http://dexautomation.com/googlechartgenerator.php

This will enable you to design Charts using guided steps and then it will generate IMG tag which you can embed in your code. Using Google Charting API you could build a nice dashboard for you test suite result reporting.

However Image charts have some limitation on amount of data we can pass. Other alternative is Interactive Chart feature.

Interactive Charts

Unlike image charts which are static, Interactive are charts more advanced. Interactive charts support user interaction and event handling. These are built using JavaScript.

Google Visualization API provides different types of charts for data visualization. Please see following page for sample charts.

http://code.google.com/apis/visualization/documentation/gallery.html

Here is an example in VBScript on how to use Gauge charts for showing test results/build status:

Public Function GenerateTestSummaryReport(ByVal intTestsPassed, ByVal intTestsFailed)
	Set objFSO = CreateObject("Scripting.FileSystemObject")
	Set objFile = objFSO.CreateTextFile("C:\TestSummary.html",True)
	objFile.WriteLine "<html><head><title> Test Execution Summary Report </title>"
	objFile.WriteLine "<script type='text/javascript' src='http://www.google.com/jsapi'></script>"
	objFile.WriteLine "<script type='text/javascript'>"
	objFile.WriteLine "google.load('visualization', '1', {packages:['gauge']});"
	objFile.WriteLine "google.setOnLoadCallback(drawChart);"
	objFile.WriteLine "function drawChart() { var passedData = new google.visualization.DataTable();"
	objFile.WriteLine "passedData.addColumn('string', 'Label'); passedData.addColumn('number', 'Value');"
	objFile.WriteLine "passedData.addRows(1); passedData.setValue(0, 0, 'Tests Passed');"
	objFile.WriteLine "passedData.setValue(0, 1, " & intTestsPassed & ");"
	objFile.WriteLine "var passedChart = new google.visualization.Gauge(document.getElementById('chart_passed'));"
	objFile.WriteLine "var passedOptions = {width: 400, height: 120, redFrom: 0, redTo: 30, yellowFrom: 31, yellowTo: 60, greenFrom: 61, greenTo: 100, minorTicks: 5};"
	objFile.WriteLine "passedChart.draw(passedData, passedOptions);"
	objFile.WriteLine "var failedData = new google.visualization.DataTable();"
	objFile.WriteLine "failedData.addColumn('string', 'Label');"
	objFile.WriteLine "failedData.addColumn('number', 'Value');"
	objFile.WriteLine "failedData.addRows(1);"
	objFile.WriteLine "failedData.setValue(0, 0, 'Tests Failed');"
	objFile.WriteLine "failedData.setValue(0, 1, " & intTestsFailed & ");"
	objFile.WriteLine "var failedChart = new google.visualization.Gauge(document.getElementById('chart_failed'));"
	objFile.WriteLine "var failedOptions = {width: 400, height: 120, redFrom: 31, redTo: 100, yellowFrom: 11, yellowTo: 30, greenFrom: 0, greenTo: 10, minorTicks: 5};"
	objFile.WriteLine "failedChart.draw(failedData, failedOptions);}"
	objFile.WriteLine "</script></head>"
	objFile.WriteLine "<body style='font-weight: bold; font-size: 10pt; color: black; font-family: Tahoma'><br />Test Execution Summary"
	objFile.WriteLine "<br /><br />Test Suite Summary<table><tr><td style='width: 200px; height: 56px; background-color: #669918'>Total Tests Executed</td><td style='height: 56px; background-color: #669918'>Passed</td><td style='height: 56px; background-color: #669918'>Failed</td></tr>"
	objFile.WriteLine "<tr><td style='width: 200px; background-color: #D0F0A2'>100</td><td style='background-color: #D0F0A2'>85</td><td style='background-color: #D0F0A2'>15</td></tr>"
	objFile.WriteLine "<tr><td><div id='chart_passed'></div></td><td><div id='chart_failed'></div></td></tr></table></body></html>"
	Set objFile = Nothing
	Set objFSO = Nothing
End Function

'Call Function
Call GenerateTestSummaryReport(15,85)

Here is interactive Chart Generated:

You can add more metrics to your reports and build more meaningful Charts.

Recording Screencast of Selenium Tests

I was looking for ways to record a video/screencast of Selenium Test Run in Java and came across this brilliant tool called Monte Media Library developed by Werner Randelshofer. This post describes using the ScreenRecorder class from Monte Media Library for recording screencast of Selenium Tests in Java.

Little about ScreenRecorder

ScreenRecoder supports “AVI” and “QuickTime” format for recording the video. For “AVI” format you need to install TSCC Codec (Techsmith Screen Capture Codec) while “QuickTime” format is supported by Apple’s QuickTime Player. ScreenRecorder provides multiple configurations for colors, mouse cursor, screen rate, mouse rate, audio etc. on GUI as well as programmatically.

You need to download ScreenRecorder.jar file from Monte’s Home Page. ScreenRecorder.jar can be launched as a standalone GUI for recording actions from Desktop window or you can add this to your project and import ScreenRecorder class for recording screen video programmatically.

Using ScreenRecorder Class

Following example is created in Eclipse and you need to add ScreenRecorder.jar to the build path of Project.

ScreenRecorder.jar contains ScreenRecorder class which can be called from a Selenium Script for recording the test session in following way:

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.support.ui.ExpectedCondition;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.monte.media.math.Rational;
import org.monte.media.Format;
import org.monte.screenrecorder.ScreenRecorder;
import static org.monte.media.AudioFormatKeys.*;
import static org.monte.media.VideoFormatKeys.*;
import java.awt.*;

public class SeScreenCastDemo {
	
	public static void main(String[] args) throws Exception {

		//Create a instance of GraphicsConfiguration to get the Graphics configuration
		//of the Screen. This is needed for ScreenRecorder class.
		GraphicsConfiguration gc = GraphicsEnvironment//
		.getLocalGraphicsEnvironment()//
		.getDefaultScreenDevice()//
		.getDefaultConfiguration();

		//Create a instance of ScreenRecorder with the required configurations
		ScreenRecorder screenRecorder = new ScreenRecorder(gc,
		new Format(MediaTypeKey, MediaType.FILE, MimeTypeKey, MIME_AVI),
		new Format(MediaTypeKey, MediaType.VIDEO, EncodingKey, ENCODING_AVI_TECHSMITH_SCREEN_CAPTURE,
		CompressorNameKey, ENCODING_AVI_TECHSMITH_SCREEN_CAPTURE,
		DepthKey, (int)24, FrameRateKey, Rational.valueOf(15),
		QualityKey, 1.0f,
		KeyFrameIntervalKey, (int) (15 * 60)),
		new Format(MediaTypeKey, MediaType.VIDEO, EncodingKey,"black",
		FrameRateKey, Rational.valueOf(30)),
		null);

	//Create a new instance of the Firefox driver
	WebDriver driver = new FirefoxDriver();

	//Call the start method of ScreenRecorder to begin recording
	screenRecorder.start();

	//And now use this to visit Google
	driver.get("http://www.google.com");

	//Find the text input element by its name
	WebElement element = driver.findElement(By.name("q"));

	//Enter something to search for
	element.sendKeys("Cheese!");

	//Now submit the form. WebDriver will find the form for us from the element
	element.submit();

	//Check the title of the page
	System.out.println("Page title is: " + driver.getTitle());

	//Google's search is rendered dynamically with JavaScript.
	//Wait for the page to load, timeout after 10 seconds
	(new WebDriverWait(driver, 10)).until(new ExpectedCondition<Boolean>() {
	public Boolean apply(WebDriver d) {
	return d.getTitle().toLowerCase().startsWith("cheese!");
	}});

	//Should see: "cheese! - Google Search"
	System.out.println("Page title is: " + driver.getTitle());

	//Close the browser
	driver.quit();

	//Call the stop method of ScreenRecorder to end the recording
	screenRecorder.stop();
	}
}

The ScreenRecorder class will create recordings in the home folder i.e. “Videos” on Windows, and “Movies” on Mac OS X by default. The ScreenRecorder class needs first argument as GraphicsConfiguration. You can get this by accessing GraphicsEnvironment member of AWT class by following way:

GraphicsConfiguration gc = GraphicsEnvironment//
 .getLocalGraphicsEnvironment()//
 .getDefaultScreenDevice()//
 .getDefaultConfiguration();

The ScreenRecorder captures screen interactions like a charm which can be very useful for analysing Selenium Tests.

Update:

Same can be achieved with Selenium .NET Binding (in C#/VB.NET) by using the Microsoft Expression Encoder 4 SDK. You can download the Encoder SDK from http://www.microsoft.com/expression/products/encoder4_overview.aspx. Use the ScreenCaptureJob class for recording video of Selenium Tests in C#/VB.NET.

Update:
This code has been tested on ScreenRecorder Version 0.6.6

Using XPath Axes for locating elements in Selenium

XPath is a very powerful and widely used mechanism in Selenium for locating elements on a Web Page. XPath is used for locating nodes in an XML document and it can also be used for locating HTML elements in XHTML. Selenium supports XPath along with various other locator strategies. XPath in Selenium extends beyond the simple methods of locating by id or name attributes, and opens up all sorts of new possibilities for locating complex & dynamically rendered elements.

Developers not always follow best practices or testability guidelines while building the applications. XPath comes to your help when you don’t have a suitable id or name attribute for the element you wish to locate. You can use XPath to either locate the element in absolute terms or relative to an element that does have an id or name attribute. XPath locators can also be used to specify elements via attributes other than id and name.

Recently a team approached me for solving a locator problem in Selenium. They were testing an e-commerce web application. They had a difficulty in selecting & adding an item to shopping cart. The table listing the items was rendered dynamically using dynamic ids and complex HTML structure. Following figure shows the structure of page:

ShoppingCart
Shopping Cart Page

The textbox to enter the Quantity and image to add item was buried under layer of div elements inside a td element in a table. The ids for these elements were generated dynamically and it was difficult to add an item dynamically from the test script. Here is html code for Quantity textbox:

For Product 1


<input type="text" id="count_670756" value="" maxlength="3" size="3" name="qty">

For Product 2


<input type="text" id="count_670759" value="" maxlength="3" size="3" name="qty">

As you can see in the above code, we cannot rely on id attribute as it changes every time page is refreshed and secondly name of textbox is not unique.

XPath Axes comes to a great help to resolve this problem and develop a generic location strategy. In simple terms XPath Axes helps to locate elements based on element’s relationship with other elements in a document. For more information, there is a nice tutorial available on W3Schools on XPath & XPath Axes.

Coming back to the problem, first we need to find a unique way to identify a Product in the table. There are two columns which contain unique values namely Product & Article column. However Article column contains primary key from the database so it is highly recommended to use this value as basis to locate the product. We can also use Product column otherwise. Following XPath Query will locate the cell containing the specified Article using XPath functions contains() & text():


xpath=//td[contains(text(),'0002')]

Now we need to find the cell which contains the elements. Here we will use following-sibling axis and find the third cell from the current cell. Following query will return the cell containing Quantity & Add to cart image:


xpath=//td[contains(text(),'0002')]/following-sibling::td[3]

In next step we need to get the actual elements which are located inside the layer of div elements. Here we need to use descendent axis to find the child elements in the cell:


xpath=//td[contains(text(),'0002')]/following-sibling::td[3]/descendant::div[2]/input[@name='qty']

xpath=//td[contains(text(),'0002')]/following-sibling::td[3]/descendant::div[3]/input[@name='add']

Following screen-shot shows the Selenium IDE commands to enter the Quantity and click on the Shopping Cart image:

seleide
Selenium IDE

These actions will be used to add multiple items in various test cases. We need to make these actions more generic and remove the hardcoded Article Id. Following example shows the custom command implemented in Selenium User Extension and Selenium IDE:


Selenium.prototype.doAddItemToShoppingCart = function(locator, value){

                this.doType("xpath=//td[contains(text(),'"+locator+"')]/following-sibling::td[3]/descendant::div[2]/input[@name='qty']", value);

                this.doClick("xpath=//td[contains(text(),'"+locator+"')]/following-sibling::td[3]/descendant::div[3]/input[@name='add']");

}

custcommand
Custom Command in Selenium IDE

We can also create reusable function around these actions in following way for Selenium RC in Java:


public void addItemToShoppingCart(String ArticleId, String Qty) throws Exception
	{
		selenium.type("xpath=//td[contains(text(),'" + ArticleId + "')]/following-sibling::td[3]/descendant::div[2]/input[@name='qty']", Qty);
		selenium.click("xpath=//td[contains(text(),'" + ArticleId + "')]/following-sibling::td[3]/descendant::div[3]/input[@name='add']");
	}

However while using XPath we need to consider the fact that these locators are dependent on structure of the page and the layout of the elements. Any changes to the structure or layout will affect the locators and tests will result into failures.

Update:

Sometimes XPath queries may not work well or tests run slow in browsers that do not have good support for XPaths. CSS selectors is a good alternative in such situations. CSS selectors are considerably faster than XPath. Selenium supports CSS 1.0, 2.0 & 3.0. Following code sample describes CSS selectors for locating the elements from above example:


css=td:contains('0002')+td+td+td>div>form>div>input[name='qty']
css=td:contains('0002')+td+td+td>div>form>div>input[name='add']