Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs – Part 3

This series was originally posted on Sauce Labs Blog. This is a reblog with revisions.

Here is last installment of  Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs series, which takes the mobile tests running on a local environment and running them on Sauce with Appium. If you missed the earlier parts from this series here are the links

Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs Part 1

Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs Part 2

Running on Sauce Labs

So far we executed features in a local environment. Now this is just one feature, there could be 100s of such and you might want to run these features frequently. Setting up dedicated machine may not be always possible and involves costs. Sauce labs, the company behind Appium provides you an ability to run Appium tests in a virtual environment without needing you to setup everything from scratch. This also saves you costs for setting up your own infrastructure.

But at times you may also want to run these on a local environment, how do we support both running in local environment and sauce?

Let’s use Tags and Hook feature of Cucumber. We already have a setup method which connects to a local Appium sever using the RemoteWebDriver. Let’s add a tag to this method

@Before("local")
public void setUpLocal()  throws Throwable {
	...
}

And add another setup method setUpiOS() and add “ios-sauce” tag to it

@Before("@ios-sauce")
public void setUpiOS()  throws Throwable
{
	DesiredCapabilities capabilities = new DesiredCapabilities();
	capabilities.setCapability(CapabilityType.VERSION, "6.1");
	capabilities.setCapability(CapabilityType.PLATFORM, "Mac");
	capabilities.setCapability("app", "sauce-storage:my_ios_app.zip");
	capabilities.setCapability("device", "iPhone Simulator");
	String sauceUserName = System.getenv("SAUCE_USER_NAME");
	String sauceAccessKey = System.getenv("SAUCE_ACCESS_KEY");

	//Create an instance of RemoteWebDriver and connect to the Appium server.
	//Appium will launch the BmiCalc App in iPhone Simulator using the configurations specified in Desired Capabilities
	driver = new RemoteWebDriver(new URL(MessageFormat.format("http://{0}:{1}@ondemand.saucelabs.com:80/wd/hub", sauceUserName, sauceAccessKey)), capabilities);
}

This allows selectively run features on desired environment. For example if you wish to run scenario on Sauce Labs, then you need to specify the “ios-sauce” tag on the scenario definition in feature file

@ios-sauce
Scenario Outline: Calculate Body Mass Index
…

This will run the feature on Sauce Labs environment. If you specify “local” then it will run the feature on local Appium setup. Here is complete code example

package bmicalculator.test;

import java.net.URL;
import java.text.MessageFormat;

import cucumber.annotation.*;
import cucumber.annotation.en.*;
import static org.junit.Assert.assertEquals;

import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.By;

public class Bmi_Calculator_Step_Defs {

    private WebDriver driver;

    @Before("@local")
    public void setUpLocal()  throws Throwable
    {
        //Setup for running Appium test in local environment
        DesiredCapabilities capabilities = new DesiredCapabilities();
        capabilities.setCapability(CapabilityType.BROWSER_NAME, "iOS");
        capabilities.setCapability(CapabilityType.VERSION, "6.1");
        capabilities.setCapability(CapabilityType.PLATFORM, "Mac");
        capabilities.setCapability("app", "/Users/upgundecha/Desktop/AppExamples/BmiCalculator/build/Release-iphonesimulator/BmiCalculator.app");

        //Create an instance of RemoteWebDriver and connect to the Appium server.
        //Appium will launch the BmiCalc App in iPhone Simulator using the configurations specified in Desired Capabilities
        driver = new RemoteWebDriver(new URL("http://0.0.0.0:4723/wd/hub"), capabilities);
    }

	@Before("@ios-sauce")
	public void setUpiOS()  throws Throwable
	{
		DesiredCapabilities capabilities = new DesiredCapabilities();
		capabilities.setCapability(CapabilityType.VERSION, "6.1");
		capabilities.setCapability(CapabilityType.PLATFORM, "Mac");
		capabilities.setCapability("app", "sauce-storage:my_ios_app.zip");
		capabilities.setCapability("device", "iPhone Simulator");
		String sauceUserName = System.getenv("SAUCE_USER_NAME");
		String sauceAccessKey = System.getenv("SAUCE_ACCESS_KEY");

		//Create an instance of RemoteWebDriver and connect to the Appium server.
		//Appium will launch the BmiCalc App in iPhone Simulator using the configurations specified in Desired Capabilities
		driver = new RemoteWebDriver(new URL(MessageFormat.format("http://{0}:{1}@ondemand.saucelabs.com:80/wd/hub", sauceUserName, sauceAccessKey)), capabilities);
	}

    @Given("^I enter \"([^\"]*)\" as height$")
    public void I_enter_as_height(String height) throws Throwable {
        WebElement heightTextField = driver.findElement(By.name("Height"));
        heightTextField.sendKeys(height);

    }

    @Then("^I enter \"([^\"]*)\" as weight$")
    public void I_enter_as_weight(String weight) throws Throwable {
        WebElement weightTextField = driver.findElement(By.name("Weight"));
        weightTextField.sendKeys(weight);
    }

    @Then("^I press the Calculate button$")
    public void I_press_the_Calculate_button() throws Throwable {
        WebElement calculateButton =  driver.findElement(By.name("Calculate"));
        calculateButton.click();
    }

    @Then("^I should see \"([^\"]*)\" as bmi and \"([^\"]*)\" as category$")
    public void I_should_see_as_bmi_and_Normal_as_category(String bmi, String category) throws Throwable {

        WebElement bmilabel = driver.findElement(By.name("bmi"));
        WebElement bmiCategorylabel = driver.findElement(By.name("category"));

        //Check the calculated Bmi and Category displayed
        assertEquals(bmi,bmilabel.getText());
        assertEquals(category,bmiCategorylabel.getText());
    }

    @After
    public void tearDown()
    {
        driver.quit();
    }
}

Sauce Labs Dashboard

The complete source code for the sample App and test is available at https://github.com/upgundecha/BmiCalculator

Using Continues Integration

Continuous Integration (CI) is becoming a widely accepted practice in agile projects for early feedback. Each integration is verified by an automated build to detect errors as quickly as possible. CI also gives great visibility to all stakeholders about the overall health of the project and its progress.

Jenkins is one of the popular CI tool used by the development teams.  Let’s setup a build and test jobs on Jenkins for this project.

Building the App with Jenkins

For building the App with Jenkins, create a new Build a free-style software project Job. The sample App source is hosted on GitHub. Configure the Git repository and point it to the GitHub repo as show in below screen shot

Jenkins Project Configuration

Next, configure the build trigger, so every time a change is submitted to Git, this job will be trigger a new build of the App. Once the App is built it will be uploaded to the Sauce Labs temporary storage area so the test job can run acceptance tests on the build

Jenkins Build Trigger

Running Acceptance tests with Jenkins

For running acceptance test after a successful build, create a new Build a maven2/3 project job in Jenkins with following parameters

Jenkins Test Project

In the Build Trigger section configure Build after other project are built and specify name of the earlier job. This will automatically run acceptance test (i.e. current job) when the App build is successful

Jenkins Test Project Build Configuration

Cucumber provides a report plugin for Jenkins which create a nicely formatted interactive web reports as shown in below screenshot:

Cucumber Report

Summary

We can write automated acceptance tests for native mobile Apps with Cucumber, Selenium WebDriver API and Appium. These can be run on local, remote or Sauce Labs environment. We can also get benefits of whole team communication and early feedback by applying these methods in mobile application development.

Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs – Part 2

This series was originally posted on Sauce Labs Blog. This is a reblog with revisions.

Welcome to the second part of Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs. In this post we will define steps for the feature we identified in Part 1. We will run this feature with Maven and look at Cucumber results.

Running features with Maven

Before we move forward, we need to configure a support/helper class which tells Cucumber options for running  features that we have added to the project. Add a new Java class RunCukesTest to scr/test/java/bmicalculator.test and copy following code

package bmicalculator.test;

import cucumber.junit.Cucumber;
import org.junit.runner.RunWith;

@RunWith(Cucumber.class)
@Cucumber.Options(format = {"pretty", "html:target/cucumber-html-report", "json-pretty:target/cucumber-report.json"})
public class RunCukesTest {

}

Now let’s run this feature file, in terminal navigate to the project folder and run following command

mvn test

After executing above command, Cucumber will suggest some step definitions for missing steps as below:

@Given("^I enter \"([^\"]*)\" as height$")
public void I_enter_as_height(String arg1) throws Throwable {
 // Express the Regexp above with the code you wish you had
 throw new PendingException();
}

@Given("^I enter \"([^\"]*)\" as weight$")
public void I_enter_as_weight(String arg1) throws Throwable {
 // Express the Regexp above with the code you wish you had
 throw new PendingException();
}

@Given("^I press the Calculate button$")
public void I_press_the_Calculate_button() throws Throwable {
 // Express the Regexp above with the code you wish you had
 throw new PendingException();
}

@Then("^I should see \"([^\"]*)\" as bmi and \"([^\"]*)\" as category$")
public void I_should_see_as_bmi_and_as_category(String arg1, String arg2) throws Throwable {
 // Express the Regexp above with the code you wish you had
 throw new PendingException();
}

Undefined step: Given I enter "170" as height
Undefined step: And I enter "50" as weight
Undefined step: And I press the Calculate button
Undefined step: Then I should see "17.30" as bmi and "Underweight" as category

Undefined step: Given I enter "181" as height
Undefined step: And I enter "80" as weight
Undefined step: And I press the Calculate button
Undefined step: Then I should see "24.42" as bmi and "Normal" as category

Undefined step: Given I enter "180" as height
Undefined step: And I enter "90" as weight
Undefined step: And I press the Calculate button
Undefined step: Then I should see "27.78" as bmi and "Overweight" as category
Undefined step: Given I enter "175" as height

Undefined step: And I enter "100" as weight
Undefined step: And I press the Calculate button
Undefined step: Then I should see "32.65" as bmi and "Obese" as category

1 scenario (0 passed)
16 steps (0 passed)

Implementing Step Definitions

Cucumber drives features & scenarios with automated steps using step definitions. Step definitions are glue between the steps described in a Scenario to the actual automation code which drive the App either through API or User Interface. The App user interface many not be ready at this time.

While development team is busy in developing the App we can create place holders for the steps from the sample scenario as suggested by Cucumber.

Add new Java class Bmi_Calculator_Step_Defs to src/test/java/bmicalculator.test package and implement steps for our example feature. We will mark them as pending with Cucumber PendingExeception() as below:

package bmicalculator.test;

import cucumber.annotation.en.*;
import cucumber.runtime.PendingException;

public class Bmi_Calculator_Step_Defs {

@Given("^I enter \"([^\"]*)\" as height$")
public void I_enter_as_height(String arg1) throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}

@Given("^I enter \"([^\"]*)\" as weight$")
public void I_enter_as_weight(String arg1) throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}

@Given("^I press the Calculate button$")
public void I_press_the_Calculate_button() throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}

@Then("^I should see \"([^\"]*)\" as bmi and \"([^\"]*)\" as category$")
public void I_should_see_as_bmi_and_as_category(String arg1, String arg2) throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}
}

When this feature is executed by Cucumber, it will show these steps as pending.

Once developers are done with the initial version of the App and it’s ready for testing we will refactor these steps and add code to drive the user interface of the App using Appium and Selenium WebDriver API to check the expected behavior of the App.

We need to declare an instance of WebDriver class from Selenium WebDriver API. We will add a setup method setupLocal() which will initialize the driver instance and connect to Appium running on a local machine

@Before
public void setUpLocal()  throws Throwable {
	//Setup for running Appium test in local environment
	DesiredCapabilities capabilities = new DesiredCapabilities();
	capabilities.setCapability(CapabilityType.BROWSER_NAME, "iOS");
	capabilities.setCapability(CapabilityType.VERSION, "6.1");
	capabilities.setCapability(CapabilityType.PLATFORM, "Mac");
	capabilities.setCapability("app", "/Users/upgundecha/Desktop/AppExamples/BmiCalculator/build/Release-iphonesimulator/BmiCalculator.app");

	//Create an instance of RemoteWebDriver and connect to the Appium server.
	//Appium will launch the BmiCalc App in iPhone Simulator using the configurations specified in Desired Capabilities
	driver = new RemoteWebDriver(new URL("http://0.0.0.0:4723/wd/hub"), capabilities);
}

Also implement the steps with Selenium WebDriver API to interact with the App elements in following way

public void I_enter_as_height(String height) throws Throwable {
	WebElement heightTextField = driver.findElement(By.name("Height"));
	heightTextField.sendKeys(height);

}

@Then("^I enter \"([^\"]*)\" as weight$")
public void I_enter_as_weight(String weight) throws Throwable {
	WebElement weightTextField = driver.findElement(By.name("Weight"));
	weightTextField.sendKeys(weight);
}

@Then("^I press the Calculate button$")
public void I_press_the_Calculate_button() throws Throwable {
	WebElement calculateButton =  driver.findElement(By.name("Calculate"));
	calculateButton.click();
}

@Then("^I should see \"([^\"]*)\" as bmi and \"([^\"]*)\" as category$")
public void I_should_see_as_bmi_and_Normal_as_category(String bmi, String category) throws Throwable {

	WebElement bmilabel = driver.findElement(By.name("bmi"));
	WebElement bmiCategorylabel = driver.findElement(By.name("category"));

	//Check the calculated Bmi and Category displayed
	assertEquals(bmi,bmilabel.getText());
	assertEquals(category,bmiCategorylabel.getText());
}

At the end add a tearDown() method to close the App

public void tearDown() {
	driver.quit();
}

This makes the step definitions complete. When this feature is run, Cucumber will call the Selenium WebDriver API which in turn calls the Appium and steps are performed on the App’s user interface in iOS Simulator. At the end of execution Cucumber will display results as below

1 scenario (1 passed)
16 steps (16 passed)

In the next post we will explore how to run Cucumber features with Sauce Labs cloud and Jenkins.