Automated Acceptance testing for Mainframe with Cucumber and Jagacy

 

fullsizerender-1I often get questions around Mainframe Automation using Selenium. However Selenium automates browsers, that’s it! Selenium does not automate Mainframe Green screens, and it’s completely different technology.

Automating Mainframe Green screen is primarily needed to test front to back scenarios in complex transaction processing systems with Web and Mobile integration.

There are tools available that can be used to automate Mainframe Green screen interaction. In this post, we will see how to use Jagacy3270 from Jagacy Software along with Cucumber for writing Automated Acceptance Tests on Mainframe Green Screens also known as CICS interface.

About Jagacy3270

Jagacy3270 is a 3270 screen-scraping library written entirely in Java. As described on Jagacy product website it supports SSL, TN3270E, Internationalization, and over thirty languages. This library can be used to create highly reliable and faster screen-scraping applications. However, this tool comes with a cost and details are available on the website.

We can use this same screen-scraping capability to create automated tests on Mainframe Green screens.

Jagacy provides Session3270 class through which we can connect to a Mainframe host and read or write to screens to perform actions.

Writing Automated Acceptance Tests for Green Screens

We can use Jagacy along with testing frameworks like Cucumber in Agile Software development to automate acceptance criteria for Mainframe user stories involving Green screen interaction. These tests can be integrated into CI/CD pipeline and run in headless mode (Jagacy provides an Emulator screen mode or headless mode).

In this post, we will use a Mainframe host used in Jagacy examples to create a simple Automated Acceptance test for a Phonebook application. Here is our example user story

As I university mainframe user
I should be able to search faculty members in Phonebook
So I can contact them for help

Here is Feature file with one of the acceptance criteria or scenario that searches for Faculty phone number on Phonebook application:

Feature: Phonebook

  As I university mainframe user
  I should be able to search faculty members in Phonebook
  So I can contact them for help

  Scenario: Search faculty phone number using name
    Given I start a new emulator session
    When I open phonbook application
    And search for faculty name "NAME"
    Then I should see the results matching with my search criteria
      |NAME1              111-111-1111  LIBRARIES                    5000|
      |NAME2             UNAVAILABLE  MARY KAY O'CONNOR PROCESS    3122|

To automate Screen entry and retrieve values, we need to use screen coordinates by using Row and Column numbers (Typically 3270 sessions are 24×80 rows and columns long).

We can use PageObject Pattern to abstract Mainframe Green screens and provide screen actions and state to tests. For example, here is HomeScreen object which provides feature on the Home Screen:

package com.example.screens;

import com.example.Fields.EntryField;
import com.example.session.Session;
import com.example.Fields.LabelField;
import com.jagacy.Key;
import com.jagacy.util.JagacyException;

/**
 * Created by upgundecha on 14/10/16.
 */
public class HomeScreen {

    private Session session;
    private String screenCrc = "0xb0c10358";

    // Screen fields
    private LabelField waitForLabel =
            new LabelField(17, 6, "TEXAS A & M UNIVERSITY");
    private EntryField applicationEntryField = new EntryField(23, 1);

    public HomeScreen(final Session s) throws JagacyException {
        this.session = s;
        if (!session.waitForTextLabel(waitForLabel)) {
            throw new IllegalStateException("Not Home screen!");
        }

        if (session.getCrc32() != Long.decode(screenCrc)) {
            throw new IllegalStateException("Home Screen has been changed!");
        }
    }

    /**
     * Open Phonbook Menu screen.
     * @return Phonbook Menu Screen
     * @throws JagacyException JagacyException
     */
    public final PhonbookMenuScreen openPhonbook() throws JagacyException {
        session.setEntryFieldValue(applicationEntryField, "PHONBOOK");
        session.writeKey(Key.ENTER);
        session.waitForChange(10000);
        return new PhonbookMenuScreen(session);
    }
}

We can check if a correct page is displayed in the emulator by using the arbitrary text displayed on the screen. We can also use CRC of the screen to make sure it is not updated otherwise tests might fail as fields or text values on screen are not found at specified locations.

Finally, we will call the PageObjects in step definitions to perform the scenario and validate the output:

package com.example.test;

import com.example.screens.HomeScreen;
import com.example.screens.PhonbookMenuScreen;
import com.example.screens.PhonbookSearchScreen;
import com.example.session.Session;
import com.jagacy.util.JagacyException;
import cucumber.api.Scenario;
import cucumber.api.java.After;
import cucumber.api.java.Before;
import cucumber.api.java.en.Given;
import cucumber.api.java.en.Then;
import cucumber.api.java.en.When;

import static org.junit.Assert.*;

import java.util.List;


/**
 * Created by upgundecha on 14/10/16.
 */
public class Stepdefs {

    private Session session;
    private HomeScreen homeScreen;
    private PhonbookMenuScreen phonbookMenuScreen;
    private PhonbookSearchScreen phonbookSearchScreen;
    private Scenario scenario;

    @Before
    public void setUp(Scenario scenario){
        this.scenario = scenario;
    }

    @Given("^I start a new emulator session$")
    public void i_start_a_new_emulator_session() throws Throwable {

        session = new Session("test");
        session.open();

    }

    @When("^I open phonbook application$")
    public void i_open_phonbook_application() throws Throwable {

        homeScreen = new HomeScreen(session);
        scenario.embed(session.getScreenshot(), "image/png");
        phonbookMenuScreen = homeScreen.openPhonbook();

    }

    @When("^search for faculty name \"([^\"]*)\"$")
    public void search_for_faculty_name(String q) throws Throwable {

        scenario.embed(session.getScreenshot(), "image/png");
        phonbookSearchScreen = phonbookMenuScreen.openFacultyStaffListing();
        phonbookSearchScreen.searchByFirstOrMiddleName(q);

    }

    @Then("^I should see the results matching with my search criteria$")
    public void i_should_see_the_results_matching_with_my_search_criteria(List<String> records) throws Throwable {

        scenario.embed(session.getScreenshot(), "image/png");
        assertEquals(records, phonbookSearchScreen.getResults());

    }

    @After
    public void tearDown() throws JagacyException {
        session.close();
    }
}

We can also get the screenshot (not available as part of Jagacy API) embedded in the reports.

And if you already have a well established Selenium framework (on JVM) and want to automate Mainframe Green screens, you can use above approach with Cucumber or any other xUnit testing framework that you might be using.

The complete source code for this post is available here. Please drop me a message for access with your Github username. I have built this simple framework on top of Jagacy API. If you need any further information or have suggestions, please do reach out to me.


Setting up minimal Selenium Grid with Docker

Here’s simple guide to setup a minimal Selenium Grid with Docker. For running Docker on your machine you will need Docker toolbox installed from https://www.docker.com/products/docker-toolbox. Below steps are done on a Mac.

We will use Hub and Node images from Selenium project hosted at Docker Hub https://hub.docker.com/r/selenium/

Next we need to create a docker-compose file describing how we want to run the Selenium Grid Hub and connect nodes to the Hub. In this example we will launch a multi-container setup with a Hub connected to Firefox and Chrome nodes:

If you don’t have Docker running, then start the Docker daemon with default machine by using following command:

docker-machine start default

To connect to the Docker shell run following command:

docker-machine env

and then:

eval $(docker-machine env)

This will connect the terminal session to the Docker shell

Finally run the docker-compose command from the directory where docker-compose.yml file is stored:

docker-compose up

This will get required images from the Docker hub and launch the Hub node followed by Firefox and Chrome nodes which will be registered to the Hub. Now we have a minimal Selenium Grid up and running. We can point Selenium tests to this Grid for execution. In the next post we’ll see some advanced options and integration with Maven and Cloud tools.


Using Tesseract with Selenium WebDriver for checking text on images using OCR

Recently a team approached me looking for a solution to extract text from an image displayed on a web page and verify it’s contents as part of Selenium tests.

This post explains the solution using Tesseract, Tess4J along with Selenium for checking text displayed on images.

Tesseract is a famous open source OCR engine. It uses the Leptonica Image Processing Library. Tesseract support a wide variety of image formats and convert them to text in over 60 languages.

Tesseract works on Linux, Windows and Mac OSX. Please refer Readme page for installation instructions.

This sample is built on Mac. You can install Tesseract on Mac using homebrew:

brew install tesseract

In addition to Tesseract (written in C++), we need a Java wrapper called Tess4J which provides JNA wrapper for Tesseract OCR API.

Here is a sample page which has a barcode displayed as image. We will extract the barcode number and assert it’s value.

ocr_example

Since I am using Maven for this project, I added Tess4j dependency to my pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>me.unmesh</groupId>
    <artifactId>selenium-ocr-example</artifactId>
    <version>1.0-SNAPSHOT</version>
    <dependencies>
        <dependency>
            <groupId>net.sourceforge.tess4j</groupId>
            <artifactId>tess4j</artifactId>
            <version>2.0.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>2.46.0</version>
        </dependency>
    </dependencies>
</project>

Here’s JUnit test which navigates to the sample page and checks the number displayed on the barcode image:

package me.unmesh.selenium.ocr.example;

import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.By;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import static org.junit.Assert.*;

import net.sourceforge.tess4j.*;
import java.io.File;

/**
 * A demo test to verify text from an image using Tesseract OCR API
 *
 * @author  upgundecha
 *
 */
public class BarcodeTest {
    private WebDriver driver;

    @Before
    public void setUp() {
        driver = new FirefoxDriver();
        // navigate to the dummy page with a barcode image
        driver.get("https://dl.dropboxusercontent.com/u/55228056/barcode.html");
    }

    @After
    public void tearDown() {
        driver.quit();
    }

    @Test
    public void testBarcodeNumber() throws Exception {
        // get and capture the picture of the img element used to display the barcode image
        WebElement barcodeImage = driver.findElement(By.id("barcode"));
        File imageFile = WebElementExtender.captureElementPicture(barcodeImage);

        // get the Tesseract direct interace
        Tesseract instance = new Tesseract();

        // the doOCR method of Tesseract will retrive the text
        // from image captured by Selenium
        String result = instance.doOCR(imageFile);

        // check the the result
        assertEquals("Application number did not match", "123-45678", result.trim());
    }
}

Instead of capturing screenshot of the entire page using Selenium, I captured screenshot of the image element where the barcode is displayed on the page.

<html>
  <head>
    <title>Barcode Sample</title>
   </head>
  <body>
    <table>
      <tr>
        <td style="padding:10px; font-size:15px; font-family:Arial, Helvetica; text-align:center;">
          <p> Please write down your application id</p>
        <td>
          <img id="barcode" src="barcode.png" />
        </td>
      </tr>
  </table>
  </body>
 </html>

The captured image is then passed to doOCR() method of Tesseract instance to retrieve the text.

To capture the image of a WebElement I used captureElementPicture() method from WebElementExtender class which is described in my book Selenium Testing Tools Cookbook:

package me.unmesh.selenium.ocr.example;

import java.awt.Rectangle;
import java.awt.image.BufferedImage;
import java.io.File;

import javax.imageio.ImageIO;

import org.openqa.selenium.OutputType;
import org.openqa.selenium.Point;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.internal.WrapsDriver;

/**
 * This class provides various additional helper methods on elements
 *
 * @author upgundecha
 *
 */

public class WebElementExtender {

    /**
     * Gets a picture of specific element displayed on the page
     * @param element The element
     * @return File
     * @throws Exception
     */
    public static File captureElementPicture(WebElement element)
            throws Exception {

        // get the WrapsDriver of the WebElement
        WrapsDriver wrapsDriver = (WrapsDriver) element;

        // get the entire screenshot from the driver of passed WebElement
        File screen = ((TakesScreenshot) wrapsDriver.getWrappedDriver())
                .getScreenshotAs(OutputType.FILE);

        // create an instance of buffered image from captured screenshot
        BufferedImage img = ImageIO.read(screen);

        // get the width and height of the WebElement using getSize()
        int width = element.getSize().getWidth();
        int height = element.getSize().getHeight();

        // create a rectangle using width and height
        Rectangle rect = new Rectangle(width, height);

        // get the location of WebElement in a Point.
        // this will provide X & Y co-ordinates of the WebElement
        Point p = element.getLocation();

        // create image  for element using its location and size.
        // this will give image data specific to the WebElement
        BufferedImage dest = img.getSubimage(p.getX(), p.getY(), rect.width,
                rect.height);

        // write back the image data for element in File object
        ImageIO.write(dest, "png", screen);

        // return the File object containing image data
        return screen;
    }
}

Tesseract is clean, fast and accurate for OCR testing needs. Similar approach can be followed for .NET using Emgu library


Implementing BDD/ATDD with TestComplete and SpecFlow.NET

TestComplete is a popular functional testing tool from SmartBear with support for testing Desktop, Web and Mobile Application. It supports more than dozen technology platforms, tools and number of third party controls.

SpecFlow.NET is popular BDD framework available on .NET platform. It integrates with Visual Studio. You can write Feature in 100% Gherkin with SpecFlow and automation code in C#.

In this video I’ll show you how to integrate SpecFlow.NET and TestComplete in Visual Studio for implementing BDD/ Automated Acceptance tests


Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs – Part 3

This series was originally posted on Sauce Labs Blog. This is a reblog with revisions.

Here is last installment of  Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs series, which takes the mobile tests running on a local environment and running them on Sauce with Appium. If you missed the earlier parts from this series here are the links

Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs Part 1

Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs Part 2

Running on Sauce Labs

So far we executed features in a local environment. Now this is just one feature, there could be 100s of such and you might want to run these features frequently. Setting up dedicated machine may not be always possible and involves costs. Sauce labs, the company behind Appium provides you an ability to run Appium tests in a virtual environment without needing you to setup everything from scratch. This also saves you costs for setting up your own infrastructure.

But at times you may also want to run these on a local environment, how do we support both running in local environment and sauce?

Let’s use Tags and Hook feature of Cucumber. We already have a setup method which connects to a local Appium sever using the RemoteWebDriver. Let’s add a tag to this method

@Before(&quot;local&quot;)
public void setUpLocal()  throws Throwable {
	...
}

And add another setup method setUpiOS() and add “ios-sauce” tag to it

@Before(&quot;@ios-sauce&quot;)
public void setUpiOS()  throws Throwable
{
	DesiredCapabilities capabilities = new DesiredCapabilities();
	capabilities.setCapability(CapabilityType.VERSION, &quot;6.1&quot;);
	capabilities.setCapability(CapabilityType.PLATFORM, &quot;Mac&quot;);
	capabilities.setCapability(&quot;app&quot;, &quot;sauce-storage:my_ios_app.zip&quot;);
	capabilities.setCapability(&quot;device&quot;, &quot;iPhone Simulator&quot;);
	String sauceUserName = System.getenv(&quot;SAUCE_USER_NAME&quot;);
	String sauceAccessKey = System.getenv(&quot;SAUCE_ACCESS_KEY&quot;);

	//Create an instance of RemoteWebDriver and connect to the Appium server.
	//Appium will launch the BmiCalc App in iPhone Simulator using the configurations specified in Desired Capabilities
	driver = new RemoteWebDriver(new URL(MessageFormat.format(&quot;http://{0}:{1}@ondemand.saucelabs.com:80/wd/hub&quot;, sauceUserName, sauceAccessKey)), capabilities);
}

This allows selectively run features on desired environment. For example if you wish to run scenario on Sauce Labs, then you need to specify the “ios-sauce” tag on the scenario definition in feature file

@ios-sauce
Scenario Outline: Calculate Body Mass Index
…

This will run the feature on Sauce Labs environment. If you specify “local” then it will run the feature on local Appium setup. Here is complete code example

package bmicalculator.test;

import java.net.URL;
import java.text.MessageFormat;

import cucumber.annotation.*;
import cucumber.annotation.en.*;
import static org.junit.Assert.assertEquals;

import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.By;

public class Bmi_Calculator_Step_Defs {

    private WebDriver driver;

    @Before(&quot;@local&quot;)
    public void setUpLocal()  throws Throwable
    {
        //Setup for running Appium test in local environment
        DesiredCapabilities capabilities = new DesiredCapabilities();
        capabilities.setCapability(CapabilityType.BROWSER_NAME, &quot;iOS&quot;);
        capabilities.setCapability(CapabilityType.VERSION, &quot;6.1&quot;);
        capabilities.setCapability(CapabilityType.PLATFORM, &quot;Mac&quot;);
        capabilities.setCapability(&quot;app&quot;, &quot;/Users/upgundecha/Desktop/AppExamples/BmiCalculator/build/Release-iphonesimulator/BmiCalculator.app&quot;);

        //Create an instance of RemoteWebDriver and connect to the Appium server.
        //Appium will launch the BmiCalc App in iPhone Simulator using the configurations specified in Desired Capabilities
        driver = new RemoteWebDriver(new URL(&quot;http://0.0.0.0:4723/wd/hub&quot;), capabilities);
    }

	@Before(&quot;@ios-sauce&quot;)
	public void setUpiOS()  throws Throwable
	{
		DesiredCapabilities capabilities = new DesiredCapabilities();
		capabilities.setCapability(CapabilityType.VERSION, &quot;6.1&quot;);
		capabilities.setCapability(CapabilityType.PLATFORM, &quot;Mac&quot;);
		capabilities.setCapability(&quot;app&quot;, &quot;sauce-storage:my_ios_app.zip&quot;);
		capabilities.setCapability(&quot;device&quot;, &quot;iPhone Simulator&quot;);
		String sauceUserName = System.getenv(&quot;SAUCE_USER_NAME&quot;);
		String sauceAccessKey = System.getenv(&quot;SAUCE_ACCESS_KEY&quot;);

		//Create an instance of RemoteWebDriver and connect to the Appium server.
		//Appium will launch the BmiCalc App in iPhone Simulator using the configurations specified in Desired Capabilities
		driver = new RemoteWebDriver(new URL(MessageFormat.format(&quot;http://{0}:{1}@ondemand.saucelabs.com:80/wd/hub&quot;, sauceUserName, sauceAccessKey)), capabilities);
	}

    @Given(&quot;^I enter \&quot;([^\&quot;]*)\&quot; as height$&quot;)
    public void I_enter_as_height(String height) throws Throwable {
        WebElement heightTextField = driver.findElement(By.name(&quot;Height&quot;));
        heightTextField.sendKeys(height);

    }

    @Then(&quot;^I enter \&quot;([^\&quot;]*)\&quot; as weight$&quot;)
    public void I_enter_as_weight(String weight) throws Throwable {
        WebElement weightTextField = driver.findElement(By.name(&quot;Weight&quot;));
        weightTextField.sendKeys(weight);
    }

    @Then(&quot;^I press the Calculate button$&quot;)
    public void I_press_the_Calculate_button() throws Throwable {
        WebElement calculateButton =  driver.findElement(By.name(&quot;Calculate&quot;));
        calculateButton.click();
    }

    @Then(&quot;^I should see \&quot;([^\&quot;]*)\&quot; as bmi and \&quot;([^\&quot;]*)\&quot; as category$&quot;)
    public void I_should_see_as_bmi_and_Normal_as_category(String bmi, String category) throws Throwable {

        WebElement bmilabel = driver.findElement(By.name(&quot;bmi&quot;));
        WebElement bmiCategorylabel = driver.findElement(By.name(&quot;category&quot;));

        //Check the calculated Bmi and Category displayed
        assertEquals(bmi,bmilabel.getText());
        assertEquals(category,bmiCategorylabel.getText());
    }

    @After
    public void tearDown()
    {
        driver.quit();
    }
}
Sauce Labs Dashboard

Sauce Labs Dashboard

The complete source code for the sample App and test is available at https://github.com/upgundecha/BmiCalculator

Using Continues Integration

Continuous Integration (CI) is becoming a widely accepted practice in agile projects for early feedback. Each integration is verified by an automated build to detect errors as quickly as possible. CI also gives great visibility to all stakeholders about the overall health of the project and its progress.

Jenkins is one of the popular CI tool used by the development teams.  Let’s setup a build and test jobs on Jenkins for this project.

Building the App with Jenkins

For building the App with Jenkins, create a new Build a free-style software project Job. The sample App source is hosted on GitHub. Configure the Git repository and point it to the GitHub repo as show in below screen shot

Jenkins Project Configuration

Jenkins Project Configuration

Next, configure the build trigger, so every time a change is submitted to Git, this job will be trigger a new build of the App. Once the App is built it will be uploaded to the Sauce Labs temporary storage area so the test job can run acceptance tests on the build

Jenkins Build Trigger

Jenkins Build Trigger

Running Acceptance tests with Jenkins

For running acceptance test after a successful build, create a new Build a maven2/3 project job in Jenkins with following parameters

Jenkins Test Project

Jenkins Test Project

In the Build Trigger section configure Build after other project are built and specify name of the earlier job. This will automatically run acceptance test (i.e. current job) when the App build is successful

Jenkins Test Project Build Configuration

Jenkins Test Project Build Configuration

Cucumber provides a report plugin for Jenkins which create a nicely formatted interactive web reports as shown in below screenshot:

Cucumber Report

Cucumber Report

Summary

We can write automated acceptance tests for native mobile Apps with Cucumber, Selenium WebDriver API and Appium. These can be run on local, remote or Sauce Labs environment. We can also get benefits of whole team communication and early feedback by applying these methods in mobile application development.


Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs – Part 2

This series was originally posted on Sauce Labs Blog. This is a reblog with revisions.

Welcome to the second part of Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs. In this post we will define steps for the feature we identified in Part 1. We will run this feature with Maven and look at Cucumber results.

Running features with Maven

Before we move forward, we need to configure a support/helper class which tells Cucumber options for running  features that we have added to the project. Add a new Java class RunCukesTest to scr/test/java/bmicalculator.test and copy following code


package bmicalculator.test;

import cucumber.junit.Cucumber;
import org.junit.runner.RunWith;

@RunWith(Cucumber.class)
@Cucumber.Options(format = {"pretty", "html:target/cucumber-html-report", "json-pretty:target/cucumber-report.json"})
public class RunCukesTest {

}

Now let’s run this feature file, in terminal navigate to the project folder and run following command


mvn test

After executing above command, Cucumber will suggest some step definitions for missing steps as below:


@Given("^I enter \"([^\"]*)\" as height$")
public void I_enter_as_height(String arg1) throws Throwable {
 // Express the Regexp above with the code you wish you had
 throw new PendingException();
}

@Given("^I enter \"([^\"]*)\" as weight$")
public void I_enter_as_weight(String arg1) throws Throwable {
 // Express the Regexp above with the code you wish you had
 throw new PendingException();
}

@Given("^I press the Calculate button$")
public void I_press_the_Calculate_button() throws Throwable {
 // Express the Regexp above with the code you wish you had
 throw new PendingException();
}

@Then("^I should see \"([^\"]*)\" as bmi and \"([^\"]*)\" as category$")
public void I_should_see_as_bmi_and_as_category(String arg1, String arg2) throws Throwable {
 // Express the Regexp above with the code you wish you had
 throw new PendingException();
}

Undefined step: Given I enter "170" as height
Undefined step: And I enter "50" as weight
Undefined step: And I press the Calculate button
Undefined step: Then I should see "17.30" as bmi and "Underweight" as category

Undefined step: Given I enter "181" as height
Undefined step: And I enter "80" as weight
Undefined step: And I press the Calculate button
Undefined step: Then I should see "24.42" as bmi and "Normal" as category

Undefined step: Given I enter "180" as height
Undefined step: And I enter "90" as weight
Undefined step: And I press the Calculate button
Undefined step: Then I should see "27.78" as bmi and "Overweight" as category
Undefined step: Given I enter "175" as height

Undefined step: And I enter "100" as weight
Undefined step: And I press the Calculate button
Undefined step: Then I should see "32.65" as bmi and "Obese" as category

1 scenario (0 passed)
16 steps (0 passed)

Implementing Step Definitions

Cucumber drives features & scenarios with automated steps using step definitions. Step definitions are glue between the steps described in a Scenario to the actual automation code which drive the App either through API or User Interface. The App user interface many not be ready at this time.

While development team is busy in developing the App we can create place holders for the steps from the sample scenario as suggested by Cucumber.

Add new Java class Bmi_Calculator_Step_Defs to src/test/java/bmicalculator.test package and implement steps for our example feature. We will mark them as pending with Cucumber PendingExeception() as below:

package bmicalculator.test;

import cucumber.annotation.en.*;
import cucumber.runtime.PendingException;

public class Bmi_Calculator_Step_Defs {

@Given("^I enter \"([^\"]*)\" as height$")
public void I_enter_as_height(String arg1) throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}

@Given("^I enter \"([^\"]*)\" as weight$")
public void I_enter_as_weight(String arg1) throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}

@Given("^I press the Calculate button$")
public void I_press_the_Calculate_button() throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}

@Then("^I should see \"([^\"]*)\" as bmi and \"([^\"]*)\" as category$")
public void I_should_see_as_bmi_and_as_category(String arg1, String arg2) throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}
}

When this feature is executed by Cucumber, it will show these steps as pending.

Once developers are done with the initial version of the App and it’s ready for testing we will refactor these steps and add code to drive the user interface of the App using Appium and Selenium WebDriver API to check the expected behavior of the App.

We need to declare an instance of WebDriver class from Selenium WebDriver API. We will add a setup method setupLocal() which will initialize the driver instance and connect to Appium running on a local machine

@Before
public void setUpLocal()  throws Throwable {
	//Setup for running Appium test in local environment
	DesiredCapabilities capabilities = new DesiredCapabilities();
	capabilities.setCapability(CapabilityType.BROWSER_NAME, "iOS");
	capabilities.setCapability(CapabilityType.VERSION, "6.1");
	capabilities.setCapability(CapabilityType.PLATFORM, "Mac");
	capabilities.setCapability("app", "/Users/upgundecha/Desktop/AppExamples/BmiCalculator/build/Release-iphonesimulator/BmiCalculator.app");

	//Create an instance of RemoteWebDriver and connect to the Appium server.
	//Appium will launch the BmiCalc App in iPhone Simulator using the configurations specified in Desired Capabilities
	driver = new RemoteWebDriver(new URL("http://0.0.0.0:4723/wd/hub"), capabilities);
}

Also implement the steps with Selenium WebDriver API to interact with the App elements in following way

public void I_enter_as_height(String height) throws Throwable {
	WebElement heightTextField = driver.findElement(By.name("Height"));
	heightTextField.sendKeys(height);

}

@Then("^I enter \"([^\"]*)\" as weight$")
public void I_enter_as_weight(String weight) throws Throwable {
	WebElement weightTextField = driver.findElement(By.name("Weight"));
	weightTextField.sendKeys(weight);
}

@Then("^I press the Calculate button$")
public void I_press_the_Calculate_button() throws Throwable {
	WebElement calculateButton =  driver.findElement(By.name("Calculate"));
	calculateButton.click();
}

@Then("^I should see \"([^\"]*)\" as bmi and \"([^\"]*)\" as category$")
public void I_should_see_as_bmi_and_Normal_as_category(String bmi, String category) throws Throwable {

	WebElement bmilabel = driver.findElement(By.name("bmi"));
	WebElement bmiCategorylabel = driver.findElement(By.name("category"));

	//Check the calculated Bmi and Category displayed
	assertEquals(bmi,bmilabel.getText());
	assertEquals(category,bmiCategorylabel.getText());
}

At the end add a tearDown() method to close the App

public void tearDown() {
	driver.quit();
}

This makes the step definitions complete. When this feature is run, Cucumber will call the Selenium WebDriver API which in turn calls the Appium and steps are performed on the App’s user interface in iOS Simulator. At the end of execution Cucumber will display results as below


1 scenario (1 passed)
16 steps (16 passed)

In the next post we will explore how to run Cucumber features with Sauce Labs cloud and Jenkins.


Automated Acceptance Testing iOS Mobile Apps with Appium, Cucumber-JVM, Jenkins, and Sauce Labs – Part 1

This series was originally posted on Sauce Labs Blog This is a reblog with revisions.

Overview

Agile development projects are adopting practices such as automated acceptance testing, BDD, continuous integration, and others. There is a lot written about these approaches for developing software, their benefits and limitations etc. However, one of the fundamental benefits these practices offers is enhanced communication between project stakeholders including users, product owner, and the development team, a shorter feedback loop and agility. These practices requires all the project participants to come together, discuss, and elicit the behavior of the application in agreed upon format of features, user stories and acceptance criteria and shared definition of “Done!”. Read more about these in Wikipedia

ATDD

Mobile application development project can leverage on these advantages in building mobile apps in shorter cycles and feedback loops to build a successful applications. In this series of posts I will explain how you can apply automated acceptance testing to iOS native applications using Appium and Cucumber-JVM. You will also see how these applications can be tested in cloud using Sauce Labs platform.

Cucumber-JVM

Cucumber-JVM is a pure Java implementation of original Cucumber BDD/ATDD framework. It supports the most popular programming languages on the JVM. It’s already been used by various teams along with Selenium WebDriver for testing Web applications. Cucumber support creating features files which are written in a ubiquitous language understood by the whole team. These feature file describe the expected behavior of the application and are used as tests to run against the application. Cucumber can be used for API, integration and functional testing of the application.

In this example we will see functional testing of a sample native iOS App using Cucumber-JVM and Appium.

The Sample App

This example is based on a sample BMI Calculator application which is used by Health & Nutrition specialists to calculate the Body Mass Index of patients by submitting Height and Weight values to the App.

Bmi Calculator App

Bmi Calculator App

Let’s work on one of the main feature of this App as described below

Feature: Calculating Body Mass Index
 As a health specialist
 I want a BMI Calculator
 So that I can calculate patient's Body Mass Index

Setting up Test Project

Let’s setup a new project using with IntelliJ IDEA using Maven & Cucumber-JVM with following steps:

  1. Create a new Project as Maven module, provide appropriate values for <bGroupId and ArtifactId. In this example GroupId is set with org.bmicalc.test value and ArtifactId as bmicalculator.test
  2. Once Intellij IDEA creates project with appropriate folder structure, locate and modify pom.xml file. Add below highlighted dependencies to pom.xml

&amp;lt;?xml version=&amp;quot;1.0&amp;quot; encoding=&amp;quot;UTF-8&amp;quot;?&amp;gt;
&amp;lt;project xmlns=&amp;quot;http://maven.apache.org/POM/4.0.0&amp;quot;
xmlns:xsi=&amp;quot;http://www.w3.org/2001/XMLSchema-instance&amp;quot;
xsi:schemaLocation=&amp;quot;http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd&amp;quot;&amp;gt;
&amp;lt;modelVersion&amp;gt;4.0.0&amp;lt;/modelVersion&amp;gt;
&amp;lt;groupId&amp;gt;org.bmicalc.test&amp;lt;/groupId&amp;gt;
&amp;lt;artifactId&amp;gt;bmicalculator.test&amp;lt;/artifactId&amp;gt;
&amp;lt;version&amp;gt;1.0-SNAPSHOT&amp;lt;/version&amp;gt;
&amp;lt;dependencies&amp;gt;
&amp;lt;dependency&amp;gt;
&amp;lt;groupId&amp;gt;junit&amp;lt;/groupId&amp;gt;
&amp;lt;artifactId&amp;gt;junit&amp;lt;/artifactId&amp;gt;
&amp;lt;version&amp;gt;4.11&amp;lt;/version&amp;gt;
&amp;lt;scope&amp;gt;test&amp;lt;/scope&amp;gt;
&amp;lt;/dependency&amp;gt;
&amp;lt;dependency&amp;gt;
&amp;lt;groupId&amp;gt;org.seleniumhq.selenium&amp;lt;/groupId&amp;gt;
&amp;lt;artifactId&amp;gt;selenium-java&amp;lt;/artifactId&amp;gt;
&amp;lt;version&amp;gt;LATEST&amp;lt;/version&amp;gt;
&amp;lt;scope&amp;gt;test&amp;lt;/scope&amp;gt;
&amp;lt;/dependency&amp;gt;
&amp;lt;dependency&amp;gt;
&amp;lt;groupId&amp;gt;info.cukes&amp;lt;/groupId&amp;gt;
&amp;lt;artifactId&amp;gt;cucumber-java&amp;lt;/artifactId&amp;gt;
&amp;lt;version&amp;gt;1.0.14&amp;lt;/version&amp;gt;
&amp;lt;scope&amp;gt;test&amp;lt;/scope&amp;gt;
&amp;lt;/dependency&amp;gt;
&amp;lt;dependency&amp;gt;
&amp;lt;groupId&amp;gt;info.cukes&amp;lt;/groupId&amp;gt;
&amp;lt;artifactId&amp;gt;cucumber-junit&amp;lt;/artifactId&amp;gt;
&amp;lt;version&amp;gt;1.0.14&amp;lt;/version&amp;gt;
&amp;lt;scope&amp;gt;test&amp;lt;/scope&amp;gt;
&amp;lt;/dependency&amp;gt;
&amp;lt;/dependencies&amp;gt;
&amp;lt;/project&amp;gt;

This will add Cucumber-JVM and Selenium dependencies to the project.

Writing Feature File

In Cucumber-JVM specifications or requirements are expressed in a plain text, Given/When/Then kind of syntax known as Gherkin language (https://github.com/cucumber/cucumber/wiki/Gherkin) which is understood by the whole team. So let’s create a feature file for above feature in the project.

Add a new package bmicalculator.test under src/test/resources as shown in below screenshot and add a new file with name bmi_calculator.feature

Project Folder

Project Folder

Copy following feature text to bmi_calculator.feature file


Feature: Calculating Body Mass Index
  As a health specialist
  I want a BMI Calculator
  So that I can calculate patient's Body Mass Index
  BmiCalculator

  @ios-sauce
  Scenario Outline: Calculate Body Mass Index
    Given I enter &amp;quot;&amp;lt;Height&amp;gt;&amp;quot; as height
    And I enter &amp;quot;&amp;lt;Weight&amp;gt;&amp;quot; as weight
    And I press the Calculate button
    Then I should see &amp;quot;&amp;lt;BMI&amp;gt;&amp;quot; as bmi and &amp;quot;&amp;lt;Category&amp;gt;&amp;quot; as category

  Examples:
    |Height |Weight |BMI  |Category   |
    |170    |50     |17.30|Underweight|
    |181    |80     |24.42|Normal     |
    |180    |90     |27.78|Overweight |
    |175    |100    |32.65|Obese      |

Every feature file contains a single feature. A feature usually contains a list of scenarios.  Every scenario consists of a list of steps, which must start with one of the keywords Given, When, Then, But or And. Scenarios express expected behavior of the system under given conditions.

In addition to a scenario, a feature may contain a background, scenario outline and examples. Our example scenario contains Scenario background and examples for number of BMI calculations representing each category.

This scenario outlines allow us to more concisely express these examples through use of a template with placeholders. In this example Calculate Body Mass Index is run once for each row in the Examples section beneath it (not counting the first row which is a header). This is similar to data driven testing.

In next part, we’ll see how to run features with Maven, and enter step definitions.