Assumptions in JUnit5

HOME

 Assumptions is a collection of utility methods that support conditional test execution based on assumptions.

In direct contrast to failed assertions, failed assumptions do not result in a test failure; rather, a failed assumption results in a test being aborted.

Assumptions are typically used whenever it does not make sense to continue execution of a given test method — for example, if the test depends on something that does not exist in the current runtime environment.

Junit 5 comes with a subset of the assumption methods that JUnit 4 provides with Java 8 lambda expressions and method references. All JUnit Jupiter assumptions are static methods in the org.junit.jupiter.api.Assumptions class.

  1. Assumptions.assumeTrue() – If the condition is true, then run the test, else aborting the test.
  2. Assumptions.false() – If the condition is false, then run the test, else aborting the test.
  3. Assumptions.assumingThat() – is much more flexible, If condition is true then executes, else do not abort test continue rest of code in test.
1. assumeTrue

The assumeTrue() method validates the given assumption to be true and if the assumption is true – the test proceed, otherwise, test execution is aborted.

    int num1 = 4;
    int num2=6;
    int num3 = 24;
    int num4=10;

    @Test
    void assumeTrueTest() {
        System.setProperty("ENV", "TEST");
        assumeTrue("TEST".equals(System.getProperty("ENV")));

        // Since the condition is true rest of it will get executed
        assertEquals((num1*num2),num3,"The product of "+num1+"and "+num2+"is equal to "+num3);

    }

The output of the above program is

In the below example, assumeTrue() is false. So, the execution is skipped.

    int num1 = 4;
    int num2=6;
    int num3 = 24;
    int num4=10;

    @Test
    void assumeTrueTest1() {
        System.setProperty("ENV", "TEST");
        assumeTrue("QA".equals(System.getProperty("ENV")));

        // Since the condition is true rest of it will not get executed
        assertEquals((num1*num2),num3,"The product of "+num1+"and "+num2+"is equal to "+num3);

    }

The output of the above program is

2. assumeFalse()

The assumeFalse() method validates the given assumption to false and if the assumption is false – test proceed, otherwise, test execution is aborted. In the below example, the test is false and we are using assumeFalse(), so the tests will be executed.

    int num1 = 4;
    int num2=6;
    int num3 = 24;
    int num4=10;

    @Test
    void assumeFalseTest1() {
        System.setProperty("ENV", "TEST");
        assumeFalse("DEV".equals(System.getProperty("ENV")));

        // Since the condition is true rest of it will get executed
        assertEquals((num1*num2),num3,"The product of "+num1+"and "+num2+"is equal to "+num3);

    }

The output of the above program is

In the below example, the test is false and we are using assumeFalse(), so the tests will be executed.

 @Test
    void assumeFalseTest() {
        System.setProperty("ENV", "TEST");
        assumeFalse("TEST".equals(System.getProperty("ENV")));

        // Since the condition is false rest of it will not get executed
        assertEquals((num1*num2),num3,"The product of "+num1+"and "+num2+"is equal to "+num3);

    }

The output of the above program is

3. assertThat()

This method executes the supplied Executable, but only if the supplied assumption is valid.

Unlike the other assumption methods, this method will not abort the test.

If the assumption is invalid, this method does nothing.
If the assumption is valid and the executable throws an exception, it will be treated like a regular test failure. The thrown exception will be rethrown as is but masked as an unchecked exception.

    int num1 = 4;
    int num2=6;
    int num3 = 24;
    int num4=10;
    int num5=8;
    int num6=2;

   @Test
    void assumingThatTest() {
        System.setProperty("ENV", "UAT");
        assumingThat(
                "UAT".equals(System.getProperty("ENV")),
                () -> {
                    // Since the condition is true, this assertion will get executed
                    System.out.println("Assuming that executable executed");
                    assertEquals((num1+num2),num4,"The product of "+ num1 +" and "+ num2 +" is not equal to "+num4);
                });
        // Since the condition is false rest of it will get executed
        System.out.println("Loop outside");
        assertEquals((num5-num2),num6,"The difference of "+ num5 +" and "+num2+" is not equal to " + num6);
    }

The output of the above program is

In the below example, the condition is false , so we skip the execution of that condition. But, we execute the rest of the code.

    int num1 = 4;
    int num2=6;
    int num3 = 24;
    int num4=10;

    @Test
    void assumingThatTest1() {
        System.setProperty("ENV", "UAT");
        assumingThat(
                "DEV".equals(System.getProperty("ENV")),
                () -> {
                    // Since the condition is false, this assertion will not get executed
                    System.out.println("Assuming that executable executed");
                    assertEquals((num1+num2),num4,"The sum of "+num1+"and "+num2+"is not equal to "+num4);
                });
        
        System.out.println("Loop outside");
        assertEquals((num1*num2),num3,"The product of "+num1+"and "+num2+"is not equal to "+num3);
    }

The output of the above program is

Difference between Assumption and Assertion

The main difference between the assertions and assumptions is –

The assumption is use to decide whether we want to execute a section or the rest of the test method or not and if the condition is false then the test is skipped.

Whereas if a condition in an assertion fails then it fails the test and something needs to be fixed.

How to Retry failed tests in TestNG – IRetryAnalyzer

HOME

TestNG is a well thought Test Framework. It provides a lot of different features which makes the life of a tester a little easy. It happens sometimes that a test execution fails, but the failure is not a product bug, but there can be different reasons for the failure such as the environment is down, third party web service is down, or the browser becomes unresponsive. Imagine a scenario where we need to run a test suite consisting of 100 tests and a few tests failed as a result of a known intermittent environment issue. We know that these tests can pass if rerun a couple of times. So, in this case, the retry functionality of TestNG comes to the rescue. This is one of the best and most frequently used functionality.

In this tutorial let us study how we can implement retry on failed tests in TestNG. In order to achieve this, we have to first understand the org.testng.IRetryAnalyzer interface.

To start with, please add the below dependencies to the Maven Project.

<dependencies>
  
      <dependency>
          <groupId>org.seleniumhq.selenium</groupId>
          <artifactId>selenium-java</artifactId>
          <version>3.141.59</version>
      </dependency>
      
      <dependency>
          <groupId>io.github.bonigarcia</groupId>
          <artifactId>webdrivermanager</artifactId>
          <version>5.1.0</version>
       </dependency>

      <dependency>
           <groupId>org.testng</groupId>
           <artifactId>testng</artifactId>
           <version>7.5</version>
           <scope>test</scope>
      </dependency>

  </dependencies>

IRetryAnalyzer – It is an interface to implement to be able to have a chance to retry a failed test. The definition of this interface is

public interface IRetryAnalyzer {

  /**
   * Returns true if the test method has to be retried, false otherwise.
   *
   * @param result The result of the test method that just ran.
   * @return true if the test method has to be retried, false otherwise.
   */
  boolean retry(ITestResult result);
}

This method implementation returns true if you want to re-execute your failed test and false if you don’t want to re-execute your test.

When you bind a retry analyzer to a test, TestNG automatically invokes the retry analyzer to determine if TestNG can retry a test case again in an attempt to see if the test that just fails now passes. Here is how you use a retry analyzer:

  1. Bind this implementation to the @Test annotation for e.g., @Test(retryAnalyzer = Retry.class)
  2. Build an implementation of the interface org.testng.IRetryAnalyzer

1. Add IRetryAnalyzer to the @Test Annotation

First of all, you need to create a class that implements the IRetryAnalyzer like the below example:

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

public class Retry implements IRetryAnalyzer {
	
	int retryCount = 0;
	int maxRetryCount = 2;

	public boolean retry(ITestResult result) {
 
	if(!result.isSuccess()) {                         //Check if test is failed
		
		if(retryCount<maxRetryCount) {                //Check if the maximum number of test execution is reached
			System.out.println("Retrying Test : Re-running " + result.getName() +
 " for " + (retryCount+1) + " time(s)."); //Print the number of Retry attempts
			
			retryCount++;                             //Increase the maxRetryCount by 1
			
			result.setStatus(ITestResult.FAILURE);    //Mark test as failed
         return true;                                 //Rerun the failed test
		} else {
			result.setStatus(ITestResult.FAILURE);    //TestNG marks last run as failed, if last run is max retry
		} 
	  }else {
			result.setStatus(ITestResult.SUCCESS);    //TestNG parks test as passed when the test test passes
			
	  }
	
return false;
	}
}

This example shows that failed test case will run 3 times till it passes. In case it fails the third time, test execution will stop and TestNG will mark this case as failed. We can change the number of tries by changing the value of maxRetryCount.

Using retryAnalyzer attribute in the @Test annotation

The next step is to associate your test cases with IRetryAnalyzer. In order to do this, you need to use the method below.

@Test(retryAnalyzer = Retry.class)
public void verifyLoginPage() {
}

Let us see the complete implementation with the help of the below example.

import java.util.concurrent.TimeUnit;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.testng.Assert;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;
import io.github.bonigarcia.wdm.WebDriverManager;

public class RetryFailedTests {
	
	WebDriver driver;
	 
    @BeforeTest
    public void setUp() {
    	 
    	WebDriverManager.chromedriver().setup();
    	 
        ChromeOptions chromeOptions = new ChromeOptions();
  
        driver = new ChromeDriver(chromeOptions);
        driver.get("https://opensource-demo.orangehrmlive.com/");
 
        driver.manage().window().maximize();
        driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
    }
 
    @Test(retryAnalyzer = Retry.class)
    public void verifyLoginPage() {
 
        String expectedTitle = driver.findElement(By.xpath("//*[@id='logInPanelHeading']")).getText();
 
        System.out.println("Title :" + expectedTitle);
        Assert.assertTrue(expectedTitle.equalsIgnoreCase("LOGIN Panel !!"));
    }
 
    @Test(retryAnalyzer = Retry.class)
    public void verifyHomePage() {
 
        System.out.println("Username Entered");
        driver.findElement(By.name("txtUsername")).sendKeys("Admin");
 
        System.out.println("Password Entered");
        driver.findElement(By.name("txtPassword")).sendKeys("admin123");
 
        driver.findElement(By.id("btnLogin")).submit();
 
        String newPageText = driver.findElement(By.id("welcome")).getText();
        System.out.println("newPageText :" + newPageText);
        Assert.assertTrue(newPageText.contains("Welcome"));
    }
 
    @AfterTest
    public void teardown() {
 
        driver.quit();
    }
 
}

In the above example, test – verifyLoginPage() will be retried a maximum of 3 times, if the test fails. To run the tests, Right-click on the class and select Run As ->TestNG Suite.

The output of the above program is

2. Implement Interface ITestAnnotationTransformer to retry failed tests

In this case, you would need to implement ITestAnnotationTransformer interface. The implementation of this interface is

public interface IAnnotationTransformer extends ITestNGListener {

  /**
   * This method will be invoked by TestNG to give you a chance to modify a TestNG annotation read
   * from your test classes. You can change the values you need by calling any of the setters on the
   * ITest interface.
   *
   * <p>Note that only one of the three parameters testClass, testConstructor and testMethod will be
   * non-null.
   *
   * @param annotation The annotation that was read from your test class.
   * @param testClass If the annotation was found on a class, this parameter represents this class
   *     (null otherwise).
   * @param testConstructor If the annotation was found on a constructor, this parameter represents
   *     this constructor (null otherwise).
   * @param testMethod If the annotation was found on a method, this parameter represents this
   *     method (null otherwise).
   */
  default void transform(
      ITestAnnotation annotation, Class testClass, Constructor testConstructor, Method testMethod) {
    // not implemented
  }

The transform method is called for every test during the test run. We can use this listener for our retry analyzer as shown below:

import java.lang.reflect.Constructor;
import java.lang.reflect.Method;
import org.testng.IAnnotationTransformer;
import org.testng.annotations.ITestAnnotation;

public class RetryListener implements IAnnotationTransformer{

	public void transform(ITestAnnotation arg0, Class arg1, Constructor arg2,Method arg3) {
		
			arg0.setRetryAnalyzer(Retry.class);
		}

	}

Now let us create a class that contains all the tests.

import java.util.concurrent.TimeUnit;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.testng.Assert;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;
import io.github.bonigarcia.wdm.WebDriverManager;

public class RetryTests {
	
	WebDriver driver;
	 
    @BeforeTest
    public void setUp() {
    	 
    	 WebDriverManager.chromedriver().setup();
    	 
         ChromeOptions chromeOptions = new ChromeOptions();
  
         driver = new ChromeDriver(chromeOptions);
         driver.get("https://opensource-demo.orangehrmlive.com/");
 
         driver.manage().window().maximize();
         driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
    }
 
    @Test(description = "This test validates title of login functionality")
    public void verifyLoginPage() {
 
        String expectedTitle = driver.findElement(By.xpath("//*[@id='logInPanelHeading']")).getText();
 
        System.out.println("Title :" + expectedTitle);
        Assert.assertTrue(expectedTitle.equalsIgnoreCase("LOGIN Panel !!"));
    }
 
    @Test(description = "This test validates  successful login to Home page")
    public void verifyHomePage() {
 
        System.out.println("Username Entered");
        driver.findElement(By.name("txtUsername")).sendKeys("Admin");
 
        System.out.println("Password Entered");
        driver.findElement(By.name("txtPassword")).sendKeys("admin123");
 
        driver.findElement(By.id("btnLogin")).submit();
 
        String newPageText = driver.findElement(By.id("welcome")).getText();
        System.out.println("newPageText :" + newPageText);
        Assert.assertTrue(newPageText.contains("Welcome"));
    }
 
    @AfterTest
    public void teardown() {
 
        driver.quit();
    }
 
}

Once we have the implementation of IAnnotationTransformer, we just need to add it as a listener in the testng.xml. Like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">

<listeners>
<listener class-name="com.example.retrydemo.RetryListener"></listener>

</listeners>

  <test name="Test">
    <classes>
      <class name="com.example.retrydemo.RetryTests"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

Now let us run the tests. Right-click on testng.xml and select Run As -> TestNG Suite.

The output of the above program is

This is pretty much it on this topic. Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

Selenium 4 Grid : How to use Standalone Grid

HOME

Selenium Grid routes command received by the client to distant browser instances, allowing WebDriver scripts to be run on remote machines (virtual or real). Its goal is to make running tests in parallel on numerous machines as simple as possible.

Selenium Grid allows us to run tests on numerous workstations in parallel and centrally manage diverse browser versions and setups (instead of in each individual test).

Selenium Grid 4 makes use of a variety of new technologies to make scaling up easier while still allowing for local execution.

Selenium Grid 4 is a brand-new implementation that doesn’t use the same codebase as the previous version.

A Selenium Grid is made up of several components. Using a Grid role, you can start each one individually or all of them at once, depending on your needs.

Why do we need Selenium Grid?

Imagine we need to run 1000 tests and it takes 20 hrs to run these tests. With the help pf Selenium Grid, we can have 4 different machines (VMs or separate physical machines) to run those tests. We can roughly reduce the execution time to one-fourth, which means the test execution will be finished in 5 hrs.

Grid is also used to support running tests against multiple runtime environments, specifically, against different browsers at the same time. For example, a ‘grid’ of virtual machines can be set up with each supporting a different browser that the application to be tested must support. So, machine 1 has Google Chrome, machine 2, has Edge, and machine 3 has the latest Firefox. When the test suite is run, Selenium-Grid receives each test-browser combination and assigns each test to run against its required browser. Grid makes cross-browser and parallel testing very easy.

1. Standalone

The term “standalone” refers to the combination of all components that, in the view of the user, operate as if they were one. After launching it in Standalone mode, you’ll have a fully functional Grid of one.

Standalone is also the quickest way to get a Selenium Grid up and running. The server will be listening on http://localhost:4444 by default, and you should use that URL in your RemoteWebDriver tests. The server will look for available drivers in the System PATH and utilize them.

java -jar selenium-server-4.1.2.jar standalone

2. Hub and Nodes

It enables the classic Hub & Node(s) setup. These roles are suitable for small and middle-sized Grids.

Hub – A Hub is the union of the following components:

  • Router
  • Distributor
  • Session Map
  • New Session Queue
  • Event Bus

By default, the server will be listening on http://localhost:4444, and that’s the URL you should point your RemoteWebDriver tests.

java -jar selenium-server-4.1.2.jar hub

Node – One or more Nodes can be started in this setup, and the server will detect the available drivers that it can use from the System PATH.

java -jar selenium-server-4.1.2.jar node --detect-drivers true

3. Distributed

On Distributed mode, each component needs to be started on its own. This setup is more suitable for large Grids.

  1. Event Bus: serves as a communication path to other Grid components in subsequent steps.
java -jar selenium-server-4.1.2.jar  event-bus

2. Session Map: responsible for mapping session IDs to the Node where the session is running.

java -jar selenium-server-4.1.2.jar sessions

3. New Session Queue: adds the new session request to a queue, then the distributor processes it.

java -jar selenium-server-4.1.2.jar sessionqueue

4. Distributor: Nodes register to it, and assigns a Node for a session request.

java -jar selenium-server-4.1.2.jar distributor --sessions http://localhost:5556 --sessionqueue http://localhost:5559 --bind-bus false

5. Router: the Grid entrypoint, in charge of redirecting requests to the right component.

java -jar selenium-server-4.1.2.jar router --sessions http://localhost:5556 --distributor http://localhost:5553 --sessionqueue http://localhost:5559

6. Nodes

java -jar selenium-server-4.1.2.jar node 

Implementation of Standalone Selenium Grid

Now, let us see how to run Selenium tests in Selenium Grid using the Standalone option.

Step 1 – Download driver.exe for the browsers

I have downloaded the driver.exe (chrome, gecko) in Downloads.

Step 2 – Download selenium-server jar

Go to Selenium Official website. Navigate to Downloads and it shows that the latest stable version of Selenium Grid is 4.3.0. Click on that and download the jar file. Make sure to keep the browsers.exe and selenium-server jar file in the same folder.

Step 3 – Execute selenium-server jar

Open a command line terminal. Use the below command to run selenium-server standalone jar files.

java -jar selenium-server-4.3.0.jar standalone --port 4446

It’s optional to mention the port number at the end of the command. By default, it will use port 4444. It is good practice to mention the port at the end to avoid any conflict between the ports.

Step 4 – Add dependencies to the Maven project

Let us add the necessary dependencies to the pom.xml in the case of the Maven Project.

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>org.example</groupId>
  <artifactId>Selenium4Demo</artifactId>
  <version>0.0.1-SNAPSHOT</version>

  <name>Selenium4Demo</name>
  <url>http://www.example.com</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
  </properties>

<dependencies>
   <dependency>
     <groupId>org.seleniumhq.selenium</groupId>
     <artifactId>selenium-java</artifactId>
     <version>4.1.2</version>
  </dependency>

   <dependency>
      <groupId>io.github.bonigarcia</groupId>
      <artifactId>webdrivermanager</artifactId>
      <version>5.1.0</version>
   </dependency>
  
   <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>7.4.0</version>
      <scope>test</scope>
   </dependency>
</dependencies>

Step 5 – Create the tests

Creating an instance of the Remote WebDriver and passing the selenium endpoint and chrome options defined in it.

To run a Remote WebDriver client, we first need to connect to the RemoteWebDriver. We do this by pointing the URL to the address of the server running our tests. In order to customize our configuration, we set desired capabilities. Below is an example of instantiating a remote WebDriver object pointing to our remote web server running our tests on Chrome.

HelperClass

This class contains the initialization of the Remote web driver and the closing of the web driver.

import java.net.MalformedURLException;
import java.net.URL;
import java.time.Duration;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;


public class HelperClass {

	protected static ThreadLocal<RemoteWebDriver> driver = new ThreadLocal<RemoteWebDriver>();
    public static String remote_url = "http://localhost:4446";

    @BeforeMethod
    public void setDriver() throws MalformedURLException {
    	
        ChromeOptions options = new ChromeOptions();
        driver.set(new RemoteWebDriver(new URL(remote_url), options));
        driver.get().get("https://opensource-demo.orangehrmlive.com/");
        driver.get().manage().window().maximize();

        driver.get().manage().timeouts().pageLoadTimeout(Duration.ofSeconds(10));
        WebDriverWait wait = new WebDriverWait(driver.get(), Duration.ofSeconds(10));
        wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//*[@id='divUsername']/span")));
    }

    public WebDriver getDriver() {
        return driver.get();
    }

    @AfterMethod
    public  void closeBrowser() {
        driver.get().quit();
        driver.remove();
    }
}

Selenium4GridTest

This class contains all the tests like login test, invalid credential, verify heading of the login page, verify LinkedIn link on the login page, and verify the heading of Forgot Password page.

import static org.testng.Assert.assertEquals;
import static org.testng.Assert.assertTrue;
import org.openqa.selenium.By;
import org.testng.annotations.Test;

public class Selenium4GridTest extends HelperClass{

	    @Test
	    public void invalidCredentials() {
	    	
	        getDriver().findElement(By.xpath("//*[@id='txtUsername']")).sendKeys("1234");
	        getDriver().findElement(By.xpath("//*[@id='txtPassword']")).sendKeys("12342");
	        getDriver().findElement(By.xpath("//*[@id='btnLogin']")).click();
	        String actualErrorMessage = getDriver().findElement(By.xpath("//*[@id='spanMessage']")).getText();
	        System.out.println("Actual ErrorMessage :" + actualErrorMessage);
	        assertEquals(actualErrorMessage,"Invalid credentials");

	    }

	    @Test
	    public void loginPageHeading() {

	        String loginText = getDriver().findElement(By.xpath("//*[@id='logInPanelHeading']")).getText();
	        System.out.println("Actual loginText :" + loginText);
	        assertEquals(loginText,"LOGIN Panel");

	    }

	    @Test
	    public void forgotPasswordPageHeading()  {

	        getDriver().findElement(By.xpath("//*[@id='forgotPasswordLink']/a")).click();
	        String forgetPasswordTitle= getDriver().findElement(By.xpath(" //*[@id='content']/div[1]/div[2]/h1")).getText();
	        System.out.println("Actual Page Title of Forgot Password Page :" + forgetPasswordTitle);
	        assertEquals(forgetPasswordTitle,"Forgot Your Password?");
	    }

	    @Test
	    public void verifyLinkedIn() {

	        Boolean linkedInIcon = getDriver().findElement(By.xpath("//*[@id='social-icons']/a[1]/img")).isEnabled();
	        System.out.println("Actual linkedIn Text :" + linkedInIcon);
	        assertTrue(linkedInIcon);

	    }
	}

Step 6 – Create a testng.xml

It is very easy to create testng.xml for Eclipse IDE. Right-click on the project -> TestNG -> Convert to TestNG.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test thread-count="5" name="Test">
    <classes>
      <class name="org.example.Selenium4GridTest"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

Step 7 – Execute the tests through the command line

To run it from the command prompt, open a command prompt and go to the project and run the following command:

mvn clean test

Step 8 – Execute the tests from testng.xml

We can right-click on testng.xml, and select Run As -> TestNG Suite.

Step 9 – Navigate to the sessions tab on the Selenium Grid UI upon running the command.

It would reflect an active session.

Step 10 – Review the test execution result

The logs can be viewed in the command prompt as shown below:

Step 11 – TestNG Report Generation

TestNG generates the test reports in test-output folder.

We are interested in index.html report. This report contains various information, like time taken by each step in the test, time taken by each test, testng.xml, and number of tests passed or failed.

Emailable-Report.html

The output reports in TestNG reporting look like below as all tests are passed:

Congratulations!! We are able to run Selenium 4 tests on the new Selenium 4 Grid. Happy Learning!!

Integration Testing of Springboot with Cucumber and TestNG

HOME

In this tutorial, I am going to build an automation framework to test the Springboot application with Cucumber, Rest Assured, and TestNG.

What is Springboot?

Spring Boot is an open-source micro-framework maintained by a company called Pivotal. It provides Java developers with a platform to get started with an auto-configurable production-grade Spring application. With it, developers can get started quickly without losing time on preparing and configuring their Spring application.

What is Cucumber?

Cucumber is a software tool that supports behavior-driven development (BDD). Cucumber can be defined as a testing framework, driven by plain English. It serves as documentation, automated tests, and development aid – all in one.

This framework consists of:

  1. Springboot – 2.5.2
  2. Cucumber – 7.3.4
  3. Java 11
  4. TestNG – 7.3.4
  5. Maven – 3.8.1
  6. RestAssured – 5.1.1

Steps to setup Cucumber Test Automation Framework for API Testing using Rest-Assured

  1. Add SpringbootTest, Rest-AssuredJUnit, and Cucumber dependencies to the project
  2. Create a source folder src/test/resources and create a feature file under src/test/resources
  3. Create the Step Definition class or Glue Code for the Test Scenario under the src/test/java directory
  4. Create a Cucumber Runner class under the src/test/java directory
  5. Run the tests from Cucumber Test Runner
  6. Run the tests from Command Line
  7. Run the tests from TestNG
  8. Generation of TestNG Reports
  9. Cucumber Report Generation

Below is the structure of a SpringBoot application project

We need the below files to create a SpringBoot Application.

SpringBootRestServiceApplication.java

The Spring Boot Application class is generated with Spring Initializer. This class acts as the launching point for the application.

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class SpringBootRestServiceApplication {

    public static void main(String[] args) {

        SpringApplication.run(SpringBootRestServiceApplication.class, args);
    }
}

Student.java

This is JPA Entity for Student class

import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.Id;
import javax.validation.constraints.NotBlank;
import javax.validation.constraints.NotNull;
import javax.validation.constraints.Size;

@Entity
public class Student {
    @Id
    @GeneratedValue
    private Long id;

    @NotNull
    @Size(min = 4, message = "Name should have atleast 4 characters")
    private String name;

    @NotBlank(message = "passportNumber is mandatory")
    private String passportNumber;

    public Student() {
        super();
    }

    public Student(Long id, String name, String passportNumber) {
        super();
        this.id = id;
        this.name = name;
        this.passportNumber = passportNumber;
    }

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getPassportNumber() {
        return passportNumber;
    }

    public void setPassportNumber(String passportNumber) {
        this.passportNumber = passportNumber;
    }
}

StudentRepository.java 

This is JPA Repository for Student. This is created using Spring Data JpaRepository.

import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;

@Repository
public interface StudentRepository extends JpaRepository<Student, Long>{

}

StudentController.java

Spring Rest Controller exposes all services on the student resource.

import static org.springframework.hateoas.server.mvc.WebMvcLinkBuilder.linkTo;
import static org.springframework.hateoas.server.mvc.WebMvcLinkBuilder.methodOn;
import java.net.URI;
import java.util.List;
import java.util.Optional;
import javax.validation.Valid;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.hateoas.EntityModel;
import org.springframework.hateoas.server.mvc.WebMvcLinkBuilder;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.servlet.support.ServletUriComponentsBuilder;

@RestController
public class StudentController {

    @Autowired
    private StudentRepository studentRepository;

    @GetMapping("/students")
    public List<Student> retrieveAllStudents() {
        return studentRepository.findAll();
    }

    @GetMapping("/students/{id}")
    public EntityModel<Student> retrieveStudent(@PathVariable long id) {
        Optional<Student> student = studentRepository.findById(id);

        if (!student.isPresent())
            throw new StudentNotFoundException("id-" + id);

        EntityModel<Student> resource = EntityModel.of(student.get());

        WebMvcLinkBuilder linkTo = linkTo(methodOn(this.getClass()).retrieveAllStudents());

        resource.add(linkTo.withRel("all-students"));

        return resource;
    }

    @PostMapping("/students")
    public ResponseEntity<Object> createStudent(@Valid @RequestBody Student student) {
        Student savedStudent = studentRepository.save(student);

        URI location = ServletUriComponentsBuilder.fromCurrentRequest().path("/{id}")
                .buildAndExpand(savedStudent.getId()).toUri();

        return ResponseEntity.created(location).build();

    }
}

application.properties

Spring Boot automatically loads the application.properties whenever it starts up. You can de-reference values from the property file in the java code through the environment.

spring.jpa.defer-datasource-initialization=true

data.sql 

Data is loaded from data.sql into the Student table. Spring Boot would execute this script after the tables are created from the entities.

insert into student values(10001,'Annie', 'E1234567');
insert into student values(20001,'John', 'A1234568');
insert into student values(30001,'David','C1232268');
insert into student values(40001,'Amy','D213458');

Test Automation Framework Implementation

Step 1 – Add SpringbootTest, Cucumber, Rest-Assured, and TestNG dependencies to the project (Maven project)

 <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <rest-assured.version>5.1.1</rest-assured.version>
        <cucumber.version>7.3.4</cucumber.version>
    </properties>

<dependencies>
        
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>io.rest-assured</groupId>
            <artifactId>rest-assured</artifactId>
            <version>${rest-assured.version}</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>${cucumber.version}</version>
        </dependency>

        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-testng</artifactId>
            <version>${cucumber.version}</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-spring</artifactId>
            <version>${cucumber.version}</version>
            <scope>test</scope>
        </dependency>

</dependencies>

Step 2 – Create a source folder src/test/resources and create a feature file under src/test/resources

By default, the Maven project has an src/test/java directory only. Create a new Source Folder under src/test with the name of resources. Create a folder name as Features within the src/test/resources directory.

Create a feature file to test the Springboot application. Below is a sample feature file.

Feature: Verify springboot application using Cucumber and TestNG

  @ReceiveUserDetails
  Scenario Outline: Send a valid Request to get user details
    Given I send a request to the URL "/students" to get user details
    Then The response will return status 200 
    And The response contains id <studentID> and names "<studentNames>" and passport_no "<studentPassportNo>"

    Examples:
      |studentID    |studentNames  |studentPassportNo|
      |10001        |Annie         |E1234567         |
      |20001        |John          |A1234568         |
      |30001        |David         |C1232268         |
      |40001        |Amy           |D213458          |
      
   
  @CreateUser
  Scenario: Send a valid Request to create a user 
    Given I send a request to the URL "/students" to create a user with name "Annie" and passportNo "E1234567"
    Then The response will return status 201
    And Resend the request to the URL "/students" and the response returned contains name "Annie" and passport_no "E1234567"

Step 3 – Create the Step Definition class or Glue Code for the Test Scenario under src/test/java

The corresponding step definition file of the above feature file is shown below.

import static io.restassured.RestAssured.given;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.hasItem;
import org.json.JSONObject;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.web.server.LocalServerPort;
import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.spring.CucumberContextConfiguration;
import io.restassured.RestAssured;
import io.restassured.http.ContentType;
import io.restassured.response.ValidatableResponse;
import io.restassured.specification.RequestSpecification;

@CucumberContextConfiguration
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class SpringbootDefinitions {

	private final static String BASE_URI = "http://localhost";

	@LocalServerPort
	private int port;

	private ValidatableResponse validatableResponse, validatableResponse1;

	private void configureRestAssured() {
		RestAssured.baseURI = BASE_URI;
		RestAssured.port = port;
	}

	protected RequestSpecification requestSpecification() {
		configureRestAssured();
		return given();
	}

	@Given("I send a request to the URL {string} to get user details")
	public void getStudentDetails(String endpoint) throws Throwable {
		validatableResponse = requestSpecification().contentType(ContentType.JSON).when().get(endpoint).then();
		System.out.println("RESPONSE :" + validatableResponse.extract().asString());
	}

	@Given("I send a request to the URL {string} to create a user with name {string} and passportNo {string}")
	public void createStudent(String endpoint, String studentName, String studentPassportNumber) throws Throwable {

		JSONObject student = new JSONObject();
		student.put("name", studentName);
		student.put("passportNumber", studentPassportNumber);

		validatableResponse = requestSpecification().contentType(ContentType.JSON).body(student.toString()).when()
				.post(endpoint).then();
		System.out.println("RESPONSE :" + validatableResponse.extract().asString());
	}

	@Then("The response will return status {int}")
	public void verifyStatusCodeResponse(int status) {
		validatableResponse.assertThat().statusCode(equalTo(status));

	}

	@Then("The response contains id {int} and names {string} and passport_no {string}")
	public void verifyResponse(int id, String studentName, String passportNo) {
		validatableResponse.assertThat().body("id", hasItem(id)).body(containsString(studentName))
				.body(containsString(passportNo));

	}

	@Then("Resend the request to the URL {string} and the response returned contains name {string} and passport_no {string}")
	public void verifyNewStudent(String endpoint, String studentName, String passportNo) {

		validatableResponse1 = requestSpecification().contentType(ContentType.JSON).when().get(endpoint).then();
		System.out.println("RESPONSE :" + validatableResponse1.extract().asString());
		validatableResponse1.assertThat().body(containsString(studentName)).body(containsString(passportNo));

	}
}

To make Cucumber aware of your test configuration you can annotate a configuration class on your glue path with @CucumberContextConfiguration and with one of the following annotations: @ContextConfiguration, @ContextHierarchy, or @BootstrapWith.It is imported from:

import io.cucumber.spring.CucumberContextConfiguration;

As we are using SpringBoot, we are annotating the configuration class with @SpringBootTest. It is imported from:

import org.springframework.boot.test.context.SpringBootTest;

By default, @SpringBootTest does not start the webEnvironment to refine further how your tests run. It has several options: MOCK(default), RANDOM_PORT, DEFINED_PORT, NONE.

RANDOM_PORT loads a WebServerApplicationContext and provides a real web environment. The embedded server is started and listens on a random port. LocalServerPort is imported from the package:

import org.springframework.boot.web.server.LocalServerPort;

Step 4 – Create a Cucumber TestNG Runner class under src/test/java

A runner will help us to run the feature file and acts as an interlink between the feature file and StepDefinition Class. The TestRunner should be created within the directory src/test/java.

import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;

@CucumberOptions(features = {"src/test/resources/Features"}, glue = {"com.example.demo.definitions"})
public class CucumberRunnerTests extends AbstractTestNGCucumberTests {

}

The @CucumberOptions annotation is responsible for pointing to the right feature package, configuring the plugin for a better reporting of tests in the console output, and specifying the package where extra glue classes may be found. We use it to load configuration and classes that are shared between tests.

Step 5 – Run the tests from Cucumber Test Runner

You can execute the test script by right-clicking on TestRunner class -> Run As TestNG in Eclipse.

In case you are using IntelliJ, select Run CucumberRunnerTests.

SpringBootTest creates an application context containing all the objects we need for the Integration Testing It, starts the embedded server, creates a web environment, and then enables methods to do Integration testing.

Step 6 – Run the tests from Command Line

Use the below command to run the tests through the command line.

mvn clean test

Step 7 – Run the tests from TestNG

Create a testng.xml in the project as shown below:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name = "Suite1">
  <test name = "SpringBoot Cucumber TestNG Demo">
    <classes>
          <class name = "com.example.demo.runner.CucumberRunnerTests"/>
     </classes>  
   </test>
</suite>

Step 8 – Generation of TestNG Reports

TestNG generates various types of reports under the test-output folder like emailable-report.html, index.html, testng-results.xml.

We are interested in the “emailable-report.html” report. Open “emailable-report.html”, as this is an HTML report, and open it with the browser. The below image shows emailable-report.html.

TestNG also produce “index.html” report, and it resides under test-output folder. The below image shows index.html report.

Step 9 – Cucumber Report Generation

Add cucumber.properties under src/test/resources and add the below instruction in the file.

cucumber.publish.enabled=true

The link to the Cucumber Report is present in the execution status.

Below is the image of the Cucumber Report generated using the Cucumber Service.

Complete Source Code:
Refer to GitHub for the source code.

Congratulations!! We are able to build a test framework to test the SpringBoot application using Cucumber, Rest Assured, and TestNG.

How to run Selenium 3 on Docker

HOME

What is Docker and Container?

Docker is an open platform for developing, shipping, and running applications.
Docker provides the ability to package and run an application in a loosely isolated environment called a container. A container is a runnable instance of an image. You can create, start, stop, move, or delete a container using the Docker API or CLI. You can connect a container to one or more networks, attach storage to it, or even create a new image based on its current state.

By default, a container is relatively well isolated from other containers and its host machine
It contains multiple components such as Docker Daemon, Docker Clients, Docker Registry, and Docker Compose. It works on a client-server architecture. The Docker client communicates with the Docker Daemon handles the complex part of building, running, and distributing the Docker containers.

Docker Desktop is an easy-to-install application for your Mac or Windows environment that enables you to build and share containerized applications and microservices. Docker Desktop includes the Docker daemon (dockerd), the Docker client (docker), Docker Compose, Docker Content Trust, Kubernetes, and Credential Helper.

Why use Selenium with Docker for web application testing

When Selenium is used with Docker, there is no need to install the necessary browsers and browser drivers on the host machine. Overall, it is significantly quicker to get started with Selenium web automation testing using pre-made Docker containers. A range of Docker images (with Selenium) on the Docker Hub can be used by running a few commands on the terminal. Some of these images on the Docker Hub were developed and maintained by Selenium HQ.

One of the most common uses of Docker containers in selenium testing is cross-browser testing. Setting up test setups with all of the Browser-OS combinations gets difficult. These can be set up on the go and taken down once the tests are completed using Docker containers.

Another use of Docker containers in Selenium testing is parallel testing. Because there are so many sorts of tests to conduct, doing them sequentially takes a long time. As a result, parallelizing the testing saves time and provides faster feedback. Parallel testing, on the other hand, necessitates a significant amount of infrastructure setup. This would have a direct bearing on the price. Multiple containers can be launched on a single server using docker containers, making the most of the underlying hardware while also allowing for concurrent testing

Let’s discuss how to set up docker and run Selenium tests on it.

Setting up Window Docker

Step 1 – Download the Docker Installer

Docker provides an installer for Windows which can be downloaded from the official docker website.

Step 2 – Install Docker

Launch the installer by double-clicking on it. Select the Enable Hyper-V Windows Features option on the configuration page.

If the user account and admin accounts are different, the user account must be added to the docker-users group as shown below:

To do that, you will need to run Computer Management as an administrator and navigate to Local Users and Groups > Groups > docker-users. Then right-click to add the user to the group. You will need to log and log back in for the changes to take effect.

Step 3 – Start Docker Desktop

After the installation process is complete, the tool does not start automatically. To start the Docker tool, search for the tool, and select Docker Desktop in your desktop search results or Docker Desktop can be started from the start menu.

Docker is free for small businesses, personal usage, education, and non-commercial purposes.

Step 4 – Verify the installation of Desktop Docker

To see if Docker is correctly configured, run the following line in Command Prompt. The version of Docker installed on the system is provided.

docker --version

Running Selenium Tests in Docker

The Docker Desktop can execute a few docker images after it is installed. You can either build a Docker image from the scratch or start with a pre-configured base image from the Docker hub and add to it.

The selenium/standalone-firefox image hosted by selenium on DockerHub is used in this tutorial.

Step 1 – Pull the docker image

Run the following command to download a copy of the image onto the system.

docker pull selenium/standalone-firefox

Step 2 – Running the Selenium Webdriver Docker container

Upon pulling the selenium/standalone-firefox image onto the system, start the container by running the following command:

docker run -d -p 4444:4444 -v /dev/shm:/dev/shm selenium/standalone-firefox

The above command starts a container from the image specified in detached mode (background mode). It also maps Port 4444 on the container to Port 4444 on your local browser.

Open the browser and navigate to http://localhost:4444/. It reflects Selenium Grid UI, as shown below.

Step 3 – Add the dependencies to the POM.xml

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>org.example</groupId>
  <artifactId>SeleniumDockerDemo</artifactId>
  <version>0.0.1-SNAPSHOT</version>

  <name>SeleniumDockerDemo</name>
  <url>http://www.example.com</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
  </properties>

<dependencies>
 
  <dependency>
     <groupId>org.seleniumhq.selenium</groupId>
     <artifactId>selenium-java</artifactId>
     <version>3.141.59</version>
  </dependency>

  <dependency>
     <groupId>io.github.bonigarcia</groupId>
     <artifactId>webdrivermanager</artifactId>
     <version>5.1.0</version>
   </dependency>
  
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.13.2</version>
      <scope>test</scope>
    </dependency>
   
</dependencies>

Step 4 – Create a sample test

I have created a base class where the WebDriver is initialized and at the end is closed.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.After;
import org.junit.Before;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.remote.RemoteWebDriver;
import java.net.MalformedURLException;
import java.net.URL;

public class BaseTest {
    protected static ThreadLocal<RemoteWebDriver> driver = new ThreadLocal<>();
    public static String remote_url_firefox = "http://localhost:4444/wd/hub";


    @Before
    public void setDriver() throws MalformedURLException {

        WebDriverManager.firefoxdriver().setup();
        FirefoxOptions options = new FirefoxOptions();
        options.addArguments("window-size=1920,1200");
        driver.set(new RemoteWebDriver(new URL(remote_url_firefox), options));
        driver.get().get("https://opensource-demo.orangehrmlive.com/");

    }

    public WebDriver getDriver() {
        return driver.get();
    }

    @After
    public  void closeBrowser() {
        driver.get().quit();
        driver.remove();
    }
}

The below class contains the various tests. The tests are related to verifying the login to the application, verifying the title of the current page, verifying the error message generated on providing the invalid credentials, verifying the LinkedIn link, and verifying the heading of forgot password page.

import org.junit.Test;
import org.openqa.selenium.By;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;

public class LoginPageTest extends BaseTest{

    @Test
    public void validCredentials() {

        getDriver().findElement(By.name("txtUsername")).sendKeys("Admin");
        getDriver().findElement(By.name("txtPassword")).sendKeys("admin123");
        getDriver().findElement(By.id("btnLogin")).click();
        String newPageText = getDriver().findElement(By.xpath("//*[@id='content']/div/div[1]/h1")).getText();
        System.out.println("newPageText :" + newPageText);
        assertEquals(newPageText,"Dashboard");

    }

    @Test
    public void invalidCredentials() {

        getDriver().findElement(By.name("txtUsername")).sendKeys("1234");
        getDriver().findElement(By.name("txtPassword")).sendKeys("12342");
        getDriver().findElement(By.id("btnLogin")).click();
        String actualErrorMessage = getDriver().findElement(By.id("spanMessage")).getText();
        System.out.println("Actual ErrorMessage :" + actualErrorMessage);
        assertEquals(actualErrorMessage,"Invalid credentials");

    }

    @Test
    public void loginPageHeading() {

        String loginText = getDriver().findElement(By.id("logInPanelHeading")).getText();
        System.out.println("Actual loginText :" + loginText);
        assertEquals(loginText,"LOGIN Panel");

    }

    @Test
    public void forgotPasswordPageHeading() {

        getDriver().findElement(By.xpath("//*[@id='forgotPasswordLink']/a")).click();
        String forgetPasswordTitle= getDriver().findElement(By.xpath(" //*[@id='content']/div[1]/div[2]/h1")).getText();
        System.out.println("Actual Page Title of Forgot Password Page :" + forgetPasswordTitle);
        assertEquals(forgetPasswordTitle,"Forgot Your Password?");
    }

    @Test
    public void verifyLinkedIn() {

        Boolean linkedInIcon = getDriver().findElement(By.xpath("//*[@id='social-icons']/a[1]/img")).isEnabled();
        System.out.println("Actual linkedIn Text :" + linkedInIcon);
        assertTrue(linkedInIcon);

    }

}

Step 5 – Executing the test case

To run it from the command prompt, open a command prompt and run the following command:

mvn clean test

Navigate to the sessions tab on the Selenium Grid UI upon running the command. It would reflect an active session.

The logs can be viewed in the command prompt as shown below:

The same test can be run using Chrome browser too. To run the tests using Chrome browser, we need to make 2 changes.

1 Firstly, download selenium/standalone-chrome image hosted by selenium on DockerHub.

Use the below command to pull the image in Docker

docker pull selenium/standalone-chrome

Start the container by running the following command:

docker run -d -p 4444:4444 -v /dev/shm:/dev/shm selenium/standalone-chrome

2. Secondly make the changes in the test code.

WebDriverManager.chromedriver().setup();
ChromeOptions options = new ChromeOptions();

Congratulations!!. The above steps allow running Selenium tests in Docker seamlessly. Happy Learning

Step Definition in Cucumber

HOME

The previous tutorial explained the Feature File in Cucumber. This tutorial explains the step definition of the Cucumber.

To start with, please add the below dependencies to the POM.xml, in the case of the Maven project.

<dependencies>
  
   <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-java</artifactId>
      <version>7.18.1</version>
   </dependency>

   <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-junit</artifactId>
      <version>7.18.1</version>
      <scope>test</scope>
    </dependency>
       
   <dependency>
       <groupId>junit</groupId>
       <artifactId>junit</artifactId>
       <version>4.13.2</version>
       <scope>test</scope>
   </dependency>
    
</dependencies>

For the Gradle project, add the below dependencies to build.gradle

implementation 'io.cucumber:cucumber-java:718.1'
testImplementation 'io.cucumber:cucumber-junit:7.18.1'
testImplementation 'junit:junit:4.13.2'

What is a Step Definition?

A Step Definition is a Java method with an expression that links it to one or more Gherkin steps. When Cucumber executes a Gherkin step in a scenario, it will look for a matching step definition to execute.

Cucumber finds the Step Definition file with the help of the Glue code in Cucumber Options.

By storing state in instance variables, a step definition can transfer state to a subsequent step definition.

Step definitions are not associated with a specific feature file or scenario. The name of a step definition’s file, class, or package has no bearing on which Gherkin steps it will match. The formulation of the step definition is the only thing that matters, which means the step definition should only match Gherkin’s steps.

Imagine, we want to test a web application. One of the first steps is Login to the website and then check the various functionalities on the website. We can create a Gherkin step like “I login to the website” and the corresponding step definition of this Gherkin Step. This Gherkin step can be used in multiple feature files, and we don’t need to create the step definition of this Gherkin step for each feature file.

In the previous tutorial, we have seen that when the Feature file is executed without the Step Definition file, the runner shows the missing steps with the snippet in the console.

When a Cucumber encounters a Gherkin step without a matching step definition, it will print a step definition snippet with a matching Cucumber Expression. You can use this as a starting point for new step definitions.

It is very easy to implement all the steps, all you need to do is copy the complete text marked in the above box and paste it into the MyHolidayDefinitions class.

@Given, @When, and @Then are imported from packages:-

import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;

Feature File

Feature: Book flight ticket 

@BookOneWayFlight
Scenario: Book Flight for one way trip

Given I live in Dublin with 2 adults and 2 kids
And I want to book one way flight ticket from Dublin to London on 22 Jan 2020
When I search online
Then TripAdvisor should provide me options of flights on 22 Jan 2020
And Cost of my flight should not be more than 50 Euro per person
And Tickets should be refundable

Let me create the step definition for the above Feature file

import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;

public class MyHolidayDefinitions {

	@Given("I live in Dublin with {int} adults and {int} kids")
	public void liveInDublin(Integer int1, Integer int2) {

		System.out.println("I live in Dublin with 2 adults and 2 kids");
	}

	@Given("I want to book one way flight ticket from Dublin to London on {int} Jan {int}")
	public void bookFlightTicket(Integer int1, Integer int2) {

		System.out.println("I want to book one way flight ticket from Dublin to London on 22 Jan 2020");
	}

	@When("I search online")
	public void searchOnline() {

		System.out.println("I search online");
	}

	@Then("TripAdvisor should provide me options of flights on {int} Jan {int}")
	public void tripAdvisor(Integer int1, Integer int2) {

		System.out.println("TripAdvisor should provide me options of flights on 22 Jan 2020");
	}

	@Then("Cost of my flight should not be more than {int} Euro per person")
	public void costOfFlightLimit(Integer int1) {

		System.out.println("Cost of my flight should not be more than 50 Euro per person");
	}

	@Then("Tickets should be refundable")
	public void refundableTickets() {

		System.out.println("Tickets should be refundable");
	}

}

To run the scenarios present in the Feature File, we need TestRunner class. To learn more about the TestRunner class, please refer to this tutorial – Cucumber Tutorial – JUnit Test Runner Class

import org.junit.runner.RunWith;
import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;

@RunWith(Cucumber.class)
@CucumberOptions(features = "src/test/resources/Features/MyHoliday.feature", 
tags = "@BookOneWayFlight", glue = "com.cucumber.MyCucumberProject.definitions")

public class CucumberRunnerTest {

}

The output of the above program is

Congratulations. We have created the setup definition for the feature file successfully and are able to run it.

Refer to the next tutorials to know the integration of Cucumber with Selenium – Integration of Cucumber with Selenium and JUnit and JUnit4 and Integration of Cucumber with Selenium and TestNG

Happy Learning!!

How to install Cucumber Eclipse Plugin

Last Updated on

HOME

The Cucumber plugin is an Eclipse plugin that allows Eclipse to understand the Gherkin syntax. When we are working with cucumber we will write the feature files that contain Feature, Scenario, Given, When, Then, And, But, Tags, Scenario Outline, and Examples. By default, Eclipse doesn’t understand these keywords, so it doesn’t show any syntax highlighter. Cucumber Eclipse Plugin highlights the keywords present in the Feature File.

When we create a feature file in Eclipse, it looks something like the below without Cucumber Eclipse Plugin installed.

It is easy to install Cucumber Eclipse Plugin, as it comes as a plugin for Eclipse IDE. A prerequisite for installing this plugin is your Internet connection should be up & running during the installation of this plugin and Eclipse IDE should be installed on your computer.

Steps to follow:

Step 1 – Launch the Eclipse IDE and, from the Help menu, click “Install New Software”.

Step 2 – You will see a dialog window, click the Add button.

Step 3 – Type the name as you wish, let’s take Cucumber and type “https://cucumber.github.io/cucumber-eclipse-update-site-snapshot” as the location. Click the OK button.

Step 4 – You come back to the previous window, but this time you must see the Cucumber Eclipse Plugin option in the available software list. Just check the box and press the Next button.

Step 5 – Click on the Next Button.

Step 6 – Click “I accept the terms of the licence agreement” and then click the “Finish” button.

Step 7 – You may or may not encounter a Security warning, if in case you do just click the “OK” button.

Step 8 -You are all done now, just click the “Restart Now” button.

After restarting Eclipse, you can see the feature file is highlighted based on the keywords.

This means now Eclipse is able to understand the language we have written in the feature file as Gherkin language.

All the steps in the below scenario are highlighted in yellow colour, which indicates we don’t have any corresponding step definition for each step.

Now, in the below example, I have created the step definition for the Given statement. So, now, it is not highlighted in yellow colour.

If you press the Ctrl button and place the cursor on a Given Statement, it will take you to the corresponding step definition of that step. This is a very helpful feature. When we have multiple feature files with multiple steps, it helps us find the exact location of the step definition.

I hope this tutorial makes your learning a little easy. Thanks. Happy Learning!!

New Features in Selenium 4

Last Updated On

HOME

Selenium is moved from version 3 to version 4 which is quite a huge step. What does this change mean? It means that a few of the old features of Selenium 3 are depreciated in Selenium 4 as well some new features are added to it also. I’m trying to explain a few of the latest updates done in Selenium 4.

Table of Contents

  1. Enhanced Selenium Grid
  2. Simplification to open a new Windows browser and Tabs
  3. Relative Locators
  4. TakeElementScreenshot
  5. New additions to the Actions Class
  6. Deprecation of Desired Capabilities
  7. Chrome Dev Tools

Selenium 3 – This is the latest version of Selenium3 available.

<dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>selenium-java</artifactId>
    <version>3.141.59</version>
</dependency>

Selenium 4 – This is the latest version of Selenium 4 Libraries that are present in the Maven Central Repository.

<dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>selenium-java</artifactId>
    <version>4.2.1</version>
</dependency>

1. Enhanced Selenium Grid

Managing Selenium Grid is now smooth and easy as there will no longer be any need to set up and start hubs and nodes separately. The grid can be deployed in 3 modes:

Standalone – Standalone is the union of all components, and to the user’s eyes, they are executed as one. A fully functional Grid of one is available after starting it in the Standalone mode. By default, the server will be listening on http://localhost:4444, and that’s the URL you should point your RemoteWebDriver tests. The server will detect the available drivers that it can use from the System PATH.

Hub and Node – It enables the classic Hub & Node(s) setup. These roles are suitable for small and middle-sized Grids

Distributed – On Distributed mode, each component needs to be started on its own. This setup is more suitable for large Grids.

Grid will now support IPv6 addresses and one can communicate with the Grid using the HTTPS protocol. In Grid 4, the configuration files used for spinning up the grid instances can be written in TOML (Tom’s Obvious, Minimal Language) which will make it easier for humans to understand.

The new Selenium Grid comes with Docker support. It also supports advanced tools like AWS, Azure, and much more, useful in the DevOps process. Now Grid has a more user-friendly UI and contains relevant information related to the session, running, capacity, etc.

2. Simplification to open a new Windows browser and Tabs

There are a number of scenarios where you would want to open a new browser (or tab) and perform a certain set of actions in the newly opened window/tab. In Selenium 3, you have to create a new Web Driver object and then switch to the new window (or tab) using its unique WindowHandle to perform subsequent actions in that window (or tab).

Selenium 4 provides a new API new Window that lets you create a new window (or tab) and automatically switch to it. Since the new window or tab is created in the same session, it avoids creating a new WebDriver object. For switching to the new tab, pass WindowType.TAB to newWindow() method and for creating a new window, pass WindowType.WINDOW to newWindow() method.

public class NewWindowDemo {

	public static void main(String[] args) {
		System.setProperty("webdriver.chrome.driver",
				"C:\\Users\\Vibha\\Software\\chromedriver_win32_93.0.4577.15\\chromedriver.exe");
		WebDriver driver = new ChromeDriver();
		driver.manage().window().maximize();

		driver.get("https://www.bing.com/");
		System.out.println("New Page - Bing is opened");

		// Opens a new window and switches to new window
		driver.switchTo().newWindow(WindowType.WINDOW);

		// Opens duckduckgo homepage in the new opened window
		driver.navigate().to("https://www.duckduckgo.com/");

		System.out.println("New Page - DuckDuckGo is opened");

		driver.quit();
	}
}

Open a new Tab in Selenium 4

public class NewTabDemo {

	public static void main(String[] args) {

		System.setProperty("webdriver.chrome.driver",
				"C:\\Users\\Vibha\\Software\\chromedriver_win32_93.0.4577.15\\chromedriver.exe");
		
		WebDriver driver = new ChromeDriver();
		driver.manage().window().maximize();

		driver.get("https://www.bing.com/");
		System.out.println("New Page - Bing is opened");

		// Opens a new window and switches to new window
		driver.switchTo().newWindow(WindowType.TAB);

		// Opens duckduckgo homepage in the new opened window
		driver.navigate().to("https://www.duckduckgo.com/");

		System.out.println("New Tab is opened with DuckDuckGo");
		driver.quit();
	}
}

3. Relative Locators

Selenium 4 brings Relative Locators which are previously called as Friendly Locators. This functionality was added to help you locate elements that are nearby other elements. The Available Relative Locators are:

above
below
toLeftOf
toRightOf
near

findElement method now accepts a new method withTagName() which returns a RelativeLocator.

import static org.openqa.selenium.support.locators.RelativeLocator.with;

a) above() – Returns the WebElement, which appears above the specified element.

WebElement passwordField_above = driver.findElement(By.id("txtPassword"));
		WebElement emailAddressField_above = driver.findElement(with(By.tagName("input")).above(passwordField_above));

b) below() – Returns the WebElement, which appears below the specified element.

WebElement emailAddress_below = driver.findElement(By.id("txtUsername"));
WebElement passwordField_below = driver.findElement(with(By.tagName("input")).below(emailAddress_below));

c) toRightOf() – Target web element which is presented to the right of a specified element.

WebElement submitButton= driver.findElement(By.id("submit"));
WebElement cancelButton= driver.findElement(with(By.tagName("button"))
                                            .toLeftOf(submitButton));

d) toRightOf() – Returns the WebElement, which appears to the right of the specified element.

WebElement cancelButton= driver.findElement(By.id("cancel"));
WebElement submitButton= driver.findElement(with(By.tagName("button")).toRightOf(cancelButton));

e) near() – Returns the WebElement, which is at most 50px away from the specified element.

WebElement emailAddressLabel= driver.findElement(By.id("lbl-email"));
WebElement emailAddressField = driver.findElement(with(By.tagName("input")).near(emailAddressLabel));

4. TakeElementScreenshot

In Selenium 3, there was a provision to capture a screenshot of the entire web page. Selenium 4 onwards, there is a new option to capture screenshots of a particular WebElement. Hence, there is no need to use third-party tools like Shutterbug, Ashot, etc. (like in Selenium 3) for capturing a screenshot of WebElement.

The newly introduced method in Selenium 4 captures the screenshot of an element for the current browsing context. The screenshot returned by the WebDriver endpoint is encoded in the Base64 format.

This is how you can capture WebElement Screenshot in Selenium 4 (for Java):

import io.github.bonigarcia.wdm.WebDriverManager;
import org.apache.commons.io.FileUtils;
import org.openqa.selenium.By;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;

public class ScreenshotDemo {

	public static void main(String[] args) {
		WebDriverManager.chromedriver().setup();
		WebDriver driver = new ChromeDriver();
		driver.manage().window().maximize();
		driver.get("https://www.selenium.dev/");

		try {

			WebElement logo = driver.findElement(By.xpath("//*[@id='td-cover-block-0']/div/div/div/div/h1"));
			File source = ((TakesScreenshot) logo).getScreenshotAs(OutputType.FILE);
			FileUtils.copyFile(source, new File("./Screenshots/logo" + System.currentTimeMillis() + ".png"));
		} catch (Exception e) {
			System.out.println(e.getMessage());
		}

		System.out.println("The Screenshot is taken and saved under Screenshots folder");
		driver.quit();
	}

}

The output of the above program is

The picture will be saved in the Screenshots folder as shown below:

Below is the image of the screenshots.

5. New additions to the Actions Class

Actions Class in Selenium provides several methods for performing a single action or a series of actions on the WebElements present in the DOM. Mouse actions (e.g., click, double click, etc.) and Keyboard actions (e.g., keyUp, keyDown, sendKeys) are the two broad categories of Actions.
For demonstration, we will post the examples demonstrated in the Action class in the Selenium blog from Selenium 3 to Selenium 4.

With Selenium 4, new methods are added to the Actions class, which replaces the classes under the org.openqa.selenium.interactions package.

  • click(WebElement) is the new method added to the Actions class and it serves as the replacement of moveToElement(onElement).click() method.

Like the method in the versions before Selenium 4, click(WebElement) is used for clicking a web element.

  • doubleClick(WebElement)

This method is added to replace moveToElement(element).doubleClick(). It will perform a double-click on an element.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.openqa.selenium.Alert;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.interactions.Actions;

public class DoubleClick {
    public static void main(String[] args) {

        WebDriverManager.chromedriver().setup();
        WebDriver driver = new ChromeDriver();

        // Navigate to Url
        driver.get("https://demo.guru99.com/test/simple_context_menu.html");

        // Store 'doubleClickButton' button web element
        WebElement doubleClickButton = driver.findElement(By.xpath("//*[@id='authentication']/button"));
        Actions actionProvider = new Actions(driver);

        // Perform double-click action on the element
        actionProvider.doubleClick(doubleClickButton).build().perform();
        Alert alert = driver.switchTo().alert();
        System.out.println("Alert Text\n" +alert.getText());
        alert.accept();
        
        driver.close();
    }
}

The output of the above program is

  • clickAndHold(WebElement)

This method will replace the moveToElement(onElement).clickAndHold(). It is used to click on an element without releasing the click.

  • contextClick(WebElement)

This method will replace moveToElement(onElement).contextClick(). It will perform the right-click operation.

  • release()

This method (user for releasing the pressed mouse button) was initially a part of org.openqa.selenium.interactions.ButtonReleaseAction class. Now with the updated version of Selenium, it has been moved to the Actions class.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.interactions.Actions;

public class clickAndHold {
    public static void main(String[] args) {

        WebDriverManager.chromedriver().setup();
        WebDriver driver = new ChromeDriver();

        // Navigate to Url
        driver.get("https://crossbrowsertesting.github.io/drag-and-drop.html");
        driver.manage().window().maximize();

        // Find element xpath which we need to drag
        WebElement from = driver.findElement(By.id("draggable"));

        // Find element xpath where we need to drop
        WebElement to = driver.findElement(By.id("droppable"));

        Actions actionProvider = new Actions(driver);

        // Perform click-and-hold action on the element
        actionProvider.clickAndHold(from).build().perform();

       // Move to drop Webelement
        actionProvider.clickAndHold(to).build().perform();

        //Release drop element
        actionProvider.release(to).build().perform();
    }
}

The output of the above program is

6. Deprecation of Desired Capabilities

In Selenium 3, desired Capabilities were primarily used in the test scripts to define the test environment (browser name, version, operating system) for execution on the Selenium Grid.

In Selenium 4, capabilities objects are replaced with Options. This means testers now need to create an Options object, set test requirements, and pass the object to the Driver constructor.

Listed below are the Options objects to be used going forward for defining browser-specific capabilities:

Firefox – FirefoxOptions
Chrome – ChromeOptions
Internet Explorer (IE) – InternetExplorerOptions
Microsoft Edge – EdgeOptions
Safari – SafariOptions

Below is an example of Options

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import io.github.bonigarcia.wdm.WebDriverManager;

public class ChromeOptionsHeadless {

	public static void main(String[] args) {

		WebDriverManager.chromedriver().setup();

		ChromeOptions chromeOptions = new ChromeOptions();
		chromeOptions.setBrowserVersion("100");
		chromeOptions.setPlatformName("Windows 10");

		WebDriver driver = new ChromeDriver(chromeOptions);

		driver.get("https://duckduckgo.com/");
		System.out.println("Title of Page :" + driver.getTitle());

		// Close the driver
		driver.close();

	}
}

The output of the above program is

Similarly, we can create the action class for other browsers like Firefox.

FirefoxOptions options = new FirefoxOptions();
 
// Create an object of WebDriver class and pass the Firefox Options object as an argument
WebDriver driver = new FirefoxDriver(options);

7. Chrome Dev Tools

In the new version of Selenium, they have made some internal changes in the API. Earlier in Selenium 3, the Chrome driver extends directly to the Remote Web Driver class. But now in Selenium 4, the Chrome driver class extends to Chromium Driver. Chromium Driver class has some predefined methods to access the dev tool.

Note: Chromium Driver extends the Remote Web driver class.

By using the API, we can perform the following operations:

  • Enable Network Offline
  • Enable Network Online
  • Get Console Logs
  • Load Insure Web Site

How to parameterize Tests in JUnit5

HOME

JUnit5  enables us to execute a single test method multiple times with a different sets of data. This is called Parameterization. Parameterized Tests are declared just like regular @Test methods but use the @ParameterizedTest annotation.

This article shows you how to run a test multiple times with different arguments, so-called ‘Parameterized Tests’, let’s see the following ways to provide arguments to the test:

  • @ValueSource
  • @EnumSource
  • @MethodSource
  • @CsvSource
  • @CsvFileSource
  • @ArgumentsSource

We need to add junit-jupiter-params to support parameterized tests. In the case of Maven, add the dependency to POM.xml

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-params</artifactId>
    <version>5.8.2</version>
    <scope>test</scope>
</dependency>

In case of Gradle, add the dependency as

testCompile("org.junit.jupiter:junit-jupiter-params:5.8.2")

1. @ValueSource

Let us start with a simple example. The following example demonstrates a parameterized test that uses the @ValueSource annotation to specify an integer array as the source of arguments. The following @ParameterizedTest method will be invoked three times, with the values 5,6, and 0 respectively.

@ParameterizedTest
@ValueSource(ints = {5, 6, 0})
void test_int_arrays(int b) {

    int a= 5;
    int sum = a + b;
    assertTrue(sum>8);
 }

When executing the above-parameterized test method, each invocation will be reported separately.

The output of the above program is:

One of the limitations of value sources is that they only support these types:

  • short (with the shorts attribute)
  • byte (bytes attribute)
  • int (ints attribute)
  • long (longs attribute)
  • float (floats attribute)
  • double (doubles attribute)
  • char (chars attribute)
  • java.lang.String (strings attribute)
  • java.lang.Class (classes attribute)

Also, we can only pass one argument to the test method each time.

In the below example, an array of strings is passed as the argument to the Parameterized Test.

@ParameterizedTest(name = "#{index} - Run test with args={0}")
@ValueSource(strings = {"java", "python", "javascript","php"})
void test_string_arrays(String arg) {
        assertTrue(arg.length() > 1);
}

The output of the above program is:

@NullSource

It provides a single null an argument to the annotated @ParameterizedTest method.

    @ParameterizedTest()
    @NullSource
    void nullString(String text) {
        assertTrue(text == null);
    }
    

The output of the above program is:

@EmptySource

It provides a single empty argument to the annotated @ParameterizedTest method of the following types:

  • java.lang.String
  • java.util.List
  • java.util.Set
  • java.util.Map
  • primitive arrays (e.g. int[])
  • object arrays (e.g. String[])
 @ParameterizedTest
    @EmptySource
    void testMethodEmptySource(String argument) {
        assertTrue(StringUtils.isEmpty(argument));
        assertTrue(StringUtils.isBlank(argument));
    }

The output of the above program is:

@NullAndEmptySource

We can pass empty or null values into the test via @EmptySource, @NullSource, or @NullAndEmptySource (since JUnit 5.4).

Let’s see the following example to test an isEmpty() method.

import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.EmptySource;
import org.junit.jupiter.params.provider.NullSource;
import org.junit.jupiter.params.provider.ValueSource;
import static org.junit.jupiter.api.Assertions.*;

public class ParameterizedTestDemo {

    @ParameterizedTest(name = "#{index} - isEmpty()? {0}")
    @NullSource
    @EmptySource
    @ValueSource(strings = { " ", "   ", "\t", "\n","a"})
    void nullEmptyAndBlankStrings(String text) {
        assertTrue(text == null || text.trim().isEmpty());
    }
}

The parameterized test method result in seven invocations: 1 for null, 1 for the empty string, 4 for the explicit blank strings supplied via @ValueSource, and 1 non-blank string “a” supplied via @ValueSource.

The output of the above program is:

2. @EnumSource

@EnumSource provides a convenient way to use Enum constants.

public class EnumParameterizedTest {

    enum Error {
         Request_Invalid,
         Request_Timeout,
         RequestHeader_Invalid,
         Concurrency_Failed,
         ExternalCall_Failed,
         Schema_Invalid,
         Authentication_Failed;
    }

    @ParameterizedTest
    @EnumSource(Error.class)
    void test_enum(Error error) {
       assertNotNull(error);
    }
}

The output of the above program is:

The annotation provides an optional names attribute that lets you specify which constants shall be used, like in the following example. If omitted, all constants will be used.

    @ParameterizedTest(name = "#{index} - Is Error contains {0}?")
    @EnumSource(value = Error.class, names = {"Request_Invalid", "ExternalCall_Failed", "Concurrency_Failed", "Authentication_Failed"})
    void test_enum_include(Error error) {
       assertTrue(EnumSet.allOf(Error.class).contains(error));
    }

The output of the above program is:

The @EnumSource annotation also provides an optional mode attribute that enables fine-grained control over which constants are passed to the test method. For example, you can exclude names from the enum constant pool or specify regular expressions as in the following examples.

 @ParameterizedTest
 @EnumSource(value = Error.class, mode = EnumSource.Mode.EXCLUDE, names = {"Request_Invalid", "Request_Timeout", "RequestHeader_Invalid"})
    void test_enum_exclude(Error error) {
        EnumSet<Error> excludeRequestRelatedError = EnumSet.range(Error.Concurrency_Failed, Error.Authentication_Failed);
        assertTrue(excludeRequestRelatedError.contains(error));
  }

The output of the above program is:

EnumSource.Mode.EXCLUDE – It selects all declared enum constants except those supplied via the names attribute.

EnumSource.Mode.MATCH_ALL – It selects only those enum constants whose names match any pattern supplied via the names attribute.

  @ParameterizedTest
    @EnumSource(mode = EnumSource.Mode.MATCH_ALL, names = "^.*Invalid")
    void test_match(Error error) {
        assertTrue(error.name().contains("Invalid"));
    }

The output of the above program is

3. @MethodSource

@MethodSource allows you to refer to one or more factory methods of the test class or external classes.

Factory methods within the test class must be static unless the test class is annotated with @TestInstance(Lifecycle.PER_CLASS); whereas, factory methods in external classes must always be static. In addition, such factory methods must not accept any arguments.

If you only need a single parameter, you can return a Stream of instances of the parameter type as demonstrated in the following example.

   @ParameterizedTest(name = "#{index} - Test with String : {0}")
    @MethodSource("stringProvider")
    void test_method_string(String arg) {
        assertNotNull(arg);
    }

    // this need static
    static Stream<String> stringProvider() {
        return Stream.of("java", "junit5", null);
    }

The output of the above program is

If you do not explicitly provide a factory method name via @MethodSource, JUnit Jupiter will search for a factory method that has the same name as the current @ParameterizedTest method by convention. This is demonstrated in the following example.

    @ParameterizedTest(name = "#{index} - Test with String : {0}")
    @MethodSource
    void test_no_factory_methodname(String arg) {
        assertNotNull(arg);
    }

    static Stream<String> test_no_factory_methodname() {
        return Stream.of("java", "junit5", null);
    }

The output of the above program is

Streams for primitive types (DoubleStream, IntStream, and LongStream) are also supported as demonstrated by the following example.

 @ParameterizedTest(name = "#{index} - Test with Int : {0}")
    @MethodSource("rangeProvider")
    void test_method_int(int arg) {
        assertTrue(arg < 6);
    }
    
    static IntStream rangeProvider() {
        return IntStream.range(0, 6);
    }

The output of the above program is

If a parameterized test method declares multiple parameters, you need to return a collection, stream, or array of Arguments instances or object arrays as shown below.

    @ParameterizedTest
    @MethodSource("stringIntAndListProvider")
    void testWithMultiArgMethodSource(String str, int num, List<String> list) {
        assertEquals(5, str.length());
        assertTrue(num >=1 && num <=2);
        assertEquals(2, list.size());
    }

    static Stream<Arguments> stringIntAndListProvider() {
        return Stream.of(
                arguments("apple", 1, Arrays.asList("a", "b")),
                arguments("lemon", 2, Arrays.asList("x", "y"))
        );
    }

The output of the above program is

4. @CsvSource

@CsvSource allows you to express argument lists as comma-separated values (i.e., CSV String literals). Each string provided via the value attribute in @CsvSource represents a CSV record and results in one invocation of the parameterized test.

import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import static org.junit.jupiter.api.Assertions.assertEquals;

public class CSVParameterizedTest {

    @ParameterizedTest
    @CsvSource({
            "java,      4",
            "javascript,   7",
            "python,    6",
            "HTML,    4",
    })


    void test(String str, int length) {
        assertEquals(length, str.length());
    }
}

The output of the above program is

5. @CsvFileSource

@CsvFileSource lets us use comma-separated value (CSV) files from the classpath or the local file system. Each record from a CSV file results in one invocation of the parameterized test. The first record may optionally be used to supply CSV headers.

csvdemo.csv

    @ParameterizedTest
    @CsvFileSource(resources = "/csvdemo.csv")

    void testLength(String str, int length) {
        Assertions.assertEquals(length, str.length());
    }

The output of the above program is

csv file with the heading

JUnit can ignore the headers via the numLinesToSkip attribute.

    @ParameterizedTest
    @CsvFileSource(files = "src/test/resources/csvdemo.csv",numLinesToSkip = 1)

    void testStringLength(String str, int length) {
        Assertions.assertEquals(length, str.length());
    }

The output of the above program is

If you would like the headers to be used in the display names, you can set the useHeadersInDisplayName attribute to true. The examples below demonstrate the use of useHeadersInDisplayName.

 @ParameterizedTest(name = "[{index}] {arguments}")
    @CsvFileSource(files = "src/test/resources/csvdemo.csv",useHeadersInDisplayName = true)

    void testStringLength1(String str, int length) {
        assertEquals(length, str.length());
    }

The output of the above program is

6. @ArgumentsSource

@ArgumentsSource can be used to specify a custom, reusable ArgumentsProvider. Note that an implementation of ArgumentsProvider must be declared as either a top-level class or as a static nested class.

import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.ArgumentsProvider;
import java.util.stream.Stream;

public class CustomArgumentsProvider implements ArgumentsProvider {

    @Override
    public Stream<? extends Arguments>
    provideArguments(ExtensionContext extensionContext) throws Exception {
        return Stream.of("java", "junit5", "junit4", null).map(Arguments::of);
    }
}
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ArgumentsSource;
import static org.junit.jupiter.api.Assertions.assertNotNull;

public class ArgumentsSourceTest {

    @ParameterizedTest
    @ArgumentsSource(CustomArgumentsProvider.class)
    void test_argument_custom(String arg) {
        assertNotNull(arg);
    }
}

The output of the above program is

Congratulation. We have understood parameterization in JUnit5 tests. Happy Learning!!

How to run JUnit5 tests in order

HOME

The general practices say that automated tests should be able to run independently and with no specific order, as well as the result of the test should not depend on the results of previous tests. But there are situations where a specific order of test execution can be justified, especially in integration or end-to-end tests. The test methods don’t follow a specific order by default to execute the tests. The test cases need not necessarily execute in the order in which they have been written.

There are different ways or modes to set the order of execution for the test cases.  This article shows how to control the JUnit 5 test execution order via the following MethodOrderer classes:

  • DisplayName – sorts test methods alphanumerically based on their display names
  • MethodName – sorts test methods alphanumerically based on their names and formal parameter lists
  • Alphanumeric – sorts test methods alphanumerically based on their names and formal parameter lists. This is deprecated from JUnit Version 5.7 onwards
  • OrderAnnotation – sorts test methods numerically based on values specified via the @Order annotation
  • Random – orders test methods pseudo-randomly and support the configuration of a custom seed
  • Custom Order – A custom ordering sequence can be implemented by the interface MethodOrderer and providing it as the argument to @TestMethodOrder.

Let us create JUnit5 Tests and execute them.

public class OrderRandomDemo {

    @Test
    void test_Add() {

        System.out.println("test_Add()");
        assertEquals(10, 3 + 7);
    }

    @Test
    void test_Subtract() {

        System.out.println("test_Subtract()");
        assertEquals(10, 14 - 4);
    }

    @Test
    void test_Multiply() {

        System.out.println("test_Multiply()");
        assertEquals(10, 5 * 2);
    }

    @Test
    void test_Divide() {

        System.out.println("test_Divide()");
        assertEquals(10, 30 / 3);
    }

    @Test
    void test_IsEven() {

        System.out.println("test_IsEven()");
        assertEquals(0, 10%2);
    }

}

The output of the above program

1. DisplayName

It sorts test methods alphanumerically based on their display names. Test Method can be anything annotated with @Test, @RepeatedTest, @ParameterizedTest, @TestFactory, or @TestTemplate.

@TestMethodOrder(MethodOrderer.DisplayName.class)

@TestMethodOrder is a type-level annotation that is used to configure a MethodOrderer for the test methods of the annotated test class or test interface.

MethodOrderer defines the API for ordering the test methods in a given test class.

Test Method – It is any method annotated with @Test, @RepeatedTest, @ParameterizedTest, @TestFactory, or @TestTemplate.

DisplayName.class – MethodOrderer that sorts methods alphanumerically based on their names using String.compareTo(String).
If two methods have the same name, String representations of their formal parameter lists will be used as a fallback for comparing the methods.

An example of sorting the tests based on their display names.

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import static org.junit.jupiter.api.Assertions.assertEquals;

@TestMethodOrder(MethodOrderer.DisplayName.class)
public class DisplayNameOrderedTests {

    @DisplayName("C")
    @Test
    void test_Add() {

        System.out.println("test_Add()");
        assertEquals(10, 3 + 7);
    }


    @DisplayName("E")
    @Test
    void test_Multiply() {

        System.out.println("test_Multiply()");
        assertEquals(10, 5 * 2);
    }

    @DisplayName("A")
    @Test
    void test_Divide() {

        System.out.println("test_Divide()");
        assertEquals(10, 30 / 3);
    }


    @DisplayName("D")
    @Test
    void test_Subtract() {

        System.out.println("test_Subtract()");
        assertEquals(10, 18 - 8);
    }

    @DisplayName("B")
    @Test
    void test_IsEven() {

        System.out.println("test_IsEven()");
        assertEquals(0, 18%2);
    }
}

We can see that the test methods are sorted alphanumerically based on their display name starting from A to E. The output of the above program

2. MethodName

This annotation sorts methods alphanumerically based on their names using String.compareTo(String).
If two methods have the same name, String representations of their formal parameter lists will be used as a fallback for comparing the methods.

@TestMethodOrder(MethodOrderer.MethodName.class)

Let us see an example of MethodName.

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import static org.junit.jupiter.api.Assertions.assertEquals;

@TestMethodOrder(MethodOrderer.MethodName.class)
class MethodNameOrderedTests {

    @Test
    void testE() {

        System.out.println("testE");
        assertEquals(10, 3 + 7);
    }

    @Test
    void testA() {

        System.out.println("testA");
        assertEquals(10, 14 - 4);
    }

    @Test
    void testC() {
        System.out.println("testC");
        assertEquals(10, 5 * 2);
    }

    @Test
    void testB() {
        System.out.println("testB");
        assertEquals(10, 30 / 3);
    }

    @Test
    void testD() {
        System.out.println("testD");
        assertEquals(10, 10 + 0);
    }

}

The output of the above program

3. OrderAnnotation

This sorts test method numerically based on values specified via the @Order annotation.
Any methods that are assigned the same order value will be sorted arbitrarily adjacent to each other.
When any method is not annotated with @Order, it will be assigned the default order value, which will effectively cause them to appear at the end of the sorted list.

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)

Let us see an example of OrderAnnotation.

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import static org.junit.jupiter.api.Assertions.assertEquals;

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class OrderAnnotationDemo {

    @Test
    @Order(3)
    void test_Add() {

        System.out.println("test_Add()");
        assertEquals(10, 3 + 7);
    }

    @Test
    @Order(4)
    void test_IsOdd() {
        System.out.println("test_IsOdd()");
        assertEquals(1, 11%2);
    }

    @Test
    void test_Subtract() {
        System.out.println("test_Subtract()");
        assertEquals(10, 14 - 4);
    }

    @Test
    @Order(1)
    void test_Multiply() {
        System.out.println("test_Multiply()");
        assertEquals(10, 5 * 2);
    }

    @Test
    @Order(4)
    void test_Divide() {
        System.out.println("test_Divide()");
        assertEquals(10, 30 / 3);
    }

    @Test
    @Order(2)
    void test_IsEven() {
        System.out.println("test_IsEven()");
        assertEquals(0, 10%2);
    }
}

Here, test_Subtract() is not assigned any order value, so it is displayed as the last one in the last.

4. Random

These sorts of test methods are pseudo-randomnly.

@TestMethodOrder(MethodOrderer.Random.class)

Let us create a program to show the random order of tests in JUnit5.

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import static org.junit.jupiter.api.Assertions.assertEquals;

@TestMethodOrder(MethodOrderer.Random.class)
public class OrderRandomDemo {

    @Test
    void test_Add() {

        System.out.println("test_Add()");
        assertEquals(10, 3 + 7);
    }

    @Test
    void test_Subtract() {

        System.out.println("test_Subtract()");
        assertEquals(10, 14 - 4);
    }

    @Test
    void test_Multiply() {

        System.out.println("test_Multiply()");
        assertEquals(10, 5 * 2);
    }

    @Test
    void test_Divide() {

        System.out.println("test_Divide()");
        assertEquals(10, 30 / 3);
    }

    @Test
    void test_IsEven() {

        System.out.println("test_IsEven()");
        assertEquals(0, 10%2);
    }
}

The output of the above program

5. Custom Order

We can define our own custom ordering sequence by implementing the interface MethodOrderer and providing it as the argument to @TestMethodOrder.

Here, the tests are arranged in descending method order.

import org.junit.jupiter.api.MethodDescriptor;
import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.MethodOrdererContext;

public class DescendingMethodOrder implements MethodOrderer {

    @Override
    public void orderMethods(MethodOrdererContext context) {
        context.getMethodDescriptors().sort((MethodDescriptor m1, MethodDescriptor m2) ->
                m2.getMethod().getName().compareTo(m1.getMethod().getName()));
    }
    
}

Now, test the above custom order.

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import static org.junit.jupiter.api.Assertions.assertEquals;

@TestMethodOrder(DescendingMethodOrder.class)
public class CustomOrderTests {

    @Test
    void test_Add() {
        System.out.println("test_Add()");
        assertEquals(10 , 4 + 6);
    }

    @Test
    void test_Subtract() {
        System.out.println("test_Subtract()");
        assertEquals(10 , 17 - 7);
    }

    @Test
    void test_Multiply() {
        System.out.println("test_Multiply()");
        assertEquals(10 , 5 * 2);
    }

    @Test
    void test_Divide() {
        System.out.println("test_Divide()");
        assertEquals(10 , 20/2);
    }

    @Test
    void test_IsEven() {
        System.out.println("test_IsEven()");
        assertEquals(0 , 20%2);
    }
}

Notice the test output. The tests are executed in descending order. The result of the above program is

Test Classes Ordering

  1. ClassName: sorts test classes alphanumerically based on their fully qualified class names.
  2. DisplayName: sorts test classes alphanumerically based on their display names (see display name generation precedence rules).
  3. OrderAnnotation: sorts test classes numerically based on values specified via the @Order annotation.
  4. Random: orders test classes pseudo-randomly and support the configuration of a custom seed.

The configured ClassOrderer will be applied to all top-level test classes (including static nested test classes) and @Nested test classes.

1. ClassName

The Test Classes are sorted alphanumerically based on their fully qualified class names.

package JUnit5;

import org.junit.jupiter.api.ClassOrderer;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestClassOrder;

import static org.junit.jupiter.api.Assertions.assertEquals;

@TestClassOrder(ClassOrderer.ClassName.class)
public class ClassNameOrderTests {

    @Nested
    class Addition {

        @Test
        void test_Add() {
            System.out.println("test_Add()");
            assertEquals(10, 3 + 7);
        }
    }

    @Nested
    class IsEven {

        @Test
        void test_IsEven() {
            System.out.println("test_IsEven()");
            assertEquals(0, 10 % 2);
        }
    }

    @Nested
    class Subtraction {

        @Test
        void test_Subtract() {
            System.out.println("test_Subtract()");
            assertEquals(9, 14 - 5);
        }
    }

    @Nested
    class Multiply {

        @Test
        void test_Multiply() {
            System.out.println("test_Multiply()");
            assertEquals(10, 5 * 2);
        }
    }
}

The result of the above program is

2. DisplayName

It sorts test classes alphanumerically based on their display names.

package JUnit5;

import org.junit.jupiter.api.*;

import static org.junit.jupiter.api.Assertions.assertEquals;

@TestClassOrder(ClassOrderer.DisplayName.class)
public class ClassDisplayNameTests {

    @Nested
    @DisplayName("B")
    class AppFlowTests {

        @Test
        void test_Add() {
            System.out.println("test_Add()");
            assertEquals(10, 6 + 4);
        }
    }

    @Nested
    @DisplayName("C")
    class TearDownTests {

        @Test
        void test_Subtract() {
            System.out.println("test_Subtract()");
            assertEquals(10, 15 - 5);

        }
    }

    @Nested
    @DisplayName("A")
    class SetupTests {

        @Test
        void test_Multiply() {
            System.out.println("test_Multiply()");
            assertEquals(10, 5 * 2);
        }
    }
}

The result of the above program is

3. OrderAnnotation in Class

The test classes are sorted numerically based on values specified via the @Order annotation.

import org.junit.jupiter.api.*;

import static org.junit.jupiter.api.Assertions.assertEquals;

@TestClassOrder(ClassOrderer.ClassName.class) //sorts test classes alphanumerically based on their fully qualified class names.
public class ClassOrderedTests {

    @Nested
    @Order(2)
    class AppFlowTests {

        @Test
        void test_Add() {
            System.out.println("test_Add()");
            assertEquals(10, 3 + 7);
        }
    }

    @Nested
    @Order(3)
    class TearDownTests {

        @Test
        void test_Subtract() {
            System.out.println("test_Subtract()");
            assertEquals(9, 14 - 5);
        }
    }

    @Nested
    @Order(1)
    class SetupTests {

        @Test
        void test_Multiply() {
            System.out.println("test_Multiply()");
            assertEquals(10, 5 * 2);
        }
    }
}

The result of the above program is

4. Random

The test classes are sorted pseudo-randomly and support the configuration of a custom seed.

import org.junit.jupiter.api.*;
import static org.junit.jupiter.api.Assertions.assertEquals;

@TestClassOrder(ClassOrderer.Random.class)
public class ClassRandomTests {

    @Nested
    class Addition {

        @Test
        void test_Add() {
            System.out.println("test_Add()");
            assertEquals(10, 3 + 7);
        }
    }

    @Nested
    class IsEven {

        @Test
        void test_IsEven() {
            System.out.println("test_IsEven()");
            assertEquals(0, 10 % 2);
        }
    }

    @Nested
    class Subtraction {

        @Test
        void test_Subtract() {
            System.out.println("test_Subtract()");
            assertEquals(9, 14 - 5);
        }
    }

    @Nested
    class Multiply {

        @Test
        void test_Multiply() {
            System.out.println("test_Multiply()");
            assertEquals(10, 5 * 2);
        }
    }
}

The result of the above program is

Congratulation!! We have gone through different types of ordering in JUnit5. Happy Learning!!