Serenity Reports

HOME

Serenity Reports are living documentation that contains the meaningful report for each Test. It illustrated narrative reports that document and describe what your application does and how it works.

Chapter 1  Serenity Report for Web Application with Cucumber6 and Junit
Chapter 2  Serenity Emailable HTML Report
Chapter 3  How to report Manual Tests in Serenity Report
Chapter 4  How to attach Test Evidence to Manual Tests in Serenity Report
Chapter 5  How to manage screenshots in Serenity Report
Chapter 6  Serenity Emailable Report in Gradle
Chapter 7  How to embed Custom Data in Serenity Report

Allure Reports

HOME

Allure Framework is a lightweight, flexible multi-language test report tool that not only displays a very concise representation of what has been tested in a neat web report form but also allows everyone involved in the development process to extract the most useful information from everyday test execution.

Chapter 1 What is Allure Report?
Chapter 2 Integration of Allure Report with Selenium and JUnit4
Chapter 3 Integration of Allure Report with Selenium and JUnit5
Chapter 4 Integration of Allure Report with Selenium and TestNG
Chapter 5 Allure Report with Cucumber5, Selenium and JUnit4
Chapter 6 Allure Report with Cucumber5, Selenium and TestNG
Chapter 7 Integration of Allure Report with Rest Assured and JUnit4
Chapter 8 Integration of Allure Report with Rest Assured and TestNG
Chapter 9 Gradle – Allure Report for Selenium and TestNG
Chapter 10 Gradle – Allure Report for Selenium and JUnit4
Chapter 11 Gradle – Allure Report for Cucumber, Selenium and TestNG
Chapter 12 Integration of Allure Report with Jenkins

Gradle – Integration of Selenium and JUnit5

HOME

The previous tutorial explained How to create Gradle project with Selenium and JUnit4 in a Gradle project. In this tutorial, I will explain how we can set up a Gradle project with Selenium and JUnit5.

Pre Requisite:

  1. Java 8 or above installed
  2. Eclipse or IntelliJ IDE installed
  3. Gradle Installed
  4. Environment variables JAVA_HOME and GRADLE_HOME are correctly configured

This framework consists of:

  1. Java 11
  2. JUnit Jupiter – 5.8.2
  3. JUnit Jupiter Engine – 5.8.2
  4. Gradle – 7.3.3 (Build Tool)
  5. Selenium – 4.3.0

Steps to set up Gradle Java Project for Selenium and JUnit5

  1. Download and Install Java on the system
  2. Download and setup Eclipse IDE on the system
  3. Setup Gradle on System
  4. Create a new Gradle Project
  5. Add Selenium and JUnit5 dependencies to the Gradle project
  6. Create Pages and Test Code for the pages
  7. Run the tests from Command Line
  8. Gradle Report generation

Project Structure

Implementation Steps

Step 1- Download and Install Java

Selenium needs Java to be installed on the system to run the tests. Click here to know How to install Java.

Step 2 – Download and setup Eclipse IDE on the system

The Eclipse IDE (integrated development environment) provides strong support for Java developers. Click here to know How to install Eclipse.

Step 3 – Setup Gradle

To build a test framework, we need to add several dependencies to the project. This can be achieved by any build tool. I have used Gradle Build Tool. Click here to know How to install Gradle.

Step 4 – Create a new Gradle Project

If you want to create the Gradle project from Eclipse IDE, click here to know How to create a Gradle Java project.

Step 5 – Add Selenium and JUnit5 dependencies to the Gradle project
/*
 * This file was generated by the Gradle 'init' task.
 *
 */

plugins {
    // Apply the application plugin to add support for building a CLI application in Java.
    id 'application'
}

repositories {
    // Use Maven Central for resolving dependencies.
    mavenCentral()
}

java {
    sourceCompatibility = 11
    targetCompatibility = 11
}

dependencies {
    // Use JUnit Jupiter for testing.
    testImplementation 'org.junit.jupiter:junit-jupiter:5.8.2'
    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.8.2'

    implementation 'com.google.guava:guava:30.1.1-jre'
    implementation 'org.seleniumhq.selenium:selenium-java:4.4.0'
    implementation 'io.github.bonigarcia:webdrivermanager:5.3.0'
}

application {
    // Define the main class for the application.
    mainClass = 'com.example.App'
}

tasks.named('test') {
    // Use JUnit Platform for unit tests.
    useJUnitPlatform()  {
    }
    
 testLogging {
        events "passed", "skipped", "failed"
        showStandardStreams = true
    }
 
    systemProperties System.properties
    reports.html.setDestination(file("$projectDir/GradleReports"))
}  


Step 6 – Create Pages and Test Code for the pages

We have used PageFactory model to build the tests. I have created a package named pages and created the page classes in that folder. Page class contains the locators of each web element present on that particular page along with the methods of performing actions using these web elements.

This is the BasePage that contains the PageFactory.initElements.

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.support.PageFactory;

public class BasePage {
	
	  public WebDriver driver;

	  public BasePage(WebDriver driver) {
		  this.driver = driver;
		  PageFactory.initElements(driver,this);
	}

}

Below is the code for LoginPage and HomePage

LoginPage

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;

public class LoginPage extends BasePage{
	
	 public LoginPage(WebDriver driver) {
		 super(driver);		
    }
	
	@FindBy(name = "username")
    public WebElement userName;
 
    @FindBy(name = "password")
    public WebElement password;
    
    @FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/form/div[1]/div/span")
    public WebElement missingUsernameErrorMessage;
    
    @FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/form/div[1]/div/span")
    public WebElement missingPasswordErrorMessage;
 
    @FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/form/div[3]/button")
    public WebElement login;
 
    @FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/div/div[1]/div[1]/p")
    public  WebElement errorMessage;
          
    public String getMissingUsernameText() {
        return missingUsernameErrorMessage.getText();
    }
    
    public String getMissingPasswordText() {
        return missingPasswordErrorMessage.getText();
    }
    
    public String getErrorMessage() {
        return errorMessage.getText();
    }
   
    public void login(String strUserName, String strPassword) {
 
    	userName.sendKeys(strUserName);
    	password.sendKeys(strPassword);
    	login.click();
 
    }

}

HomePage

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;

public class HomePage extends BasePage {

	public HomePage(WebDriver driver) {
		super(driver);
	}

	  @FindBy(xpath = "//*[@id='app']/div[1]/div[2]/div[2]/div/div[1]/div[1]/div[1]/h5")
	  public  WebElement homePageUserName;

	    public String getHomePageText() {
	       return homePageUserName.getText();
   }

}

Here, we have BaseTests Class also which contains the common methods needed by other test pages.

import java.time.Duration;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import io.github.bonigarcia.wdm.WebDriverManager;

public class BaseTests {
	
	public WebDriver driver;
	public final static int TIMEOUT = 10;
    
	@BeforeEach
    public void setup() {
    	WebDriverManager.chromedriver().setup();
	    driver = new ChromeDriver();
	    driver.manage().window().maximize();
	    driver.get("https://opensource-demo.orangehrmlive.com/");	    
	    driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(TIMEOUT));

    }
 
    @AfterEach
    public void tearDown() {
        driver.quit();
    }
    
}

LoginPageTests

import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Disabled;

public class LoginPageTests extends BaseTests{
	 
    @ParameterizedTest
    @CsvSource({
            "admin$$,admin123",
            "Admin,admin123!!",
            "admin123,Admin",
            "%%%%%,$$$$$$"})
    public void invalidCredentials(String username, String password) {
   
	    LoginPage objLoginPage = new LoginPage(driver);
    	objLoginPage.login(username, password);
    	 
    	// Verify Error Message
    	 assertEquals("Invalid credentials",objLoginPage.getErrorMessage());
    
    }
    
    @Test
    public void validLogin() {
   
	    LoginPage objLoginPage = new LoginPage(driver);
    	objLoginPage.login("Admin", "admin123");
    	 
    	HomePage objHomePage = new HomePage(driver);
    	
    	// Verify Home Page
    	 assertEquals("Employee Information",objHomePage.getHomePageText());
    
    }
    
    @Test 
    public void missingUsername() {
   
	    LoginPage objLoginPage = new LoginPage(driver);
    	objLoginPage.login("", "admin123");
    	     	
    	// Verify Error Message
   	     assertEquals("Invalid credentials",objLoginPage.getMissingUsernameText());
   	        
    }
	
    @Test @Disabled
    public void missingPassword() {
   
	    LoginPage objLoginPage = new LoginPage(driver);
    	objLoginPage.login("admin", "");
    	    	
    	// Verify Error Message
   	     assertEquals("Invalid credentials",objLoginPage.getMissingPasswordText());
    
    }    
   
}

Step 7 – Run the tests from Command Line

Note:- As you can see, my project has two parts – GradleSeleniumJUnit5_Demo and app.

Go to the app project and run the tests, using the below command.

gradle clean test

The output of the above program is

Step 8 – Gradle Report generation

Once the test execution is finished, refresh the project. We will see a folder – GradleReports. This report is generated when the tests are executed through the command line.

This folder contains index.html.

Right-click on index.html and select open with Web Browser. This report shows the summary of all the tests executed. As you can see that Failed tests are selected (highlighted in blue), so the name of the test failed along with the class name is displayed here.

This report contains detailed information about the failed test, which is shown below.

This shows the list of all the tests – passed, failed, or ignored.

Extent Reports Version 5 for Cucumber7 and JUnit5

HOME

The previous tutorial explained the steps to generate ExtentReports Version for Cucumber7 with TestNG. This tutorial explains the steps needed to be followed to generate an ExtentReports Version5 for Cucumber 7.

Pre-Requisite:

  • Java 8 or higher is needed for ExtentReport5
  • Maven or Gradle
  • JAVA IDE (like Eclipse, IntelliJ, or soon)
  • Cucumber Eclipse plugin (in case using Eclipse)

Project Structure

There is a tutorial that explains the steps to integrate Cucumber 7 with JUnit5. Please refer to this tutorial.

Now, let us add the extra steps needed to generate the ExtentRport Version5.

New Features in ExtentReports Version 5

Report Attachments 

To add attachments, like screen images, two settings need to be added to the extent.properties. Firstly property, named screenshot.dir, is the directory where the attachments are stored. Secondly is screenshot.rel.path, which is the relative path from the report file to the screenshot directory.

extent.reporter.spark.out=Reports/Spark.html
 
screenshot.dir=/Screenshots/
screenshot.rel.path=../Screenshots/
Extent PDF Reporter

The PDF reporter summarizes the test run results in a dashboard and other sections with feature, scenario and, step details. The PDF report needs to be enabled in the extent.properties file.

#PDF Report
extent.reporter.pdf.start=true
extent.reporter.pdf.out=PdfReport/ExtentPdf.pdf 
Ported HTML Reporter

The original HTML Extent Reporter was deprecated in 4.1.3 and removed in 5.0.0. The HTML report available in the adapter is based on the same code base and is similar in appearance. The major changes are in the Freemarker template code which has been modified to work with the Extent Reports version 5. The HTML report needs to be enabled in the extent.properties file.

#HTML Report
extent.reporter.html.start=true
extent.reporter.html.out=HtmlReport/ExtentHtml.html
Customized Report Folder Name

To enable report folder name with date and\or time details, two settings need to be added to the extent.properties. These are basefolder.name and basefolder.datetimepattern. These will be merged to create the base folder name, inside which the reports will be generated.

#FolderName
basefolder.name=ExtentReports/SparkReport_
basefolder.datetimepattern=d_MMM_YY HH_mm_ss
Attach Image as Base64 String

This feature can be used to attach images to the Spark report by setting the src attribute of the img tag to a Base64 encoded string of the image. When this feature is used, no physical file is created. There is no need to modify any step definition code to use this. To enable this, use the below settings in extent.properties, which is false by default.

extent.reporter.spark.base64imagesrc=true
Environment or System Info Properties

 It is now possible to add environment or system info properties in the extent.properties or pass them in the maven command line. 

#System Info
systeminfo.os=windows
systeminfo.version=10

As mentioned above, refer to this tutorial.

Step 1 – Add Maven dependencies to the POM

Add ExtentReport dependency

<dependency>
    <groupId>com.aventstack</groupId>
    <artifactId>extentreports</artifactId>
    <version>5.0.9</version>
</dependency>

Add tech grasshopper maven dependency for Cucumber.

<dependency>
    <groupId>tech.grasshopper</groupId>
    <artifactId>extentreports-cucumber7-adapter</artifactId>
    <version>1.7.0</version>
</dependency>

The complete POM.xml will look like as shown below with other Selenium and JUnit5 dependencies.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.example</groupId>
  <artifactId>ExtentReportsCucumber7JUnit5</artifactId>
  <version>0.0.1-SNAPSHOT</version>

 <properties>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
		<cucumber.version>7.6.0</cucumber.version>
		<extentreports.cucumber7.adapter.version>1.7.0</extentreports.cucumber7.adapter.version>
        <extentreports.version>5.0.9</extentreports.version>
		<selenium.version>4.3.0</selenium.version>
		<webdrivermanager.version>5.2.1</webdrivermanager.version>
		<junit.jupiter.version>5.9.0</junit.jupiter.version>
		<apache.common.version>2.4</apache.common.version>
		<projectlombok.version>1.18.24</projectlombok.version>
		<maven.compiler.plugin.version>3.10.1</maven.compiler.plugin.version>
		<maven.surefire.plugin.version>3.0.0-M7</maven.surefire.plugin.version>
		<maven.compiler.source.version>11</maven.compiler.source.version>
		<maven.compiler.target.version>11</maven.compiler.target.version>
	</properties>

	<dependencyManagement>
		<dependencies>
			<dependency>
				<groupId>io.cucumber</groupId>
				<artifactId>cucumber-bom</artifactId>
				<version>${cucumber.version}</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
			<dependency>
				<groupId>org.junit</groupId>
				<artifactId>junit-bom</artifactId>
				<version>${junit.jupiter.version}</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
		</dependencies>
	</dependencyManagement>

	<dependencies>

		<dependency>
			<groupId>io.cucumber</groupId>
			<artifactId>cucumber-java</artifactId>
			<scope>test</scope>
		</dependency>

		<dependency>
			<groupId>io.cucumber</groupId>
			<artifactId>cucumber-junit-platform-engine</artifactId>
			<scope>test</scope>
		</dependency>

		<!-- JUnit Platform -->
		<dependency>
			<groupId>org.junit.platform</groupId>
			<artifactId>junit-platform-suite</artifactId>
			<scope>test</scope>
		</dependency>

		<dependency>
			<groupId>org.junit.jupiter</groupId>
			<artifactId>junit-jupiter-engine</artifactId>
			<scope>test</scope>
		</dependency>

		<dependency>
			<groupId>org.junit.jupiter</groupId>
			<artifactId>junit-jupiter</artifactId>
			<scope>test</scope>
		</dependency>
		
		<!-- Cucumber ExtentReport Adapter -->
        <dependency>
            <groupId>tech.grasshopper</groupId>
            <artifactId>extentreports-cucumber7-adapter</artifactId>
            <version>${extentreports.cucumber7.adapter.version}</version>
        </dependency>
 
        <!-- Extent Report -->
        <dependency>
            <groupId>com.aventstack</groupId>
            <artifactId>extentreports</artifactId>
            <version>${extentreports.version}</version>
        </dependency>

		<!-- Selenium -->
		<dependency>
			<groupId>org.seleniumhq.selenium</groupId>
			<artifactId>selenium-java</artifactId>
			<version>${selenium.version}</version>
		</dependency>

		<!-- Web Driver Manager -->
		<dependency>
			<groupId>io.github.bonigarcia</groupId>
			<artifactId>webdrivermanager</artifactId>
			<version>${webdrivermanager.version}</version>
		</dependency>

		<!-- Apache Common -->
		<dependency>
			<groupId>org.apache.directory.studio</groupId>
			<artifactId>org.apache.commons.io</artifactId>
			<version>${apache.common.version}</version>
		</dependency>

		<dependency>
			<groupId>org.projectlombok</groupId>
			<artifactId>lombok</artifactId>
			<version>${projectlombok.version}</version>
			<scope>provided</scope>
		</dependency>

	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-compiler-plugin</artifactId>
				<version>${maven.compiler.plugin.version}</version>
				<configuration>
					<source>${maven.compiler.source.version}</source>
					<target>${maven.compiler.target.version}</target>
				</configuration>
			</plugin>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-surefire-plugin</artifactId>
				<version>${maven.surefire.plugin.version}</version>
				<configuration>
					<properties>
						<configurationParameters>
                cucumber.junit-platform.naming-strategy=long
            </configurationParameters>
					</properties>
				</configuration>
			</plugin>
		</plugins>
	</build>
</project>

Step 2 – Create extent.properties file in src/test/resources

We need to create the extent.properties file at the src/test/resources folder for the grasshopper extent report adapter to recognize it. Using a property file for reporting is quite helpful if you want to define several different properties.

#Extent Report
extent.reporter.spark.start=true
extent.reporter.spark.out=Reports/Spark.html
 
#PDF Report
extent.reporter.pdf.start=true
extent.reporter.pdf.out=PdfReport/ExtentPdf.pdf
 
#HTML Report
extent.reporter.html.start=true
extent.reporter.html.out=HtmlReport/ExtentHtml.html
 
#FolderName
basefolder.name=ExtentReports/SparkReport_
basefolder.datetimepattern=d_MMM_YY HH_mm_ss
 
#Screenshot
screenshot.dir=/Screenshots/
screenshot.rel.path=../Screenshots/
 
#Base64
extent.reporter.spark.base64imagesrc=true
 
#System Info
systeminfo.os=windows
systeminfo.version=10

Step 3 – Create a Cucumber Test Runner class in src/test/java

Add the extent report cucumber adapter to the runner class.

import static io.cucumber.junit.platform.engine.Constants.GLUE_PROPERTY_NAME;
import static io.cucumber.junit.platform.engine.Constants.PLUGIN_PROPERTY_NAME;
import org.junit.platform.suite.api.ConfigurationParameter;
import org.junit.platform.suite.api.IncludeEngines;
import org.junit.platform.suite.api.SelectClasspathResource;
import org.junit.platform.suite.api.Suite;

@Suite
@IncludeEngines("cucumber")
@SelectClasspathResource("com.example")
@ConfigurationParameter(key = PLUGIN_PROPERTY_NAME, value = "com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter:") 
@ConfigurationParameter(key = GLUE_PROPERTY_NAME, value = "com.example")
 
public class CucumberRunnerTests  {
 
}

Step 4 – Execute the code

To execute the code, run the tests from command line by using the below command

mvn clean test -Dcucumber.features="src/test/resources/features"

Step 5 – View ExtentReport

Refresh the project and will see a new folder – SparkReport_ which further contains 4 folders -HtmlReport,PdfReport, Reports, and Screenshots.

The ExtentReport will be present in the Reports folder with the name Spark.html. PDF Report is present in PdfReport folder and HTML Report is present in HtmlReport folder. We can see that the Screenshots folder is empty because we have used base64imagesrc feature that results in no physical screenshots. The screenshots are embedded in the reports.

Right-click and open the ExtentHtml.html report with Web Browser. The report also has a summary section that displays the summary of the execution. The summary includes the overview of the pass/fail using a pictogram, start time, end time, and pass/fail details of features as shown in the image below.

ExtentHtml.html

This is the image of Dashboard of the ExtentReport

The failed test has screenshot embedded in it. Double click on mase64image and it will open the screenshot in full screen.

PDF Report

Spark Report

Right-click and open the Spark.html report with Web Browser.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

Integration of Cucumber7 with Selenium and JUnit5

HOME

I have created a lot of tutorials on creating Test Frameworks by integrating JUnit4 with Selenium, Cucumber, Serenity, Rest API, Springboot. This tutorial explain the steps to Integrate Cucumber7 with JUnit5.

JUnit 5 is composed of several different modules from three different sub-projects.

JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage

We can use the JUnit Platform to execute Cucumber scenarios.

Add the cucumber-junit-platform-engine dependency to your pom.xml:

<dependency>
   <groupId>io.cucumber</groupId>
   <artifactId>cucumber-junit-platform-engine</artifactId>
   <version>${cucumber.version}</version>
   <scope>test</scope>
</dependency>

This will allow IntelliJ IDEA, Eclipse, Maven, Gradle, etc, to discover, select and execute Cucumber scenarios.

Pre-Requisite:

  1. Java Version 11 installed
  2. Eclipse or IntelliJ installed
  3. Maven or Gradle installed and setup
  4. Cucumber Eclipse Plugin installed

Project Structure

Step 1- Download and Install Java

Cucumber and Selenium need Java to be installed on the system to run the tests. Click here to know How to install Java.

Step 2 – Download and setup Eclipse IDE on the system

The Eclipse IDE (integrated development environment) provides strong support for Java developers, which is needed to write Java code. Click here to know How to install Eclipse.

Step 3 – Setup Maven

To build a test framework, we need to add a number of dependencies to the project. It is a very tedious and cumbersome process to add each dependency manually. So, to overcome this problem, we use a build management tool. Maven is a build management tool that is used to define project structure, dependencies, build, and test management. Click here to know How to install Maven.

Step 4 – Install Cucumber Eclipse Plugin (Only for Eclipse IDE)

The Cucumber Eclipse plugin is an plugin that allows the eclipse to understand the Gherkin syntax. The Cucumber Eclipse Plugin highlights the keywords present in Feature File. Click here to know more – Install Cucumber Eclipse Plugin.

Step 5 – Create a new Maven Project

Click here to know How to create a Maven project

Below is the Maven project structure. Here,

Group Id – com.example
Artifact Id – Cucumber7JUnit5Demo
Version – 0.0.1-SNAPSHOT
Package – com. example. Cucumber7JUnit5Demo

Step 6 – Add Maven dependencies to the POM

Add the dependencies to the POM.xml.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>

	<groupId>com.example</groupId>
	<artifactId>Cucumber7Junit5</artifactId>
	<version>1.0.0-SNAPSHOT</version>
	<packaging>jar</packaging>

	<properties>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
		<cucumber.version>7.6.0</cucumber.version>
		<selenium.version>4.3.0</selenium.version>
		<webdrivermanager.version>5.2.1</webdrivermanager.version>
		<junit.jupiter.version>5.9.0</junit.jupiter.version>
		<apache.common.version>2.4</apache.common.version>
		<projectlombok.version>1.18.24</projectlombok.version>
		<maven.compiler.plugin.version>3.10.1</maven.compiler.plugin.version>
		<maven.surefire.plugin.version>3.0.0-M7</maven.surefire.plugin.version>
		<maven.compiler.source.version>11</maven.compiler.source.version>
		<maven.compiler.target.version>11</maven.compiler.target.version>
	</properties>

	<dependencyManagement>
		<dependencies>
			<dependency>
				<groupId>io.cucumber</groupId>
				<artifactId>cucumber-bom</artifactId>
				<version>${cucumber.version}</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
			<dependency>
				<groupId>org.junit</groupId>
				<artifactId>junit-bom</artifactId>
				<version>${junit.jupiter.version}</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
		</dependencies>
	</dependencyManagement>

	<dependencies>

		<dependency>
			<groupId>io.cucumber</groupId>
			<artifactId>cucumber-java</artifactId>
			<scope>test</scope>
		</dependency>

		<dependency>
			<groupId>io.cucumber</groupId>
			<artifactId>cucumber-junit-platform-engine</artifactId>
			<scope>test</scope>
		</dependency>

		<!-- JUnit Platform -->
		<dependency>
			<groupId>org.junit.platform</groupId>
			<artifactId>junit-platform-suite</artifactId>
			<scope>test</scope>
		</dependency>

		<dependency>
			<groupId>org.junit.jupiter</groupId>
			<artifactId>junit-jupiter-engine</artifactId>
			<scope>test</scope>
		</dependency>

		<dependency>
			<groupId>org.junit.jupiter</groupId>
			<artifactId>junit-jupiter</artifactId>
			<scope>test</scope>
		</dependency>

		<!-- Selenium -->
		<dependency>
			<groupId>org.seleniumhq.selenium</groupId>
			<artifactId>selenium-java</artifactId>
			<version>${selenium.version}</version>
		</dependency>

		<!-- Web Driver Manager -->
		<dependency>
			<groupId>io.github.bonigarcia</groupId>
			<artifactId>webdrivermanager</artifactId>
			<version>${webdrivermanager.version}</version>
		</dependency>

		<!-- Apache Common -->
		<dependency>
			<groupId>org.apache.directory.studio</groupId>
			<artifactId>org.apache.commons.io</artifactId>
			<version>${apache.common.version}</version>
		</dependency>

		<dependency>
			<groupId>org.projectlombok</groupId>
			<artifactId>lombok</artifactId>
			<version>${projectlombok.version}</version>
			<scope>provided</scope>
		</dependency>

	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-compiler-plugin</artifactId>
				<version>${maven.compiler.plugin.version}</version>
				<configuration>
					<source>${maven.compiler.source.version}</source>
					<target>${maven.compiler.target.version}</target>
				</configuration>
			</plugin>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-surefire-plugin</artifactId>
				<version>${maven.surefire.plugin.version}</version>
				<configuration>
					<properties>
						<configurationParameters>
                cucumber.junit-platform.naming-strategy=long
            </configurationParameters>
					</properties>
				</configuration>
			</plugin>
		</plugins>
	</build>
</project>

Step 7 – Create a feature file in src/test/resources

Below is a sample feature file. I have also added a failed scenario in @FaceBookLink. Feature file should be saved as an extension of .feature. Add the test scenarios in this feature file. I have added sample test scenarios. The test scenarios are written in Gherkins language.

Feature: Login to HRM Application 

Background: 
    Given User is on HRMLogin page "https://opensource-demo.orangehrmlive.com/"
 
   @ValidCredentials
   Scenario: Login with valid credentials
     
    When User enters username as "Admin" and password as "admin123"
    Then User should be able to login sucessfully and new page open
    
   @InvalidCredentials
   Scenario Outline: Login with invalid credentials
     
    When User enters username as "<username>" and password as "<password>"
    Then User should be able to see error message "<errorMessage>"
    
  Examples:
  | username   | password  | errorMessage                      |
  | Admin      | admin12$$ | Invalid credentials               |
  | admin$$    | admin123  | Invalid credentials               |
  
    
  @FaceBookLink
  Scenario: Verify FaceBook Icon on Login Page
     
    Then User should be able to see FaceBook Icon
    
  @LinkedInLink
  Scenario: Verify LinkedIn Icon on Login Page
     
    Then User should be able to see LinkedIn Icon  
    
   @ForgetPasswordLink
   Scenario: Verify ForgetPassword link on Login Page
     
    When User clicks on Forgot your Password Link
    Then User should navigate to a new page
      

Step 8 – Create cucumber.properties file in src/test/resources

We need to create the junit-platform.properties file in the src/test/resources folder. Using a property file for reporting is quite helpful if you want to define several different properties.

cucumber.publish.enabled=true

Step 9 – Create a Helper class in src/main/java

We have used Page Object Model with Cucumber and TestNG. Create a Helper class where we are initializing the web driver, initializing the web driver wait, defining the timeouts, and creating a private constructor of the class, within it will declare the web driver, so whenever we create an object of this class, a new web browser is invoked. 

import java.time.Duration;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.support.ui.WebDriverWait;
import io.github.bonigarcia.wdm.WebDriverManager;

public class HelperClass {
	
	private static HelperClass helperClass;
	
	private static WebDriver driver;
	private static WebDriverWait wait;
    public final static int TIMEOUT = 10;
	
	 private HelperClass() {
		 
			WebDriverManager.chromedriver().setup();
	    	driver = new ChromeDriver();
	        wait = new WebDriverWait(driver, Duration.ofSeconds(TIMEOUT));
	        driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(TIMEOUT));
	        driver.manage().window().maximize();

	 }      
	    	
    public static void openPage(String url) {
        driver.get(url);
    }

	public static WebDriver getDriver() {
		return driver;	
		
	}
	
	public static void setUpDriver() {
		
		if (helperClass==null) {
			
			helperClass = new HelperClass();
		}
	}

	 public static void tearDown() {
		 
		 if(driver!=null) {
			 driver.close();
			 driver.quit();
		 }
		 
		 helperClass = null;

	 } 	
}

Step 10 – Create Locator classes in src/main/java

Create a locator class for each page that contains the detail of the locators of all the web elements. Here, I’m creating 3 locator classes – LoginPageLocators , HomePageLocators and ForgotPasswordLocators.

LoginPageLocators

import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;

public class LoginPageLocators {

	@FindBy(name = "username")
    public WebElement userName;
 
    @FindBy(name = "password")
    public WebElement password;
 
    @FindBy(id = "logInPanelHeading")
    public WebElement titleText;
 
    @FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/form/div[3]/button")
    public WebElement login;
 
    @FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/div/div[1]/div[1]/p")
    public  WebElement errorMessage;
    
    @FindBy(xpath = "//*[@href='https://www.linkedin.com/company/orangehrm/mycompany/']")
    public  WebElement linkedInIcon;
    
    @FindBy(xpath = "//*[@href='https://www.facebook.com/OrangeHRM/mycompany']") //Invalid Xpath
    public  WebElement faceBookIcon;
    
    @FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/form/div[4]/p")
    public  WebElement ForgotYourPasswordLink;
    
}

HomePageLocators

import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;

public class HomePageLocators {

	@FindBy(xpath = "//*[@id='app']/div[1]/div[2]/div[2]/div/div[1]/div[1]/div[1]/h5")
	public  WebElement homePageUserName;
 
}

ForgotPasswordLocators

import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;

public class ForgotPasswordLocators {
	
	@FindBy(xpath = "//*[@id='app']/div[1]/div[1]/div/form/h6")
    public WebElement ForgotPasswordHeading;

}

Step 11 – Create Action classes in src/main/java

Create the action classes for each web page. These action classes contain all the methods needed by the step definitions. In this case, I have created 2 action classes – LoginPageActions, HomePageActions and ForgotPasswordActions.

LoginPageActions

In this class, the very first thing will do is to create the object of the LoginPageLocators class so that we should be able to access all the PageFactory elements. Secondly, create a public constructor of LoginPageActions class.

import org.openqa.selenium.support.PageFactory;
import com.example.locators.LoginPageLocators;
import com.example.utils.HelperClass;

public class LoginPageActions {

	LoginPageLocators loginPageLocators = null; 
	
    public LoginPageActions() {

    	this.loginPageLocators = new LoginPageLocators();

		PageFactory.initElements(HelperClass.getDriver(),loginPageLocators);
	}
 
	// Set user name in textbox
    public void setUserName(String strUserName) {
    	loginPageLocators.userName.sendKeys(strUserName);
    }
 
    // Set password in password textbox
    public void setPassword(String strPassword) {
    	loginPageLocators.password.sendKeys(strPassword);
    }
 
    // Click on login button
    public void clickLogin() {
    	loginPageLocators.login.click();
    }
 
    // Get the title of Login Page
    public String getLoginTitle() {
        return loginPageLocators.titleText.getText();
    }
       
    // Get the title of Login Page
    public String getErrorMessage() {
        return loginPageLocators.errorMessage.getText();
    }
    
    // LinkedIn Icon is displayed
    public Boolean getLinkedInIcon() {
   
        return loginPageLocators.linkedInIcon.isDisplayed();
    }
    
    // FaceBook Icon is displayed
    public Boolean getFaceBookIcon() {
   
        return loginPageLocators.faceBookIcon.isDisplayed();
    }
    
    // Click on Forget Your Password link
    public void clickOnForgetYourPasswordLink() {
    	
    	loginPageLocators.ForgotYourPasswordLink.click();
    }
 
    public void login(String strUserName, String strPassword) {
 
        // Fill user name
        this.setUserName(strUserName);
 
        // Fill password
        this.setPassword(strPassword);
 
        // Click Login button
        this.clickLogin();
 
    }
}

HomePageActions

import org.openqa.selenium.support.PageFactory;
import com.example.locators.HomePageLocators;
import com.example.utils.HelperClass;

public class HomePageActions {

	HomePageLocators homePageLocators = null;
   
	public HomePageActions() {
    	
		this.homePageLocators = new HomePageLocators();

		PageFactory.initElements(HelperClass.getDriver(),homePageLocators);
    }

    // Get the User name from Home Page
    public String getHomePageText() {
        return homePageLocators.homePageUserName.getText();
    }

}

ForgotPasswordActions

import org.openqa.selenium.support.PageFactory;
import com.example.locators.ForgotPasswordLocators;
import com.example.utils.HelperClass;

public class ForgotPasswordActions {
	
	ForgotPasswordLocators forgotPasswordLocators = null;
	   
	public ForgotPasswordActions() {
    	
		this.forgotPasswordLocators = new ForgotPasswordLocators();

		PageFactory.initElements(HelperClass.getDriver(),forgotPasswordLocators);
    }

 
    // Get the Heading of Forgot Password page
    public String getForgotPasswordPageText() {
        return forgotPasswordLocators.ForgotPasswordHeading.getText();
    }
}

Step 12 – Create a Step Definition file in src/test/java

Create the corresponding Step Definition file of the feature file.

LoginPageDefinitions

import org.junit.jupiter.api.Assertions;
import com.example.actions.ForgotPasswordActions;
import com.example.actions.HomePageActions;
import com.example.actions.LoginPageActions;
import com.example.utils.HelperClass;
import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;

public class LoginPageDefinitions{	

	LoginPageActions objLogin = new LoginPageActions();
    HomePageActions objHomePage = new HomePageActions();
    ForgotPasswordActions objForgotPasswordPage = new ForgotPasswordActions();
 
    @Given("User is on HRMLogin page {string}")
    public void loginTest(String url) {
    	
    	HelperClass.openPage(url);
 
    }
 
    @When("User enters username as {string} and password as {string}")
    public void goToHomePage(String userName, String passWord) {
 
        // login to application
        objLogin.login(userName, passWord);
 
        // go the next page
        
    }
    
    @When("User clicks on Forgot your Password Link")
    public void goToForgotYourPasswordPage() {
    	
    	objLogin.clickOnForgetYourPasswordLink();
    	
    }
 
    @Then("User should be able to login sucessfully and new page open")
    public void verifyLogin() {
 
        // Verify home page
        Assertions.assertTrue(objHomePage.getHomePageText().contains("Employee Information"));
 
    }
    
    @Then("User should be able to see error message {string}")
    public void verifyErrorMessage(String expectedErrorMessage) {
 
        // Verify home page
    	Assertions.assertEquals(objLogin.getErrorMessage(),expectedErrorMessage);
 
    }
    
    @Then("User should be able to see LinkedIn Icon")
    public void verifyLinkedInIcon( ) {
    	
    	Assertions.assertTrue(objLogin.getLinkedInIcon());
    }
    
    @Then("User should be able to see FaceBook Icon")
    public void verifyFaceBookIcon( ) {
    	
    	Assertions.assertTrue(objLogin.getFaceBookIcon());
    }
    
    @Then("User should navigate to a new page")
    public void verfiyForgetYourPasswordPage() {
    	
   	Assertions.assertEquals(objForgotPasswordPage.getForgotPasswordPageText(), "Reset Password");
    }
      
}

Step 13 – Create Hook class in src/test/java

Create the hook class that contains the Before and After hook to initialize the web browser and close the web browser.

import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import com.example.utils.HelperClass;
import io.cucumber.java.After;
import io.cucumber.java.Before;
import io.cucumber.java.Scenario;

public class Hooks {
		
	@Before
    public static void setUp() {

       HelperClass.setUpDriver();
    }

	@After
	public static void tearDown(Scenario scenario) {

		//validate if scenario has failed
		if(scenario.isFailed()) {
			final byte[] screenshot = ((TakesScreenshot) HelperClass.getDriver()).getScreenshotAs(OutputType.BYTES);
			scenario.attach(screenshot, "image/png", scenario.getName()); 
		}	
		
		HelperClass.tearDown();
	}
}

Step 14 – Create a Cucumber Test Runner class in src/test/java

Cucumber needs a TestRunner class to run the feature files. It is suggested to create a folder with the name of the runner in the src/test/java directory and create the Cucumber TestRunner class in this folder. Below is the code of the Cucumber TestRunner class.

import static io.cucumber.junit.platform.engine.Constants.GLUE_PROPERTY_NAME;
import org.junit.platform.suite.api.ConfigurationParameter;
import org.junit.platform.suite.api.IncludeEngines;
import org.junit.platform.suite.api.SelectClasspathResource;
import org.junit.platform.suite.api.Suite;

@Suite
@IncludeEngines("cucumber")
@SelectClasspathResource("com.example")
@ConfigurationParameter(key = GLUE_PROPERTY_NAME, value = "com.example")
 
public class CucumberRunnerTests  {
 
}

Step 15 – Run the tests from Maven or Command Line

Use the below command to run the tests.

mvn clean test -Dcucumber.features="src/test/resources/features"

Step 16 – Cucumber Report Generation

Below is the image of the Cucumber Report generated using Cucumber Service.

Congratulations!! We have build the framework using Cucumber 7 with JUnit5.

Assumptions in JUnit5

HOME

 Assumptions is a collection of utility methods that support conditional test execution based on assumptions.

In direct contrast to failed assertions, failed assumptions do not result in a test failure; rather, a failed assumption results in a test being aborted.

Assumptions are typically used whenever it does not make sense to continue execution of a given test method — for example, if the test depends on something that does not exist in the current runtime environment.

Junit 5 comes with a subset of the assumption methods that JUnit 4 provides with Java 8 lambda expressions and method references. All JUnit Jupiter assumptions are static methods in the org.junit.jupiter.api.Assumptions class.

  1. Assumptions.assumeTrue() – If the condition is true, then run the test, else aborting the test.
  2. Assumptions.false() – If the condition is false, then run the test, else aborting the test.
  3. Assumptions.assumingThat() – is much more flexible, If condition is true then executes, else do not abort test continue rest of code in test.
1. assumeTrue

The assumeTrue() method validates the given assumption to be true and if the assumption is true – the test proceed, otherwise, test execution is aborted.

    int num1 = 4;
    int num2=6;
    int num3 = 24;
    int num4=10;

    @Test
    void assumeTrueTest() {
        System.setProperty("ENV", "TEST");
        assumeTrue("TEST".equals(System.getProperty("ENV")));

        // Since the condition is true rest of it will get executed
        assertEquals((num1*num2),num3,"The product of "+num1+"and "+num2+"is equal to "+num3);

    }

The output of the above program is

In the below example, assumeTrue() is false. So, the execution is skipped.

    int num1 = 4;
    int num2=6;
    int num3 = 24;
    int num4=10;

    @Test
    void assumeTrueTest1() {
        System.setProperty("ENV", "TEST");
        assumeTrue("QA".equals(System.getProperty("ENV")));

        // Since the condition is true rest of it will not get executed
        assertEquals((num1*num2),num3,"The product of "+num1+"and "+num2+"is equal to "+num3);

    }

The output of the above program is

2. assumeFalse()

The assumeFalse() method validates the given assumption to false and if the assumption is false – test proceed, otherwise, test execution is aborted. In the below example, the test is false and we are using assumeFalse(), so the tests will be executed.

    int num1 = 4;
    int num2=6;
    int num3 = 24;
    int num4=10;

    @Test
    void assumeFalseTest1() {
        System.setProperty("ENV", "TEST");
        assumeFalse("DEV".equals(System.getProperty("ENV")));

        // Since the condition is true rest of it will get executed
        assertEquals((num1*num2),num3,"The product of "+num1+"and "+num2+"is equal to "+num3);

    }

The output of the above program is

In the below example, the test is false and we are using assumeFalse(), so the tests will be executed.

 @Test
    void assumeFalseTest() {
        System.setProperty("ENV", "TEST");
        assumeFalse("TEST".equals(System.getProperty("ENV")));

        // Since the condition is false rest of it will not get executed
        assertEquals((num1*num2),num3,"The product of "+num1+"and "+num2+"is equal to "+num3);

    }

The output of the above program is

3. assertThat()

This method executes the supplied Executable, but only if the supplied assumption is valid.

Unlike the other assumption methods, this method will not abort the test.

If the assumption is invalid, this method does nothing.
If the assumption is valid and the executable throws an exception, it will be treated like a regular test failure. The thrown exception will be rethrown as is but masked as an unchecked exception.

    int num1 = 4;
    int num2=6;
    int num3 = 24;
    int num4=10;
    int num5=8;
    int num6=2;

   @Test
    void assumingThatTest() {
        System.setProperty("ENV", "UAT");
        assumingThat(
                "UAT".equals(System.getProperty("ENV")),
                () -> {
                    // Since the condition is true, this assertion will get executed
                    System.out.println("Assuming that executable executed");
                    assertEquals((num1+num2),num4,"The product of "+ num1 +" and "+ num2 +" is not equal to "+num4);
                });
        // Since the condition is false rest of it will get executed
        System.out.println("Loop outside");
        assertEquals((num5-num2),num6,"The difference of "+ num5 +" and "+num2+" is not equal to " + num6);
    }

The output of the above program is

In the below example, the condition is false , so we skip the execution of that condition. But, we execute the rest of the code.

    int num1 = 4;
    int num2=6;
    int num3 = 24;
    int num4=10;

    @Test
    void assumingThatTest1() {
        System.setProperty("ENV", "UAT");
        assumingThat(
                "DEV".equals(System.getProperty("ENV")),
                () -> {
                    // Since the condition is false, this assertion will not get executed
                    System.out.println("Assuming that executable executed");
                    assertEquals((num1+num2),num4,"The sum of "+num1+"and "+num2+"is not equal to "+num4);
                });
        
        System.out.println("Loop outside");
        assertEquals((num1*num2),num3,"The product of "+num1+"and "+num2+"is not equal to "+num3);
    }

The output of the above program is

Difference between Assumption and Assertion

The main difference between the assertions and assumptions is –

The assumption is use to decide whether we want to execute a section or the rest of the test method or not and if the condition is false then the test is skipped.

Whereas if a condition in an assertion fails then it fails the test and something needs to be fixed.

Difference between JUnit4 and JUnit5

HOME

In this article, we’ll see an overview of the differences between the two versions of the library.

1. Architecture

JUnit 4 has everything bundled into a single jar file whereas JUnit 5 is composed of 3 sub-projects i.e. JUnit Platform, JUnit Jupiter, and JUnit Vintage.

JUnit4

<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.13.2</version>
    <scope>test</scope>
</dependency>

JUnit Platform: It defines the TestEngine API for developing new testing frameworks that run on the platform.
JUnit Jupiter: It has all new JUnit annotations and TestEngine implementation to run tests written with these annotations.
JUnit Vintage: To support running JUnit 3 and JUnit 4 written tests on the JUnit 5 platform.

JUnit5

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-engine</artifactId>
    <version>5.9.0-M1</version>
    <scope>test</scope>
</dependency>
 
<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-api</artifactId>
    <version>5.9.0-M1</version>
    <scope>test</scope>
</dependency>

<dependency>
    <groupId>org.junit.vintage</groupId>
    <artifactId>junit-vintage-engine</artifactId>
    <version>5.9.0-M1</version>
    <scope>test</scope>
</dependency>

2. JDK Version

JUnit 4 requires Java 5 (or higher) whereas JUnit 5 requires Java 8 (or higher).

3. Imports

JUnit 5 uses the org.JUnit package for its annotations and classes whereas JUnit 5 uses the new org.JUnit.jupiter package for its annotations and classes. For example, org.JUnit.Test becomes org.JUnit.jupiter.api.Test.
@Before annotation of JUnit4 is renamed to @BeforeEach in JUnit5
@After annotation of JUnit4 is renamed to @AfterEach in JUnit5
@BeforeClass annotation of JUnit4 is renamed to @BeforeAll in JUnit5
@AfterClass annotation of JUnit4 is renamed to @AfterAll in JUnit5

4. Assertions

JUnit 5 assertions are now in org.JUnit.jupiter.api.Assertions whereas JUnit4 assertions are in org.JUnit.Assert. Most of the common assertions, like assertEquals() and assertNotNull() look the same as before, but there are a few key differences:

  • The error message is now the last argument, for example, assertEquals(“my message”, 1, 2) would be assertEquals(1, 2, “my message”)
  • Most assertions now accept a lambda that constructs the error message, which is only called when the assertion fails. Below is an example of the same.
    @Test
    void nullNegative() {
        String str = "Summer";

        Assertions.assertNull(str, () -> "The string should be null");
    }

The output of the above program is

  • assertTimeout() and assertTimeoutPreemptively() have replaced the @Timeout annotation (note that there is a @Timeout annotation in JUnit 5, but it works differently than JUnit 4).
  • There are several new assertions in JUnit5- assertAll(), assertIterableEquals(), assertLinesMatch(), assertThrows() and assertDoesNotThrow(). To know more about assertions in JUnit5, please refer to this tutorial – JUnit5 Assertions Example

5. Assumptions

In Junit 4, org.junit.Assume contains methods for stating assumptions about the conditions in which a test is meaningful. It has the following five methods:

  • assumeFalse()
  • assumeNoException()
  • assumeNotNull()
  • assumeThat()
  • assumeTrue()

JUnit5 has the following three methods:

  • assumeFalse()
  • assumingThat​()
  • assumeTrue()

Below is an example of assumeThat() annotation in JUnit5.

    @Test
    void assumingThatTest() {
        System.setProperty("ENV", "UAT");
        assumingThat(
                "UAT".equals(System.getProperty("ENV")),
                () -> {
                    // Since the condition is true, this assertion will get executed
                    System.out.println("Assuming that executable executed");
                    assertEquals((num1+num2),num4,"The product of "+ num1 +" and "+ num2 +" is not equal to "+num4);
                });
        
        System.out.println("Loop outside");
        assertEquals((num5-num2),num6,"The difference of "+ num5 +" and "+num2+" is not equal to " + num6);
    }

The output of the above program is

6. Conditional Test Execution

In JUnit4, @Ignore is used to skip the execution of a test whereas @Disabled or one of the other built-in execution conditions is used to skip the execution of the test in JUnit5. To know more about skipping the tests in JUnit5, please refer to this tutorial – How to disable tests in JUnit5 – @Disabled.

Below is an example of @Disabled in JUnit5.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.jupiter.api.*;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import static org.junit.jupiter.api.Assertions.*;
 
class DisabledTestsDemo {
 
    WebDriver driver;
 
    @BeforeEach
    public void setUp() {
         
        WebDriverManager.chromedriver().setup();
        ChromeOptions chromeOptions = new ChromeOptions();
        driver = new ChromeDriver(chromeOptions);
        driver.manage().window().fullscreen();
        driver.get("http://automationpractice.com/index.php");
 
    }
 
    @Disabled("This test is not applicable for Sprint 14")
    @Test
    void verifyPopularLink() {
 
        boolean displayed = driver.findElement(By.xpath("//*[@id='home-page-tabs']/li[1]/a")).isDisplayed();
        assertTrue(displayed);
    }
 
    @Test
    void verifyContactNumber() {
 
        String contactDetail = driver.findElement(By.xpath("//span[@class='shop-phone']/strong")).getText();
        assertEquals("0123-456-789", contactDetail);
    }
 
    @Disabled("This test is blocked till bug 1290 is fixed")
    @Test
    void verifyWomenLink() {
 
        boolean enabled = driver.findElement(By.xpath("//*[@id='block_top_menu']/ul/li[1]/a")).isEnabled();
        assertTrue(enabled);
    }
 
    @AfterEach
    public void tearDown() {
        driver.close();
    }
}

The output of the above program is

JUnit 5 provides the ExecutionCondition extension API to enable or disable a test or container (test class) conditionally. This is like using @Disabled on a test but it can define custom conditions. There are multiple built-in conditions, such as:

  • @EnabledOnOs and @DisabledOnOs: Enables a test only on specified operating systems.
  • @EnabledOnJre and @DisabledOnJre: Specifies the test should be enabled or disabled for specific versions of Java.
  • @EnabledIfSystemProperty: Enables a test based on the value of a JVM system property.
  • @EnabledIf: Uses scripted logic to enable a test if scripted conditions are met.

7. Extending JUnit

@RunWith no longer exists; superseded by @ExtendWith in JUnit5.

In JUnit 4, customizing the framework generally meant using a @RunWith annotation to specify a custom runner. Using multiple runners was problematic, and usually required chaining or using a @Rule. This has been simplified and improved in JUnit 5 using extensions.

import net.serenitybdd.core.Serenity;
import net.serenitybdd.junit5.SerenityJUnit5Extension;
import net.thucydides.core.annotations.Managed;
import net.thucydides.core.annotations.Steps;
import net.thucydides.core.annotations.Title;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.openqa.selenium.WebDriver;
import static org.assertj.core.api.Assertions.assertThat;
 
 @ExtendWith(SerenityJUnit5Extension.class)
 class ApplicationLoginJUnit5Tests {
 
        @Managed
        WebDriver driver;
 
        @Steps
        NavigateAction navigateAction;
 
        @Steps
        StepLoginPage loginPage;
 
        @Test
        @Title("Login to application with valid credentials navigates to DashBoard page")
 
         void successfulLogin() {
 
            navigateAction.toTheHomePage();
 
            // When
            loginPage.inputUserName("Admin");
            loginPage.inputPassword("admin123");
            loginPage.clickLogin();
 
            // Then
            Serenity.reportThat("Passing valid credentials navigates to DashBoard page",
                    () -> assertThat(dashboardPage.getHeading()).isEqualToIgnoringCase("DashBoard"));
        }
    }

8. Non-public Test Methods are Allowed

JUnit 5 test classes and test methods are not required to be public. We can now make them package protected.
JUnit internally uses reflection to find test classes and test methods. Reflection can discover them even if they have limited visibility, so there is no need for them to be public.

9. Repeat Tests

JUnit Jupiter provides the ability to repeat a test a specified number of times by annotating a method with @RepeatedTest and specifying the total number of repetitions desired. To know more about RepestedTest, please refer to this tutorial – How to Retry Test in JUnit5 – @RepeatedTest

Below is the example of @RepeatedTest in JUnit5.

    @RepeatedTest(3)
    void repeatedTestWithRepetitionInfo1(RepetitionInfo repetitionInfo) {
        assertEquals(3, repetitionInfo.getTotalRepetitions());
    }

The output of the above program is

10. Parameterized Tests

Test parameterization existed in JUnit 4 with built-in libraries like JUnit4Parameterized or third-party libraries like JUnitParams. In JUnit 5, parameterized tests are completely built-in and adopt some of the best features from JUnit4Parameterized and JUnitParams. To know more about the parameterized tests in JUnit5, please refer to this tutorial – How to parameterized Tests in JUnit5.

Below is an example of parameterized Test in JUnit5.

public class CSVParameterizedTest {

    @ParameterizedTest
    @CsvSource({
            "java,      4",
            "javascript,   7",
            "python,    6",
            "HTML,    4",
    })


    void test(String str, int length) {
        assertEquals(length, str.length());
    }
}

The output of the above program is

Congratulations. We have gone through the differences between JUnit4 and JUnit5. Happy Learning!!

How to run parameterized Selenium test using JUnit5

HOME

The previous tutorial has shown the various parameterized tests in JUnit5. This tutorial shows how to run a test multiple times with a different set of data. This helps to reduce the duplication of code. This is a very common scenario in any testing. Imagine, we want to test the requirement for a login page that uses a username and password to log in to the application. Username and password must satisfy some conditions like username can be only alphabets and no numeric and special characters. There could be multiple sets of data that can be used to test this requirement.

Pre-Requisite:

  1. Selenium – 3.141.59
  2. Maven
  3. Java 11
  4. JUnit Jupiter Engine – 5.8.2
  5. JUnit Jupiter API- 5.8.2

JUnit5 provides a lot of ways to parameterize a test – @ValueSource, @EnumSource, @MethodSource, @CsvSource, @CsvFileSource, and @ArgumentsSource.

Let us see an example where the test is not parameterized. In the below example, we want to verify the different error messages generated by passing incorrect values to username and password.

This is the base class – Login which contains the test method that uses a different set of test data.

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;
import org.openqa.selenium.support.PageFactory;

public class LoginPage {

    WebDriver driver ;

    @FindBy(name="txtUsername")
    WebElement username;

    @FindBy(name="txtPassword")
    WebElement password;

    @FindBy(id="btnLogin")
    WebElement loginButton;

    @FindBy(id="spanMessage")
    WebElement actualErrorMessage;


    public LoginPage(WebDriver driver) {

        this.driver = driver;

        // This initElements method will create all WebElements
        PageFactory.initElements(driver, this);
    }

    public void setUserName(String strUserName) {
        username.sendKeys(strUserName);
    }

    // Set password in password textbox
    public void setPassword(String strPassword) {
        password.sendKeys(strPassword);
    }

    // Click on login button
    public void clickLogin() {
        loginButton.click();
    }

    // Get the error message
    public String getErrorMessage() {
        return actualErrorMessage.getText();
    }

    public void login(String strUserName, String strPasword) {

        // Fill user name
        this.setUserName(strUserName);

        // Fill password
        this.setPassword(strPasword);

        // Click Login button
        this.clickLogin();
    }

}

The below example shows 4 tests using a common test with 4 different sets of data.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import static org.junit.jupiter.api.Assertions.assertEquals;

public class NonParameterizedLoginTest  {

    WebDriver driver;
    LoginPage login;

    @BeforeEach
    void setUp() {

        WebDriverManager.chromedriver().setup();
        ChromeOptions chromeOptions = new ChromeOptions();
        driver = new ChromeDriver(chromeOptions);
        driver.manage().window().maximize();
        driver.get("https://opensource-demo.orangehrmlive.com/");

    }

    @Test
    void invalidCredentials1() {

        login = new Login(driver);
        login.login("Admin","Admin");
        String actualErrorMessage = login.getErrorMessage();
        assertEquals("Invalid credentials", actualErrorMessage);

    }

    @Test
    void invalidCredentials2() {

        login = new Login(driver);
        login.login("","Admin");
        String actualErrorMessage = login.getErrorMessage();
        assertEquals("Username cannot be empty", actualErrorMessage);

    }

    @Test
    void invalidCredentials3() {

        login = new Login(driver);
        login.login("Admin","");
        String actualErrorMessage = login.getErrorMessage();
        assertEquals("Password cannot be empty", actualErrorMessage);

    }

    @Test
    void invalidCredentials4() {

        login = new Login(driver);
        login.login("","");
        String actualErrorMessage = login.getErrorMessage();
        assertEquals("Username cannot be empty", actualErrorMessage);

    }


    @AfterEach
    void tearDown() {
        if (driver != null) {
            driver.close();
        }
    }
    
}

We can see that the same method is called multiple times. This is a duplication of code. The output of the above program is

Now, we will parametrize the same test. To do so, we need to add a dependency to the POM.xml.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.example</groupId>
    <artifactId>JUnit5Demo</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>11</maven.compiler.source>
        <maven.compiler.target>11</maven.compiler.target>
    </properties>

    <dependencies>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>5.8.2</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.8.2</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>3.141.59</version>
        </dependency>

        <dependency>
            <groupId>io.github.bonigarcia</groupId>
            <artifactId>webdrivermanager</artifactId>
            <version>5.1.0</version>
        </dependency>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-params</artifactId>
            <version>5.8.2</version>
            <scope>test</scope>
        </dependency>

    </dependencies>

    <build>
        <plugins>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M5</version>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M5</version>

                <dependencies>
                    <dependency>
                        <groupId>org.junit.jupiter</groupId>
                        <artifactId>junit-jupiter-engine</artifactId>
                        <version>5.8.2</version>
                    </dependency>
                </dependencies>
            </plugin>
        </plugins>
    </build>

</project>

There are multiple ways to parameterize the test. To start with:

  1. Replace @Test annotation with @ParameterizedTest annotation provided by the JUnit5 framework.
  2. Add parameters to the loginTest() method. In this example, we will add a username and a password parameter.
  3. Add the parameters source. In this example, we will use the @CsvFileSource annotation.

In this example, will retrieve the data from CSV. This CSV file is placed under src/test/resources. Below is the example of the credentials.csv file.

To know all the different types of parameterization methods, please refer to this tutorial. This tutorial will show the 2 most common ways to parameterize tests in JUnit5.

1.@CsvSource

@CsvSource allows us to express argument lists as comma-separated values (i.e., CSV String literals). Each string provided via the value attribute in @CsvSource represents a CSV record and results in one invocation of the parameterized test. An empty, quoted value (”) results in an empty String. This can be seen in the below example.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvFileSource;
import org.junit.jupiter.params.provider.CsvSource;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import static org.junit.jupiter.api.Assertions.assertEquals;

public class ParameterizedLoginPageTest {

    WebDriver driver;

    LoginPage loginPage;

    @BeforeEach
     void setUp() {

        WebDriverManager.chromedriver().setup();
        ChromeOptions chromeOptions = new ChromeOptions();
        driver = new ChromeDriver(chromeOptions);
        driver.manage().window().maximize();
        driver.get("https://opensource-demo.orangehrmlive.com/");

    }

    @ParameterizedTest
    @CsvSource({
            "admin123,admin123,Invalid credentials",
            "'',admin123,Username cannot be empty",
            "Admin,'',Password cannot be empty",
            "'','',Username cannot be empty"
    })

    void invalidCredentials1(String username, String password, String errorMessage) {

        loginPage = new LoginPage(driver);
        loginPage.login(username,password);
        String actualErrorMessage = loginPage.getErrorMessage();
        assertEquals(errorMessage, actualErrorMessage);

    }

    @AfterEach
     void tearDown() {
        driver.close();
    }
}

The output of the above program is

2. @CsvFileSource

@CsvFileSource lets us use comma-separated value (CSV) files from the classpath or the local file system.

We can see in the below example, that we have skipped the first line from the credentials.csv file as it is the heading of the file. invalidCredentials() method got 4 different set of the test data from CSV file using parameterization. JUnit5 ignores the headers via the numLinesToSkip attribute.

In @CsvFileSource, an empty, quoted value (“”) results in an empty String in JUnit5.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvFileSource;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import static org.junit.jupiter.api.Assertions.assertEquals;

public class ParameterizedLoginPageTest {

    WebDriver driver;

    LoginPage loginPage;

    @BeforeEach
     void setUp() {

        WebDriverManager.chromedriver().setup();
        ChromeOptions chromeOptions = new ChromeOptions();
        driver = new ChromeDriver(chromeOptions);
        driver.manage().window().maximize();
        driver.get("https://opensource-demo.orangehrmlive.com/");

    }

    @ParameterizedTest
    @CsvSource({
            "admin123,admin123,Invalid credentials",
            "'',admin123,Username cannot be empty",
            "Admin,'',Password cannot be empty",
            "'','',Username cannot be empty"
    })


    @ParameterizedTest
    @CsvFileSource(files = "src/test/resources/credentials.csv", numLinesToSkip = 1)
    void invalidCredentials(String username, String password, String errorMessage) {

        loginPage = new LoginPage(driver);
        loginPage.login(username,password);
        String actualErrorMessage = loginPage.getErrorMessage();
        assertEquals(errorMessage, actualErrorMessage);

    }


    @AfterEach
     void tearDown() {
        driver.close();
    }
}
     

The result of the above program is

Congratulations!! We have seen how Selenium tests are parameterized in JUnit5. Happy Learning.

How to parameterize Tests in JUnit5

HOME

JUnit5  enables us to execute a single test method multiple times with a different sets of data. This is called Parameterization. Parameterized Tests are declared just like regular @Test methods but use the @ParameterizedTest annotation.

This article shows you how to run a test multiple times with different arguments, so-called ‘Parameterized Tests’, let’s see the following ways to provide arguments to the test:

  • @ValueSource
  • @EnumSource
  • @MethodSource
  • @CsvSource
  • @CsvFileSource
  • @ArgumentsSource

We need to add junit-jupiter-params to support parameterized tests. In the case of Maven, add the dependency to POM.xml

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-params</artifactId>
    <version>5.8.2</version>
    <scope>test</scope>
</dependency>

In case of Gradle, add the dependency as

testCompile("org.junit.jupiter:junit-jupiter-params:5.8.2")

1. @ValueSource

Let us start with a simple example. The following example demonstrates a parameterized test that uses the @ValueSource annotation to specify an integer array as the source of arguments. The following @ParameterizedTest method will be invoked three times, with the values 5,6, and 0 respectively.

@ParameterizedTest
@ValueSource(ints = {5, 6, 0})
void test_int_arrays(int b) {

    int a= 5;
    int sum = a + b;
    assertTrue(sum>8);
 }

When executing the above-parameterized test method, each invocation will be reported separately.

The output of the above program is:

One of the limitations of value sources is that they only support these types:

  • short (with the shorts attribute)
  • byte (bytes attribute)
  • int (ints attribute)
  • long (longs attribute)
  • float (floats attribute)
  • double (doubles attribute)
  • char (chars attribute)
  • java.lang.String (strings attribute)
  • java.lang.Class (classes attribute)

Also, we can only pass one argument to the test method each time.

In the below example, an array of strings is passed as the argument to the Parameterized Test.

@ParameterizedTest(name = "#{index} - Run test with args={0}")
@ValueSource(strings = {"java", "python", "javascript","php"})
void test_string_arrays(String arg) {
        assertTrue(arg.length() > 1);
}

The output of the above program is:

@NullSource

It provides a single null an argument to the annotated @ParameterizedTest method.

    @ParameterizedTest()
    @NullSource
    void nullString(String text) {
        assertTrue(text == null);
    }
    

The output of the above program is:

@EmptySource

It provides a single empty argument to the annotated @ParameterizedTest method of the following types:

  • java.lang.String
  • java.util.List
  • java.util.Set
  • java.util.Map
  • primitive arrays (e.g. int[])
  • object arrays (e.g. String[])
 @ParameterizedTest
    @EmptySource
    void testMethodEmptySource(String argument) {
        assertTrue(StringUtils.isEmpty(argument));
        assertTrue(StringUtils.isBlank(argument));
    }

The output of the above program is:

@NullAndEmptySource

We can pass empty or null values into the test via @EmptySource, @NullSource, or @NullAndEmptySource (since JUnit 5.4).

Let’s see the following example to test an isEmpty() method.

import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.EmptySource;
import org.junit.jupiter.params.provider.NullSource;
import org.junit.jupiter.params.provider.ValueSource;
import static org.junit.jupiter.api.Assertions.*;

public class ParameterizedTestDemo {

    @ParameterizedTest(name = "#{index} - isEmpty()? {0}")
    @NullSource
    @EmptySource
    @ValueSource(strings = { " ", "   ", "\t", "\n","a"})
    void nullEmptyAndBlankStrings(String text) {
        assertTrue(text == null || text.trim().isEmpty());
    }
}

The parameterized test method result in seven invocations: 1 for null, 1 for the empty string, 4 for the explicit blank strings supplied via @ValueSource, and 1 non-blank string “a” supplied via @ValueSource.

The output of the above program is:

2. @EnumSource

@EnumSource provides a convenient way to use Enum constants.

public class EnumParameterizedTest {

    enum Error {
         Request_Invalid,
         Request_Timeout,
         RequestHeader_Invalid,
         Concurrency_Failed,
         ExternalCall_Failed,
         Schema_Invalid,
         Authentication_Failed;
    }

    @ParameterizedTest
    @EnumSource(Error.class)
    void test_enum(Error error) {
       assertNotNull(error);
    }
}

The output of the above program is:

The annotation provides an optional names attribute that lets you specify which constants shall be used, like in the following example. If omitted, all constants will be used.

    @ParameterizedTest(name = "#{index} - Is Error contains {0}?")
    @EnumSource(value = Error.class, names = {"Request_Invalid", "ExternalCall_Failed", "Concurrency_Failed", "Authentication_Failed"})
    void test_enum_include(Error error) {
       assertTrue(EnumSet.allOf(Error.class).contains(error));
    }

The output of the above program is:

The @EnumSource annotation also provides an optional mode attribute that enables fine-grained control over which constants are passed to the test method. For example, you can exclude names from the enum constant pool or specify regular expressions as in the following examples.

 @ParameterizedTest
 @EnumSource(value = Error.class, mode = EnumSource.Mode.EXCLUDE, names = {"Request_Invalid", "Request_Timeout", "RequestHeader_Invalid"})
    void test_enum_exclude(Error error) {
        EnumSet<Error> excludeRequestRelatedError = EnumSet.range(Error.Concurrency_Failed, Error.Authentication_Failed);
        assertTrue(excludeRequestRelatedError.contains(error));
  }

The output of the above program is:

EnumSource.Mode.EXCLUDE – It selects all declared enum constants except those supplied via the names attribute.

EnumSource.Mode.MATCH_ALL – It selects only those enum constants whose names match any pattern supplied via the names attribute.

  @ParameterizedTest
    @EnumSource(mode = EnumSource.Mode.MATCH_ALL, names = "^.*Invalid")
    void test_match(Error error) {
        assertTrue(error.name().contains("Invalid"));
    }

The output of the above program is

3. @MethodSource

@MethodSource allows you to refer to one or more factory methods of the test class or external classes.

Factory methods within the test class must be static unless the test class is annotated with @TestInstance(Lifecycle.PER_CLASS); whereas, factory methods in external classes must always be static. In addition, such factory methods must not accept any arguments.

If you only need a single parameter, you can return a Stream of instances of the parameter type as demonstrated in the following example.

   @ParameterizedTest(name = "#{index} - Test with String : {0}")
    @MethodSource("stringProvider")
    void test_method_string(String arg) {
        assertNotNull(arg);
    }

    // this need static
    static Stream<String> stringProvider() {
        return Stream.of("java", "junit5", null);
    }

The output of the above program is

If you do not explicitly provide a factory method name via @MethodSource, JUnit Jupiter will search for a factory method that has the same name as the current @ParameterizedTest method by convention. This is demonstrated in the following example.

    @ParameterizedTest(name = "#{index} - Test with String : {0}")
    @MethodSource
    void test_no_factory_methodname(String arg) {
        assertNotNull(arg);
    }

    static Stream<String> test_no_factory_methodname() {
        return Stream.of("java", "junit5", null);
    }

The output of the above program is

Streams for primitive types (DoubleStream, IntStream, and LongStream) are also supported as demonstrated by the following example.

 @ParameterizedTest(name = "#{index} - Test with Int : {0}")
    @MethodSource("rangeProvider")
    void test_method_int(int arg) {
        assertTrue(arg < 6);
    }
    
    static IntStream rangeProvider() {
        return IntStream.range(0, 6);
    }

The output of the above program is

If a parameterized test method declares multiple parameters, you need to return a collection, stream, or array of Arguments instances or object arrays as shown below.

    @ParameterizedTest
    @MethodSource("stringIntAndListProvider")
    void testWithMultiArgMethodSource(String str, int num, List<String> list) {
        assertEquals(5, str.length());
        assertTrue(num >=1 && num <=2);
        assertEquals(2, list.size());
    }

    static Stream<Arguments> stringIntAndListProvider() {
        return Stream.of(
                arguments("apple", 1, Arrays.asList("a", "b")),
                arguments("lemon", 2, Arrays.asList("x", "y"))
        );
    }

The output of the above program is

4. @CsvSource

@CsvSource allows you to express argument lists as comma-separated values (i.e., CSV String literals). Each string provided via the value attribute in @CsvSource represents a CSV record and results in one invocation of the parameterized test.

import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import static org.junit.jupiter.api.Assertions.assertEquals;

public class CSVParameterizedTest {

    @ParameterizedTest
    @CsvSource({
            "java,      4",
            "javascript,   7",
            "python,    6",
            "HTML,    4",
    })


    void test(String str, int length) {
        assertEquals(length, str.length());
    }
}

The output of the above program is

5. @CsvFileSource

@CsvFileSource lets us use comma-separated value (CSV) files from the classpath or the local file system. Each record from a CSV file results in one invocation of the parameterized test. The first record may optionally be used to supply CSV headers.

csvdemo.csv

    @ParameterizedTest
    @CsvFileSource(resources = "/csvdemo.csv")

    void testLength(String str, int length) {
        Assertions.assertEquals(length, str.length());
    }

The output of the above program is

csv file with the heading

JUnit can ignore the headers via the numLinesToSkip attribute.

    @ParameterizedTest
    @CsvFileSource(files = "src/test/resources/csvdemo.csv",numLinesToSkip = 1)

    void testStringLength(String str, int length) {
        Assertions.assertEquals(length, str.length());
    }

The output of the above program is

If you would like the headers to be used in the display names, you can set the useHeadersInDisplayName attribute to true. The examples below demonstrate the use of useHeadersInDisplayName.

 @ParameterizedTest(name = "[{index}] {arguments}")
    @CsvFileSource(files = "src/test/resources/csvdemo.csv",useHeadersInDisplayName = true)

    void testStringLength1(String str, int length) {
        assertEquals(length, str.length());
    }

The output of the above program is

6. @ArgumentsSource

@ArgumentsSource can be used to specify a custom, reusable ArgumentsProvider. Note that an implementation of ArgumentsProvider must be declared as either a top-level class or as a static nested class.

import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.ArgumentsProvider;
import java.util.stream.Stream;

public class CustomArgumentsProvider implements ArgumentsProvider {

    @Override
    public Stream<? extends Arguments>
    provideArguments(ExtensionContext extensionContext) throws Exception {
        return Stream.of("java", "junit5", "junit4", null).map(Arguments::of);
    }
}
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ArgumentsSource;
import static org.junit.jupiter.api.Assertions.assertNotNull;

public class ArgumentsSourceTest {

    @ParameterizedTest
    @ArgumentsSource(CustomArgumentsProvider.class)
    void test_argument_custom(String arg) {
        assertNotNull(arg);
    }
}

The output of the above program is

Congratulation. We have understood parameterization in JUnit5 tests. Happy Learning!!

How to tag and filter JUnit5 tests – @Tag

HOME

This tutorial explains to run the specific tests in JUnit5 using @Tag annotation. Imagine, there are 500 test cases for different functionalities. Out of 500 test cases, 350 tests are related to Integration test and rest 150 are for E2E test. We want to run only Integration tests. How this can be achieved? To overcome this problem, JUnit5 provides a filtering mechanism – @Tag annotation. We can apply the @Tag annotation on a test class or test method or both.

The JUnit Platform enforces the following rules for Tag:

  • A tag must not be null or blank.
  • A trimmed tag must not contain whitespace.
  • A trimmed tag must not contain ISO control characters.
  • A trimmed tag must not contain any of the following reserved characters.
    • ,: comma
    • (: left parenthesis
    • ): right parenthesis
    • &: ampersand
    • |: vertical bar
    • !: exclamation point

Annotating JUnit Test Class with Tag

Scenario 1 – Apply @Tag on the test class. It will run all the tests present within this test class.

import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.assertEquals;

@Tag("Sprint-5")
class JUnit5TagsTests {

    @Test
    void test_Addition() {
        System.out.println("test_Add()");
        assertEquals(18, 3 + 7 + 8);
    }

    @Test
    void test_Subtraction() {
        System.out.println("test_Subtraction()");
        assertEquals(18, 26 - 8);
    }

    @Test
    void test_Calculator() {
        System.out.println("test_Calculator()");
        assertEquals(18, 10 + 8);
        assertEquals(2, 10 - 8);
    }

    @Test
    void test_Functions() {
        System.out.println("test_Functions()");
        assertEquals(8, Math.sqrt(64));
        assertEquals(64, Math.pow(8,2));
    }

    @Test
    void test_IsEven() {
        System.out.println("test_IsEven()");
        assertEquals(0, 16%2);
    }

}

Let us say we a number of classes and we want to execute only this specific test class that is tagged as – @Sprint-5.

Go to the command line or in case of IntelliJ to the terminal.

mvn clean test -Dgroups="Sprint-5"

The result of the above program is

Annotating JUnit Test Methods with Tag

With JUnit 5 we can filter tests by tagging a subset of them under a unique tag name.

Scenario 2 – Let’s say we have 5 tests and we want to run 3 tests in the development environment, 1 test in both development and QA, 1 test in Production and 1 test In-Progress. So we will tag the tests as below:

import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;

class JUnit5TagsTests {

    @Test
    @Tag("Development")
    void test1() {
        System.out.println("This test is for Development");
    }

    @Test
    @Tag("Development")
    void test2() {
        System.out.println("This test is for Development");
    }

    @Test
    @Tag("Development")
    @Tag("QA")
    void test3() {
        System.out.println("This test is for Development & QA");
    }

    @Test
    @Tag("Production")
    void test4() {
        System.out.println("This test is for Production");
    }

    @Test
    @Tag("Regression")
    @Tag("QA")
    void test5() {
        System.out.println("This is Regression Test for QA");
    }
}

To run the tests tagged with “production” in IntelliJ. Edit the configuration. Click on the Run and select “Edit Configurations”.

Select Tags from a list of components and mention the name of tag you want to execute. Apply the changes by clicking on the “Apply” button and then Click on the “OK” button.

Now, this create a new Configuration as shown in the below image.

Click on this configuration. It will only run the test method tagged with @Production.

2. We can apply multiple tags on a single test case as well. Here, test method – test_Calculator() is tagged with @Development and @QA.

    @Test
    @Tag("Development")
    @Tag("QA")
    void test_Calculator() {
        System.out.println("test_Calculator()");
        assertEquals(18, 10 + 8);
        assertEquals(2, 10 - 8);
    }

To run above test, edit the configuration as shown below.

The output of the above program is

3. Filtering Tags with Maven Surefire Plugin

In Maven, we can run tests based on tags via the configuration parameters of the maven-surefire-plugin.

<plugin>
       <groupId>org.apache.maven.plugins</groupId>
       <artifactId>maven-surefire-plugin</artifactId>
       <version>3.0.0-M5</version>
       <configuration>
           <!-- Include tags -->
           <groups>Development,QA,Production</groups>
           <!-- Exclude tags -->
           <excludedGroups>In-Progress</excludedGroups>
       </configuration>
</plugin>

If we now execute this plugin, it will execute all tests which are tagged as Development,QA,Production. 

If we want to exclude any specific test from the test execution, mention it with <excludeGroups>

Below mentioned command will run all the tests except test tagged with “In-Progress”.

mvn clean test -DexcludeGroups="In-Progress"

4. Creating your own custom tag annotation

If we are using same tag @Tag(“Security”) or combination with @Tag(“QA”) in several tests, instead of copying and pasting @Tag(“Security”), @Tag(“QA”) throughout your code base, you can create a custom composed annotation named @SecurityQATest as follows. @SecurityQATest can then be used instead using 2 annotations every time.

Following example shows you how to create custom tag annotation for @Tag(“Security”), @Tag(“QA”).

import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestTemplate;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.ANNOTATION_TYPE,ElementType.METHOD})
@Tag("Security")
@Tag("QA")
@Test
public @interface SecurityQATest {

}

    @SecurityQATest
    void test6() {
        System.out.println("This is Security Testing for QA");
    }

To run this test, use the below command:

mvn clean test -Dgroups="Security&QA"

The result of the above program is

Congratulations. We have understood the usage of @Tag annotation. Happy Learning!!