ExtentReports Version 4 for Cucumber 5, Selenium, and TestNG

HOME

The previous tutorial explained the generation of Allure Report with Cucumber5, Selenium and TestNG in a Maven project. In this tutorial, I will explain the steps to create an Extent Report4 with Cucumber5, Selenium, and TestNG in a Maven project.

Pre-Requisite:

  1. Java 8 or higher is needed for ExtentReport5
  2. Maven
  3. JAVA IDE (like Eclipse, IntelliJ, or so on)
  4. TestNG installed
  5. Cucumber Eclipse plugin (in case using Eclipse)

Project Structure

Step 1 – Add Maven dependencies to the POM

Add ExtentReport dependency.

<!-- Extent Report -->
<dependency>
	<groupId>com.aventstack</groupId>
	<artifactId>extentreports</artifactId>
	<version>${extentreports.version}</version>
</dependency>

Add tech grasshopper maven dependency for Cucumber. The below version of extentreports-cucumber5-adapter dependency needs to be added to the POM, to work with ExtentReports version 4.

<!-- Cucumber ExtentReport Adapter -->
<dependency>
	<groupId>tech.grasshopper</groupId>
	<artifactId>extentreports-cucumber5-adapter</artifactId>
	<version>1.51</version>
</dependency>

If you want to use ExtentReport Version5, then use version – 2.13.0.

The complete POM.xml will look as shown below with other Selenium and TestNG dependencies.

<properties>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
		<cucumber.version>5.7.0</cucumber.version>
		<extentreports.cucumber5.adapter.version>1.5.1</extentreports.cucumber5.adapter.version>
		<extentreports.version>4.1.7</extentreports.version>
		<selenium.version>3.141.59</selenium.version>
		<webdrivermanager.version>5.2.1</webdrivermanager.version>
		<testng.version>6.14.3</testng.version>
		<apache.common.version>2.4</apache.common.version>		
		<maven.compiler.plugin.version>3.7.0</maven.compiler.plugin.version>
		<maven.surefire.plugin.version>3.0.0-M5</maven.surefire.plugin.version>
		<maven.compiler.source.version>11</maven.compiler.source.version>
		<maven.compiler.target.version>11</maven.compiler.target.version>
	</properties>

	<dependencies>

		<dependency>
			<groupId>io.cucumber</groupId>
			<artifactId>cucumber-java</artifactId>
			<version>${cucumber.version}</version>
		</dependency>

		<dependency>
           <groupId>io.cucumber</groupId>
           <artifactId>cucumber-testng</artifactId>
           <version>${cucumber.version}</version>
           <scope>test</scope>
       </dependency>

		<!-- Cucumber ExtentReport Adapter -->
		<dependency>
			<groupId>tech.grasshopper</groupId>
			<artifactId>extentreports-cucumber5-adapter</artifactId>
			<version>${extentreports.cucumber5.adapter.version}</version>
		</dependency>

		<!-- Extent Report -->
		<dependency>
			<groupId>com.aventstack</groupId>
			<artifactId>extentreports</artifactId>
			<version>${extentreports.version}</version>
		</dependency>
		
		<!-- Selenium -->
		<dependency>
			<groupId>org.seleniumhq.selenium</groupId>
			<artifactId>selenium-java</artifactId>
			<version>${selenium.version}</version>
		</dependency>

		<!-- Web Driver Manager -->
		<dependency>
			<groupId>io.github.bonigarcia</groupId>
			<artifactId>webdrivermanager</artifactId>
			<version>${webdrivermanager.version}</version>
		</dependency>

		<!-- TestNG -->
		<dependency>
			<groupId>org.testng</groupId>
			<artifactId>testng</artifactId>
			<version>${testng.version}</version>
			<scope>test</scope>
		</dependency>

		<!-- Apache Common -->
		<dependency>
			<groupId>org.apache.directory.studio</groupId>
			<artifactId>org.apache.commons.io</artifactId>
			<version>${apache.common.version}</version>
		</dependency>

	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-compiler-plugin</artifactId>
				<version>${maven.compiler.plugin.version}</version>
				<configuration>
					<source>${maven.compiler.source.version}</source> 
					<target>${maven.compiler.target.version}</target> 
				</configuration>
			</plugin>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-surefire-plugin</artifactId>
				<version>${maven.surefire.plugin.version}</version>
				<configuration>
					<suiteXmlFiles>
						<suiteXmlFile>testng.xml</suiteXmlFile>
					</suiteXmlFiles>
				</configuration>
			</plugin>
		</plugins>
	</build>
</project>

Step 2: Create a feature file in src/test/resources

Below is a sample feature file. I have added 2 failed scenarios – @FaceBookLink(Invalid XPath) and @MissingUsername (Incorrect Verification).

Feature: Login to HRM Application 

Background: 
    Given User is on HRMLogin page "https://opensource-demo.orangehrmlive.com/"
  
   @ValidCredentials
   Scenario: Login with valid credentials
     
    When User enters username as "Admin" and password as "admin123"
    Then User should be able to login sucessfully and new page opens
    
   @InvalidCredentials
   Scenario Outline: Login with invalid credentials
     
    When User enters username as "<username>" and password as "<password>"
    Then User should be able to see error message "<errorMessage>"
    
  Examples:
   | username   | password  | errorMessage                      |
   | Admin      | admin12$$ | Invalid credentials               |
   | admin$$    | admin123  | Invalid credentials               |
   | admin123   | Admin     | Invalid credentials               |
   | $$$$$$$    | &&&&&&&&  | Invalid credentials               |
      
  @FaceBookLink
  Scenario: Verfy FaceBook Icon on Login Page
     
    Then User should be able to see FaceBook Icon
    
  @MissingUsername
  Scenario: Verify error message when username is missing
     
    When User enters username as "" and password as "admin123"
    Then User should be able to see error message for empty username as "Empty Username"
   

Step 3: Create extent.properties file in src/test/resources

We need to create the extent.properties file in the src/test/resources folder for the grasshopper extent report adapter to recognize it. Using a property file for reporting is quite helpful if you want to define several different properties.

Let’s enable spark report in an extent properties file:

extent.reporter.avent.start=false
extent.reporter.bdd.start=false
extent.reporter.cards.start=false
extent.reporter.email.start=false
extent.reporter.html.start=true
extent.reporter.klov.start=false
extent.reporter.logger.start=true
extent.reporter.tabular.start=false

extent.reporter.avent.config=
extent.reporter.bdd.config=
extent.reporter.cards.config=
extent.reporter.email.config=
extent.reporter.html.config=
extent.reporter.klov.config=
extent.reporter.logger.config=
extent.reporter.tabular.config=

extent.reporter.avent.out=Reports/AventReport/
extent.reporter.bdd.out=Reports/BddReport/
extent.reporter.cards.out=Reports/CardsReport/
extent.reporter.email.out=Reports/EmailReport/ExtentEmail.html
extent.reporter.html.out=Reports/HtmlReport/ExtentHtml.html
extent.reporter.logger.out=Reports/LoggerReport/
extent.reporter.tabular.out=Reports/TabularReport/

#Screenshot
screenshot.dir=Reports/Screenshots/
screenshot.rel.path=../Screenshots/

Step 4: Create a Helper class in src/main/java

We have used Page Object Model with Cucumber and TestNG.

Create a Helper class where we are initializing the web driver, initializing the web driver wait, defining the timeouts, and creating a private constructor of the class, it will declare the web driver, so whenever we create an object of this class, a new web browser is invoked. We are using a setter and getter method to get the object of Chromedriver with the help of a private constructor itself within the same class.

HelperClass

import java.util.concurrent.TimeUnit;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import io.github.bonigarcia.wdm.WebDriverManager;

public class HelperClass {
	
	private static HelperClass helperClass;
	
	private static WebDriver driver;
    public final static int TIMEOUT = 10;
	
	 private HelperClass() {
		 
			WebDriverManager.chromedriver().setup();
	    	driver = new ChromeDriver();
	        driver.manage().timeouts().implicitlyWait(TIMEOUT,TimeUnit.SECONDS);
	        driver.manage().window().maximize();

	 }      
	    	
    public static void openPage(String url) {
        driver.get(url);
    }
	
	public static WebDriver getDriver() {
		return driver;			
	}
	
	public static void setUpDriver() {
		
		if (helperClass==null) {		
			helperClass = new HelperClass();
		}
	}
	
	 public static void tearDown() {
		 
		 if(driver!=null) {
			 driver.close();
			 driver.quit();
		 }
		 
		 helperClass = null;

	 } 	
}

Step 5: Create Locator classes in src/main/java

Create a locator class for each page that contains the detail of the locators of all the web elements. Here, I’m creating 2 locator classes – LoginPageLocators and HomePageLocators.

LoginPageLocators

import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;

public class LoginPageLocators {

	@FindBy(name = "txtUsername")
    public WebElement userName;
 
    @FindBy(name = "txtPassword")
    public WebElement password;
 
    @FindBy(id = "logInPanelHeading")
    public WebElement titleText;
 
    @FindBy(id = "btnLogin")
    public WebElement login;
 
    @FindBy(id = "spanMessage")
    public  WebElement errorMessage;
    
    @FindBy(xpath = "//*[@id='social-icons']/a[1]/img")
    public  WebElement linkedInIcon;
    
    @FindBy(xpath = "//*[@id='social-icons']/a[6]/img")  //Invalid Xpath
    public  WebElement faceBookIcon;
     
}

HomePageLocators

import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;

public class HomePageLocators {

	@FindBy(xpath = "//*[@id='app']/div[1]/div[2]/div[2]/div/div[1]/div[1]/div[1]/h5")
	public  WebElement homePageUserName;
 
}

Step 6: Create Action classes in src/main/java

Create the action classes for each web page. These action classes contain all the methods needed by the step definitions. In this case, I have created 2 action classes – LoginPageActions and HomePageActions

LoginPageActions

In this class, the very first thing will do is to create the object of the LoginPageLocators class so that we should be able to access all the PageFactory elements. Secondly, create a public constructor of LoginPageActions class.

package com.example.testng.actions;

import org.openqa.selenium.support.PageFactory;
import com.example.testng.locators.LoginPageLocators;
import com.example.testng.utils.HelperClass;

public class LoginPageActions {

LoginPageLocators loginPageLocators = null; 
	
    public LoginPageActions() {

    	this.loginPageLocators = new LoginPageLocators();

		PageFactory.initElements(HelperClass.getDriver(),loginPageLocators);
	}
    
    public void login(String strUserName, String strPassword) {
    	 
        // Fill user name
    	loginPageLocators.userName.sendKeys(strUserName);
 
        // Fill password
    	loginPageLocators.password.sendKeys(strPassword);
 
        // Click Login button
    	loginPageLocators.login.click(); 
    }
 
    //Get the title of Login Page")
    public String getLoginTitle() {
        return loginPageLocators.titleText.getText();
    }
     
    // Get the error message of Login Page
    public String getErrorMessage() {
        return loginPageLocators.errorMessage.getText();
    }
    
    // FaceBook Icon is displayed
    public Boolean getFaceBookIcon() {
   
        return loginPageLocators.faceBookIcon.isDisplayed();
    }
    
    // Get the error message when username is blank
    public String getMissingUsernameText() {
         return loginPageLocators.missingUsernameErrorMessage.getText();
     }
}

HomePageActions

import org.openqa.selenium.support.PageFactory;
import com.example.testng.locators.HomePageLocators;
import com.example.testng.utils.HelperClass;

public class HomePageActions {

	HomePageLocators homePageLocators = null;
   
	public HomePageActions() {
    	
		this.homePageLocators = new HomePageLocators();

		PageFactory.initElements(HelperClass.getDriver(),homePageLocators);
    }
 
    // Get the User name from Home Page
    public String getHomePageText() {
        return homePageLocators.homePageUserName.getText();
    }

}

Step 7: Create a Step Definition file in src/test/java

Create the corresponding Step Definition file of the feature file.

LoginPageDefinitions

import org.testng.Assert;
import com.example.testng.actions.HomePageActions;
import com.example.testng.actions.LoginPageActions;
import com.example.testng.utils.HelperClass;
import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;

public class LoginPageDefinitions{	

	LoginPageActions objLogin = new LoginPageActions();
    HomePageActions objHomePage = new HomePageActions();
 
    @Given("User is on HRMLogin page {string}")
    public void loginTest(String url) {
    	
    	HelperClass.openPage(url);
 
    }
 
    @When("User enters username as {string} and password as {string}")
    public void goToHomePage(String userName, String passWord) {
 
        // login to application
        objLogin.login(userName, passWord);
 
        // go the next page
        
    }
    
    @Then("User should be able to login sucessfully and new page opens")
    public void verifyLogin() {
 
        // Verify home page
       Assert.assertTrue(objHomePage.getHomePageText().contains("Employee Information"));
 
    }
    
    @Then("User should be able to see error message {string}")
    public void verifyErrorMessage(String expectedErrorMessage) {
 
        // Verify home page
    	Assert.assertEquals(objLogin.getErrorMessage(),expectedErrorMessage);
 
    }
     
    @Then("User should be able to see FaceBook Icon")
    public void verifyFaceBookIcon( ) {
    	
    	Assert.assertTrue(objLogin.getFaceBookIcon());
    }     

    @Then("User should be able to see error message for empty username as {string}")
    public void verifyErrorMessageForEmptyUsername(String expectedErrorMessage) {
    	 
    	Assert.assertEquals(objLogin.getMissingUsernameText(),expectedErrorMessage);
 
    }      
}

Step 8: Create Hook class in src/test/java

Create the hook class that contains the Before and After hooks. @Before hook contains the method to call the setup driver which will initialize the chrome driver. This will be run before any test.

After Hook – Here will call the tearDown method.

import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import com.example.testng.utils.HelperClass;
import io.cucumber.java.After;
import io.cucumber.java.Before;
import io.cucumber.java.Scenario;

public class Hooks {
	
	@Before
    public static void setUp() {

       HelperClass.setUpDriver();
    }
	
	@After
	public static void tearDown(Scenario scenario) {
		//validate if scenario has failed
		if(scenario.isFailed()) {
			final byte[] screenshot = ((TakesScreenshot) HelperClass.getDriver()).getScreenshotAs(OutputType.BYTES);
			scenario.attach(screenshot, "image/png", scenario.getName()); 
		}	
		
		HelperClass.tearDown();
	}
}

Step 9: Create a Cucumber Test Runner class in src/test/java

Add the extent report cucumber adapter to the runner class’s CucumberOption annotation. 

 plugin = {"com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter:"}

This is how your runner class should look after being added to our project. Moreover, be sure to keep the colon “:” at the end.

import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;
 
@CucumberOptions(tags = "", features = "src/test/resources/features/LoginPage.feature", glue = "com.example.testng.definitions",
                 plugin = {"com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter:"})
 
public class CucumberRunnerTests extends AbstractTestNGCucumberTests {
 
}

Step 10: Create the testng.xml for the project

Right-click on the project and select TestNG -> convert to TestNG.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test name="ExtentReport4 for Cucumber">
  
  <classes>
  <class name = "com.example.testng.runner.CucumberRunnerTests"/>
  </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

Step 11: Execute the code

Right Click on the Runner class and select Run As -> TestNG Test.

Below is the screenshot of the Console. As expected, 5 tests, out of 7 are passed and 2 failed.

Step 12: View ExtentReport

Refresh the project and will see a new folder – Report. The ExtentReport will be present in that folder with the name Spark.html.

Right-click and select Open with Web Browser.

The report also has a summary section that displays the summary of the execution. The summary includes the overview of the pass/fail using a pictogram, start time, end time, and pass/fail details of features as shown in the image below.

Click on the Dashboard icon present on the left side of the report. To view the details about the steps, click on the scenarios. Clicking on the scenario will expand, showing off the details of the steps of each scenario.

The icon present at the end of the failed scenario is highlighted, click on that icon. It is the screenshot of the failed test.

Logger Report

This is the Dashboard Report.

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

Advertisement

Allure Report with Cucumber5, Selenium and TestNG

HOME

In the previous tutorial, I explained the Integration of the Allure Report with Selenium and TestNG. In this tutorial, I will explain how to Integrate Allure Report with Cucumber5, Selenium, and TestNG.

The below example covers the implementation of Allure Reports with Cucumber5, Selenium, TestNG, Java, and Maven. Before starting, make sure to install Allure on your machine. Refer to this tutorial to install allure – What is Allure Report?.

Prerequisite

  1. Java 11 installed
  2. Maven installed
  3. Eclipse or IntelliJ installed
  4. Allure installed

This framework consists of:

  1. Selenium – 3.141.59
  2. Java 11
  3. Cucumber 5 – 5.7.0
  4. Maven – 3.8.1
  5. Allure Report – 2.14.0
  6. Allure Cucumber5 – 2.14.0
  7. Aspectj – 1.9.6

Implementation Steps

  1. Update Properties section in Maven pom.xml
  2. Add Cucumber5, Selenium, TestNG, and Allure Cucumber5 dependencies in POM.xml
  3. Update Build Section of pom.xml in Allure Report Project.
  4. Create a folder – features within src/test/resources to create test scenarios in the Feature file
  5. Create the Step Definition class or Glue Code
  6. Create a TestNG Cucumber Runner class
  7. Create testng.xml for the project
  8. Run the Test and Generate Allure Report

Step 1 – Update the Properties section in Maven pom.xml

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <selenium.version>3.141.59</selenium.version>
    <cucumber.version>5.7.0</cucumber.version>
    <testng.version>7.4.0</testng.version>
    <allure.cucumber5.version>2.14.0</allure.cucumber5.version>
    <maven.compiler.plugin.version>3.5.1</maven.compiler.plugin.version>
    <maven.surefire.plugin.version>3.0.0-M5</maven.surefire.plugin.version>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <aspectj.version>1.9.6</aspectj.version>
    <allure.maven.version>2.10.0</allure.maven.version>
  </properties>

Step 2 – Add Cucumber5, Selenium, TestNG, Allure-Cucumber5, and Allure-TestNG dependencies

Add Cucumber5, Selenium, TestNG, Allure-Cucumber5, and Allure-TestNG dependencies to pom.xml (Maven Project).

<dependencies>
  <!--Cucumber Dependencies-->  
    <dependency>
       <groupId>io.cucumber</groupId>
       <artifactId>cucumber-java</artifactId>
       <version>${cucumber.version}</version>
    </dependency>
 
  <!--Selenium Dependency-->
    <dependency>
       <groupId>org.seleniumhq.selenium</groupId>
       <artifactId>selenium-java</artifactId>
       <version>${selenium.version}</version>
    </dependency>

  <!--Cucumber TestNG Dependency-->
    <dependency>
       <groupId>io.cucumber</groupId>
       <artifactId>cucumber-testng</artifactId>
       <version>${cucumber.version}</version>
        <scope>test</scope>
    </dependency>

  <!--Hamcrest Dependency-->
    <dependency>
       <groupId>org.hamcrest</groupId>
       <artifactId>hamcrest</artifactId>
       <version>2.2</version>
       <scope>test</scope>
    </dependency>
    
  <!--Allure Cucumber Dependency-->   
      <dependency>
         <groupId>io.qameta.allure</groupId>
         <artifactId>allure-cucumber5-jvm</artifactId>
         <version>${allure.cucumber5.version}</version>
    </dependency>  
</dependencies>

Step 3 – Update the Build Section of pom.xml in the Allure Report Project

<build>
    <plugins>   
      <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${maven.compiler.plugin.version}</version>
                <configuration>
                    <source>${maven.compiler.source}</source> 
                    <target>${maven.compiler.target}</target> 
                </configuration>
            </plugin>
     
     <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
             <version>${maven.surefire.plugin.version}</version>
            <configuration>
                 <suiteXmlFiles>
                        <suiteXmlFile>TestNG.xml</suiteXmlFile>
                    </suiteXmlFiles>
                <argLine>
                    -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                    -Dcucumber.options="--plugin io.qameta.allure.cucumber5jvm.AllureCucumber5Jvm"
                </argLine>
                        
            </configuration>
            <dependencies>
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjweaver</artifactId>
                    <version>${aspectj.version}</version>
                </dependency>
            </dependencies>
        </plugin>
        
         <plugin>
                <groupId>io.qameta.allure</groupId>
                <artifactId>allure-maven</artifactId>
                <version>${allure.maven.version}</version>
                <configuration>
                    <reportVersion>${allure.version}</reportVersion>
                </configuration>
            </plugin>
      </plugins>      
   </build>

Step 4 – Create a Feature file

Create a folder – features within src/test/resources to create test scenarios in the Feature file.

Feature file should be saved as an extension of .feature. Add the test scenarios in this feature file. I have added sample test scenarios. In this feature file. The test scenarios are written in Gherkins language.

@LoginPage @TestNG
@severity=blocker
Feature: Feature - Login to HRM Application 
 
   @ValidCredentials
   Scenario: Scenario - Login with valid credentials 
     
    Given User is on Home page
    When User enters username as "Admin"
    And User enters password as "admin123"
    Then User should be able to login sucessfully
    
   @InvalidCredentials
   Scenario Outline: Scenario -Login with invalid credentials
   
    Given User is on Home page
    When User enters username as "<username>"
    And User enters password as "<password>"
    Then Error message "<message>" should be displayed
    
    Examples:
    |username  |password  |message                         |
    |admin     |admin     |Invalid credentials             |
    |          |admin123  |Username can be empty           | 
    |Admin     |          |Password can be empty           |
    |          |          |Username cannot be empty        |

Step 5 – Create the Step Definition class or Glue Code

Below is the code for the CommonDefinitions.

package com.example.CucumberAllureReportDemo.definitions;
import java.util.concurrent.TimeUnit;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import io.cucumber.java.After;
import io.cucumber.java.Before;
import io.cucumber.java.Scenario;
public class CommonDefinitions {
	protected static WebDriver driver;
	
    @Before
	public void setup() {
		System.setProperty("webdriver.gecko.driver",
				"C:\\Users\\Vibha\\Software\\geckodriver-v0.26.0-win64\\geckodriver.exe");
		driver = new FirefoxDriver();
		driver.manage().window().maximize();
		driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
	}
	
    @After
	public void tearDown(Scenario scenario) {
		try {
			String screenshotName = scenario.getName().replaceAll("", "_");
			if (scenario.isFailed()) {
				TakesScreenshot ts = (TakesScreenshot) driver;
				byte[] screenshot = ts.getScreenshotAs(OutputType.BYTES);
				scenario.attach(screenshot, "img/png", screenshotName);
			}
		} catch (Exception e) {
			e.printStackTrace();
		}
		driver.quit();
	}
}

LoginPageDefinition

public class LoginDefinition {

	@Given("User is on Home page")
	public void userOnHomePage() {
		CommonDefinitions.driver.get("https://opensource-demo.orangehrmlive.com/");
	}

	@When("User enters username as {string}")
	public void entersUsername(String userName) throws InterruptedException {
		System.out.println("Username Entered");
		CommonDefinitions.driver.findElement(By.name("txtUsername")).sendKeys(userName);
	}

	@When("User enters password as {string}")
	public void entersPassword(String passWord) throws InterruptedException {
		System.out.println("Password Entered");
		CommonDefinitions.driver.findElement(By.name("txtPassword")).sendKeys(passWord);
		CommonDefinitions.driver.findElement(By.id("btnLogin")).submit();
	}

	@Then("User should be able to login sucessfully")
	public void sucessfulLogin() throws InterruptedException {
		String newPageText = CommonDefinitions.driver.findElement(By.id("welcome")).getText();
		System.out.println("newPageText :" + newPageText);
		assertThat(newPageText, containsString("Welcome"));
	}

	@Then("Error message {string} should be displayed")
	public void unsucessfulLogin(String message) throws InterruptedException {
		String errorMessage = CommonDefinitions.driver.findElement(By.id("spanMessage")).getText();
		System.out.println("Error Message :" + errorMessage);
		Assert.assertEquals(errorMessage, message);
	}
}

Step 6 – Create a TestNG Cucumber Runner class

We need to create a class called Runner class to run the tests. This class will use the JUnit annotation @RunWith(), which tells JUnit what is the test runner class.

import org.testng.annotations.Test;
import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;
@Test
@CucumberOptions(plugin = { "pretty }, tags = "", features = "src/test/resources/features/Login.feature", glue = "com.example.CucumberAllureReportDemo.definitions")
public class CucumberRunnerTests extends AbstractTestNGCucumberTests {
}

Note:- @Test annotation marks this class as part of the test. So, if we will remove this annotation, the Allure Report executes CucumberRunnerTests as a separate test suite, so there will be duplicate results.

Step 7 – Create testng.xml for the project

<?xml version = "1.0"encoding = "UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name = "Suite1">
  <test name = "Test Demo">
    <classes>
          <class name = "com.example.CucumberAllureReportDemo.runner.CucumberRunnerTests"/>
     </classes>  
   </test>
</suite>

Step 8 – Run the Test and Generate Allure Report

To run the tests, use the below command

mvn clean test

In the below image, we can see that two tests failed and three passed out of five tests.

This will create the allure-results folder with all the test reports within target folder. These files will be used to generate Allure Report.

Use the below command to generate the Allure Report

allure serve

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

Categories in Allure Report

The categories tab gives you a way to create custom defect classifications to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found. Here, we have 2 suits – Feature and Surefire test. Surefire tests are executed from CucumberRunnerTests.

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: status breakdown or severity and duration diagrams.

Timeline in Allure Report

The timeline tab visualizes retrospective test execution, allure adaptors collect precise timings of tests, and here on this tab, they are arranged accordingly to their sequential or parallel timing structure.

Behaviors of Allure Report

This tab groups test results according to Epic, Feature, and Story tags.

Screenshot attached to the failed test case

Packages in Allure Report

The packages tab represents a tree-like layout of test results, grouped by different packages.

When we don’t use @Test in CucumberRunnerTests.java, then as mentioned above the Allure report will have duplicate details as shown below:-

Congratulations!! We have integrated an allure report with Cucumber5, Selenium, and TestNG. I hope this tutorial is useful to you.

Additional Tutorials on Allure Reports

Integration of Allure Report with Selenium and JUnit4
Integration of Allure Report with Selenium and TestNG
Gradle – Allure Report for Selenium and JUnit4
Gradle – Allure Report for Cucumber, Selenium and TestNG
Integration of Allure Report with Rest Assured and JUnit4

Allure Report with Cucumber5, Selenium and JUnit4

HOME

In the previous tutorial, I have explained the Integration of the Allure Report with Selenium and JUnit4. In this tutorial, I will explain how to Integrate Allure Report with Cucumber5 and JUnit4.

Below example covers the implementation of Allure Reports in Selenium using JUnit4, Java and Maven.

Pre-Requisite

  1. Java 11 installed
  2. Maven installed
  3. Eclipse or IntelliJ installed

This framework consists of:

  1. Selenium – 3.141.59
  2. Java 11
  3. Cucumber 5 – 5.7.0
  4. Maven – 3.8.1
  5. Allure Report – 2.14.0
  6. Allure Cucumber5 – 2.14.0
  7. Allure JUnit4 – 2.14.0
  8. Aspectj – 1.9.6

Implementation Steps

  1. Update Properties section in Maven pom.xml
  2. Add Cucumber5, Selenium, JUnit4, Allure Cucumber5 and Allure-JUnit4 dependencies in POM.xml
  3. Update Build Section of pom.xml in Allure Report Project.
  4. Create source folder – src/test/resources and features folder within src/test/resources to create test scenarios in Feature file
  5. Create the Step Definition class or Glue Code
  6. Create a Cucumber Runner class
  7. Run the Test and Generate Allure Report

Step 1 – Update Properties section in Maven pom.xml

 <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <selenium.version>3.141.59</selenium.version> 
    <cucumber.version>5.7.0</cucumber.version>
    <allure.cucumber5.version>2.14.0</allure.cucumber5.version>
    <allure.junit4.version>2.14.0</allure.junit4.version>
    <maven.compiler.plugin.version>3.5.1</maven.compiler.plugin.version>
    <maven.surefire.plugin.version>3.0.0-M5</maven.surefire.plugin.version>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <aspectj.version>1.9.6</aspectj.version>
    <allure.version>2.14</allure.version>
  </properties>

Step 2 – Add Cucumber5, Selenium, JUnit4, Allure-Cucumber5 and Allure-JUnit4 dependencies in POM.xml

 <dependencies>
     
     <!--Cucumber Dependencies-->     
    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-java</artifactId>
      <version>${cucumber.version}</version>
    </dependency>
    
    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-junit</artifactId>
      <version>${cucumber.version}</version>
       <scope>test</scope>
    </dependency>
 
   <!--Selenium Dependency-->
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>${selenium.version}</version>
    </dependency>   

   <!--Hamcrest Dependency-->
    <dependency>
      <groupId>org.hamcrest</groupId>
      <artifactId>hamcrest</artifactId>
      <version>2.2</version>
      <scope>test</scope>
    </dependency>
    
   <!--Allure Cucumber Dependency-->     
      <dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-cucumber5-jvm</artifactId>
        <version>${allure.cucumber5.version}</version>
    </dependency>
    
     <!--Allure Reporting Dependency-->     
    <dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-junit4</artifactId>
        <version>${allure.junit4.version}</version>
        <scope>test</scope>
    </dependency>

  </dependencies>

Step 3 – Update Build Section of pom.xml in Allure Report Project

<build>
   <plugins>
  
       <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${maven.compiler.plugin.version}</version>
                <configuration>
                    <source>${maven.compiler.source}</source> 
                    <target>${maven.compiler.target}</target>
                </configuration>
         </plugin>
     
       <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
             <version>${maven.surefire.plugin.version}</version>
            <configuration>
                 <systemPropertyVariables>
                        <allure.results.directory>${project.build.directory}/allure-results</allure.results.directory>
                    </systemPropertyVariables>
                <argLine>
                    -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                    -Dcucumber.options="--plugin io.qameta.allure.cucumber5jvm.AllureCucumber5Jvm"
                </argLine>
                        
            </configuration>
            <dependencies>
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjweaver</artifactId>
                    <version>${aspectj.version}</version>
                </dependency>
            </dependencies>
         </plugin>
       
     </plugins> 
  </build>

Step 4 – Create source folder – src/test/resources and features folder within src/test/resources to create test scenarios in Feature file

Feature file should be saved as an extension of .feature. Add the test scenarios in this feature file. I have added sample test scenarios. In this feature file, I have created a scenario for successful login and one Scenario Outline for failed login. The test scenarios are written in Gherkins language.

@LoginPage @Junit4

@severity=blocker

Feature: Feature - Login to HRM Application 
 
   @ValidCredentials
   Scenario: Scenario - Login with valid credentials 
     
    Given User is on Home page
    When User enters username as "Admin"
    And User enters password as "admin123"
    Then User should be able to login sucessfully
    
   @InvalidCredentials
   Scenario Outline: Scenario -Login with invalid credentials
   
    Given User is on Home page
    When User enters username as "<username>"
    And User enters password as "<password>"
    Then Error message "<message>" should be displayed
    
    Examples:
    |username  |password  |message                         |
    |admin     |admin     |Invalid credentials             |
    |          |admin123  |Username cannot be empty        | 
    |Admin     |          |Password cannot be empty        |
    |          |          |Username can be empty           |

Step 5 – Create the Step Definition class or Glue Code

public class LoginDefinition {

	@Given("User is on Home page")
	public void userOnHomePage() {

		CommonDefinitions.driver.get("https://opensource-demo.orangehrmlive.com/");
	}

	@When("User enters username as {string}")
	public void entersUsername(String userName) throws InterruptedException {

		System.out.println("Username Entered");
		CommonDefinitions.driver.findElement(By.name("txtUsername")).sendKeys(userName);

	}

	@When("User enters password as {string}")
	public void entersPassword(String passWord) throws InterruptedException {

		System.out.println("Password Entered");
		CommonDefinitions.driver.findElement(By.name("txtPassword")).sendKeys(passWord);

		CommonDefinitions.driver.findElement(By.id("btnLogin")).submit();
	}

	@Then("User should be able to login sucessfully")
	public void sucessfulLogin() throws InterruptedException {

		String newPageText = CommonDefinitions.driver.findElement(By.id("welcome")).getText();
		System.out.println("newPageText :" + newPageText);
		assertThat(newPageText, containsString("Welcome"));

	}

	@Then("Error message {string} should be displayed")
	public void unsucessfulLogin(String message) throws InterruptedException {

		String errorMessage = CommonDefinitions.driver.findElement(By.id("spanMessage")).getText();
		System.out.println("Error Message :" + errorMessage);
		Assert.assertEquals(errorMessage, message);

	}

}

CommonDefinitions.java

public class CommonDefinitions {

	protected static WebDriver driver;

	@Before
	public void setup() {
		System.setProperty("webdriver.gecko.driver",
				"C:\\Users\\Vibha\\Software\\geckodriver-v0.26.0-win64\\geckodriver.exe");

		driver = new FirefoxDriver();

		driver.manage().window().maximize();
		driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
	}

	@After
	public void tearDown(Scenario scenario) {
		try {
			String screenshotName = scenario.getName().replaceAll("", "_");
			if (scenario.isFailed()) {
				TakesScreenshot ts = (TakesScreenshot) driver;
				byte[] screenshot = ts.getScreenshotAs(OutputType.BYTES);
				scenario.attach(screenshot, "img/png", screenshotName);
			}
		} catch (Exception e) {
			e.printStackTrace();
		}
		driver.quit();
	}

}

Step 6 – Create a Cucumber Runner class

We need to create a class called Runner class to run the tests. This class will use the JUnit annotation @RunWith(), which tells JUnit what is the test runner class.

import org.junit.runner.RunWith;

import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;

@RunWith(Cucumber.class)
@CucumberOptions(features = {
		"src/test/resources/features/Login.feature" }, glue = "com.example.CucumberJunitAllureReportDemo.definitions", plugin = {
				"pretty", "html:test-output", "json:target/cucumber-report/cucumber.json" })

public class CucumberRunnerTests {

}

Step 7 – Run the Test and Generate Allure Report

To run the tests, use the below command

mvn clean test

In the below image, we can see that one test is failed and four passed out of five tests.

This will create allure-results folder with all the test report. These files will be use to generate Allure Report.

Change current directory to target directory and then the below comand to generate the Allure Report

allure serve

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

It shows detail of all the test steps and the screenshot of the failed test step also as shown below.

Categories in Allure Report

Categories tab gives you the way to create custom defects classification to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: statuses breakdown or severity and duration diagrams.

Timeline in Allure Report

Timeline tab visualizes retrospective of tests execution, allure adaptors collect precise timings of tests, and here on this tab they are arranged accordingly to their sequential or parallel timing structure.

Behaviors of Allure Report

This tab groups test results according to Epic, Feature and Story tags.

Packages in Allure Report

Packages tab represents a tree-like layout of test results, grouped by different packages.