In the previous tutorial, I explained the Integration of the Allure Report with Selenium and TestNG. In this tutorial, I will explain how to Integrate Allure Report with Cucumber5, Selenium, and TestNG.
The below example covers the implementation of Allure Reports with Cucumber5, Selenium, TestNG, Java, and Maven. Before starting, make sure to install Allure on your machine. Refer to this tutorial to install allure – What is Allure Report?.
Prerequisite
- Java 11 installed
- Maven installed
- Eclipse or IntelliJ installed
- Allure installed
This framework consists of:
- Selenium – 3.141.59
- Java 11
- Cucumber 5 – 5.7.0
- Maven – 3.8.1
- Allure Report – 2.14.0
- Allure Cucumber5 – 2.14.0
- Aspectj – 1.9.6
Implementation Steps
- Update Properties section in Maven pom.xml
- Add Cucumber5, Selenium, TestNG, and Allure Cucumber5 dependencies in POM.xml
- Update Build Section of pom.xml in Allure Report Project.
- Create a folder – features within src/test/resources to create test scenarios in the Feature file
- Create the Step Definition class or Glue Code
- Create a TestNG Cucumber Runner class
- Create testng.xml for the project
- Run the Test and Generate Allure Report
Step 1 – Update the Properties section in Maven pom.xml
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<selenium.version>3.141.59</selenium.version>
<cucumber.version>5.7.0</cucumber.version>
<testng.version>7.4.0</testng.version>
<allure.cucumber5.version>2.14.0</allure.cucumber5.version>
<maven.compiler.plugin.version>3.5.1</maven.compiler.plugin.version>
<maven.surefire.plugin.version>3.0.0-M5</maven.surefire.plugin.version>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<aspectj.version>1.9.6</aspectj.version>
<allure.maven.version>2.10.0</allure.maven.version>
</properties>
Step 2 – Add Cucumber5, Selenium, TestNG, Allure-Cucumber5, and Allure-TestNG dependencies
Add Cucumber5, Selenium, TestNG, Allure-Cucumber5, and Allure-TestNG dependencies to pom.xml (Maven Project).
<dependencies>
<!--Cucumber Dependencies-->
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-java</artifactId>
<version>${cucumber.version}</version>
</dependency>
<!--Selenium Dependency-->
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>${selenium.version}</version>
</dependency>
<!--Cucumber TestNG Dependency-->
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-testng</artifactId>
<version>${cucumber.version}</version>
<scope>test</scope>
</dependency>
<!--Hamcrest Dependency-->
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest</artifactId>
<version>2.2</version>
<scope>test</scope>
</dependency>
<!--Allure Cucumber Dependency-->
<dependency>
<groupId>io.qameta.allure</groupId>
<artifactId>allure-cucumber5-jvm</artifactId>
<version>${allure.cucumber5.version}</version>
</dependency>
</dependencies>
Step 3 – Update the Build Section of pom.xml in the Allure Report Project
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven.compiler.plugin.version}</version>
<configuration>
<source>${maven.compiler.source}</source>
<target>${maven.compiler.target}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven.surefire.plugin.version}</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>
<argLine>
-javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
-Dcucumber.options="--plugin io.qameta.allure.cucumber5jvm.AllureCucumber5Jvm"
</argLine>
</configuration>
<dependencies>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjweaver</artifactId>
<version>${aspectj.version}</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>io.qameta.allure</groupId>
<artifactId>allure-maven</artifactId>
<version>${allure.maven.version}</version>
<configuration>
<reportVersion>${allure.version}</reportVersion>
</configuration>
</plugin>
</plugins>
</build>
Step 4 – Create a Feature file
Create a folder – features within src/test/resources to create test scenarios in the Feature file.
Feature file should be saved as an extension of .feature. Add the test scenarios in this feature file. I have added sample test scenarios. In this feature file. The test scenarios are written in Gherkins language.
@LoginPage @TestNG
@severity=blocker
Feature: Feature - Login to HRM Application
@ValidCredentials
Scenario: Scenario - Login with valid credentials
Given User is on Home page
When User enters username as "Admin"
And User enters password as "admin123"
Then User should be able to login sucessfully
@InvalidCredentials
Scenario Outline: Scenario -Login with invalid credentials
Given User is on Home page
When User enters username as "<username>"
And User enters password as "<password>"
Then Error message "<message>" should be displayed
Examples:
|username |password |message |
|admin |admin |Invalid credentials |
| |admin123 |Username can be empty |
|Admin | |Password can be empty |
| | |Username cannot be empty |
Step 5 – Create the Step Definition class or Glue Code
Below is the code for the CommonDefinitions.
package com.example.CucumberAllureReportDemo.definitions;
import java.util.concurrent.TimeUnit;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import io.cucumber.java.After;
import io.cucumber.java.Before;
import io.cucumber.java.Scenario;
public class CommonDefinitions {
protected static WebDriver driver;
@Before
public void setup() {
System.setProperty("webdriver.gecko.driver",
"C:\\Users\\Vibha\\Software\\geckodriver-v0.26.0-win64\\geckodriver.exe");
driver = new FirefoxDriver();
driver.manage().window().maximize();
driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
}
@After
public void tearDown(Scenario scenario) {
try {
String screenshotName = scenario.getName().replaceAll("", "_");
if (scenario.isFailed()) {
TakesScreenshot ts = (TakesScreenshot) driver;
byte[] screenshot = ts.getScreenshotAs(OutputType.BYTES);
scenario.attach(screenshot, "img/png", screenshotName);
}
} catch (Exception e) {
e.printStackTrace();
}
driver.quit();
}
}
LoginPageDefinition
public class LoginDefinition {
@Given("User is on Home page")
public void userOnHomePage() {
CommonDefinitions.driver.get("https://opensource-demo.orangehrmlive.com/");
}
@When("User enters username as {string}")
public void entersUsername(String userName) throws InterruptedException {
System.out.println("Username Entered");
CommonDefinitions.driver.findElement(By.name("txtUsername")).sendKeys(userName);
}
@When("User enters password as {string}")
public void entersPassword(String passWord) throws InterruptedException {
System.out.println("Password Entered");
CommonDefinitions.driver.findElement(By.name("txtPassword")).sendKeys(passWord);
CommonDefinitions.driver.findElement(By.id("btnLogin")).submit();
}
@Then("User should be able to login sucessfully")
public void sucessfulLogin() throws InterruptedException {
String newPageText = CommonDefinitions.driver.findElement(By.id("welcome")).getText();
System.out.println("newPageText :" + newPageText);
assertThat(newPageText, containsString("Welcome"));
}
@Then("Error message {string} should be displayed")
public void unsucessfulLogin(String message) throws InterruptedException {
String errorMessage = CommonDefinitions.driver.findElement(By.id("spanMessage")).getText();
System.out.println("Error Message :" + errorMessage);
Assert.assertEquals(errorMessage, message);
}
}
Step 6 – Create a TestNG Cucumber Runner class
We need to create a class called Runner class to run the tests. This class will use the JUnit annotation @RunWith(), which tells JUnit what is the test runner class.
import org.testng.annotations.Test;
import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;
@Test
@CucumberOptions(plugin = { "pretty }, tags = "", features = "src/test/resources/features/Login.feature", glue = "com.example.CucumberAllureReportDemo.definitions")
public class CucumberRunnerTests extends AbstractTestNGCucumberTests {
}
Note:- @Test annotation marks this class as part of the test. So, if we will remove this annotation, the Allure Report executes CucumberRunnerTests as a separate test suite, so there will be duplicate results.
Step 7 – Create testng.xml for the project
<?xml version = "1.0"encoding = "UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name = "Suite1">
<test name = "Test Demo">
<classes>
<class name = "com.example.CucumberAllureReportDemo.runner.CucumberRunnerTests"/>
</classes>
</test>
</suite>
Step 8 – Run the Test and Generate Allure Report
To run the tests, use the below command
mvn clean test
In the below image, we can see that two tests failed and three passed out of five tests.
This will create the allure-results folder with all the test reports within target folder. These files will be used to generate Allure Report.
Use the below command to generate the Allure Report
allure serve
This will generate the beautiful Allure Test Report as shown below.
Allure Report Dashboard
Categories in Allure Report
The categories tab gives you a way to create custom defect classifications to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).
Suites in Allure Report
On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found. Here, we have 2 suits – Feature and Surefire test. Surefire tests are executed from CucumberRunnerTests.
Graphs in Allure Report
Graphs allow you to see different statistics collected from the test data: status breakdown or severity and duration diagrams.
Timeline in Allure Report
The timeline tab visualizes retrospective test execution, allure adaptors collect precise timings of tests, and here on this tab, they are arranged accordingly to their sequential or parallel timing structure.
Behaviours of Allure Report
This tab groups test results according to Epic, Feature, and Story tags.
Screenshot attached to the failed test case
Packages in Allure Report
The packages tab represents a tree-like layout of test results, grouped by different packages.
When we don’t use @Test in CucumberRunnerTests.java, then as mentioned above the Allure report will have duplicate details as shown below:-
Congratulations!! We have integrated an allure report with Cucumber5, Selenium, and TestNG. I hope this tutorial is useful to you.
Additional Tutorials on Allure Reports
Integration of Allure Report with Selenium and JUnit4
Integration of Allure Report with Selenium and TestNG
Gradle – Allure Report for Selenium and JUnit4
Gradle – Allure Report for Cucumber, Selenium and TestNG
Integration of Allure Report with Rest Assured and JUnit4
QA Automation Expert 
how to add a new tab in allure report?
LikeLike