Allure Report with Cucumber, Selenium and TestNG

Last Updated On

June 24, 2024

In the previous tutorial, I explained the Integration of the Allure Report with Selenium and TestNG. In this tutorial, I will explain how to Integrate Allure Report with Cucumber, Selenium, and TestNG.

The below example covers the implementation of Allure Reports with Cucumber, Selenium, TestNG, Java, and Maven. Before starting, make sure to install Allure on your machine. Refer to this tutorial to install allure – What is Allure Report?.

Project Structure

Table of Contents

Prerequisite

Java 17 installed
Maven installed
Eclipse or IntelliJ installed
Allure installed

Dependency List

Selenium – 4.16.1
Java 17
Cucumber – 7.15.0
Maven – 3.9.6
Allure Report – 2.25.0
Allure Maven – 2.12.0
Aspectj – 1.9.21
Maven Compiler Plugin – 3.12.1
Maven Surefire Plugin – 3.2.3

Implementation Steps

Step 1 – Update the Properties section in Maven pom.xml

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <cucumber.version>7.15.0</cucumber.version>
    <selenium.version>4.16.1</selenium.version>
    <testng.version>7.9.0</testng.version>
    <maven.compiler.plugin.version>3.12.1</maven.compiler.plugin.version>
    <maven.surefire.plugin.version>3.2.3</maven.surefire.plugin.version>
    <maven.compiler.source.version>17</maven.compiler.source.version>
    <maven.compiler.target.version>17</maven.compiler.target.version>
    <allure.junit4.version>2.25.0</allure.junit4.version>
    <aspectj.version>1.9.21</aspectj.version>
    <allure.version>2.25.0</allure.version>
    <allure.maven>2.12.0</allure.maven>
</properties>

Step 2 – Add dependencies to pom.xml

Add Cucumber, Selenium, TestNG, Allure-Cucumber, and Allure-TestNG dependencies to pom.xml (Maven Project).

<dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-bom</artifactId>
        <version>${allure.version}</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>


  <dependencies>

    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-java</artifactId>
      <version>${cucumber.version}</version>
    </dependency>

    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-testng</artifactId>
      <version>${cucumber.version}</version>
      <scope>test</scope>
    </dependency>

    <!-- Selenium -->
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>${selenium.version}</version>
    </dependency>

    <!-- TestNG -->
    <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>${testng.version}</version>
      <scope>test</scope>
    </dependency>

    <!--Allure Cucumber Dependency-->
    <dependency>
      <groupId>io.qameta.allure</groupId>
      <artifactId>allure-cucumber7-jvm</artifactId>
      <scope>test</scope>
    </dependency>

    <!--Allure Reporting Dependency-->
    <dependency>
      <groupId>io.qameta.allure</groupId>
      <artifactId>allure-testng</artifactId>
      <scope>test</scope>
    </dependency>

  </dependencies>

Step 3 – Update the Build Section of pom.xml in the Allure Report Project

<build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>${maven.compiler.plugin.version}</version>
        <configuration>
          <source>${maven.compiler.source.version}</source>
          <target>${maven.compiler.target.version}</target>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>${maven.surefire.plugin.version}</version>
        <configuration>
          <suiteXmlFiles>
            <suiteXmlFile>testng.xml</suiteXmlFile>
          </suiteXmlFiles>
          <argLine>
            -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
          </argLine>
        </configuration>
        <dependencies>
          <dependency>
            <groupId>org.aspectj</groupId>
            <artifactId>aspectjweaver</artifactId>
            <version>${aspectj.version}</version>
            <scope>runtime</scope>
          </dependency>
        </dependencies>
      </plugin>
      <plugin>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-maven</artifactId>
        <version>${allure.maven}</version>
        <configuration>
          <reportVersion>${allure.maven}</reportVersion>
        </configuration>
      </plugin>
    </plugins>
  </build>

The complete pom.xml is

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.example</groupId>
  <artifactId>AllureReport_Cucumber_TestNG</artifactId>
  <version>1.0-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>AllureReport_Cucumber_TestNG</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <cucumber.version>7.15.0</cucumber.version>
    <selenium.version>4.16.1</selenium.version>
    <testng.version>7.9.0</testng.version>
    <maven.compiler.plugin.version>3.12.1</maven.compiler.plugin.version>
    <maven.surefire.plugin.version>3.2.3</maven.surefire.plugin.version>
    <maven.compiler.source.version>17</maven.compiler.source.version>
    <maven.compiler.target.version>17</maven.compiler.target.version>
    <allure.junit4.version>2.25.0</allure.junit4.version>
    <aspectj.version>1.9.21</aspectj.version>
    <allure.version>2.25.0</allure.version>
    <allure.maven>2.12.0</allure.maven>
  </properties>

  <!-- Add allure-bom to dependency management to ensure correct versions of all the dependencies are used -->
  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-bom</artifactId>
        <version>${allure.version}</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

  <dependencies>

    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-java</artifactId>
      <version>${cucumber.version}</version>
    </dependency>

    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-testng</artifactId>
      <version>${cucumber.version}</version>
      <scope>test</scope>
    </dependency>

    <!-- Selenium -->
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>${selenium.version}</version>
    </dependency>

    <!-- TestNG -->
    <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>${testng.version}</version>
      <scope>test</scope>
    </dependency>

    <!--Allure Cucumber Dependency-->
    <dependency>
      <groupId>io.qameta.allure</groupId>
      <artifactId>allure-cucumber7-jvm</artifactId>
      <scope>test</scope>
    </dependency>

    <!--Allure Reporting Dependency-->
    <dependency>
      <groupId>io.qameta.allure</groupId>
      <artifactId>allure-testng</artifactId>
      <scope>test</scope>
    </dependency>

  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>${maven.compiler.plugin.version}</version>
        <configuration>
          <source>${maven.compiler.source.version}</source>
          <target>${maven.compiler.target.version}</target>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>${maven.surefire.plugin.version}</version>
        <configuration>
          <suiteXmlFiles>
            <suiteXmlFile>testng.xml</suiteXmlFile>
          </suiteXmlFiles>
          <argLine>
            -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
          </argLine>
        </configuration>
        <dependencies>
          <dependency>
            <groupId>org.aspectj</groupId>
            <artifactId>aspectjweaver</artifactId>
            <version>${aspectj.version}</version>
            <scope>runtime</scope>
          </dependency>
        </dependencies>
      </plugin>
      <plugin>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-maven</artifactId>
        <version>${allure.maven}</version>
        <configuration>
          <reportVersion>${allure.maven}</reportVersion>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

Step 4 – Create a Feature file

Create a folder – features within src/test/resources to create test scenarios in the Feature file.

Feature file should be saved as an extension of .feature. Add the test scenarios in this feature file. I have added sample test scenarios. In this feature file. The test scenarios are written in Gherkins language.

Feature: Login to HRM Application

  Background:
    Given User is on HRMLogin page "https://opensource-demo.orangehrmlive.com/"

  @ValidCredentials
  Scenario: Login with valid credentials

    When User enters username as "Admin" and password as "admin123"
    Then User should be able to login successfully and new page open

  @InvalidCredentials
  Scenario Outline: Login with invalid credentials

    When User enters username as "<username>" and password as "<password>"
    Then User should be able to see error message "<errorMessage>"

    Examples:
      | username   | password  | errorMessage                      |
      | Admin      | admin12$$ | Invalid credentials               |
      | admin$$    | admin123  | Invalid credentials               |
      | abc123     | xyz$$     | Invalid credentials               |
      | 234        | xyz$$     | Invalid credentials!              |

Step 5 – Create the Step Definition class or Glue Code

The stepdefinition class is created in src/test/java directory.

Below is the code for the Hooks.

package com.example.definitions;

import io.cucumber.java.After;
import io.cucumber.java.Before;
import io.cucumber.java.Scenario;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;

import java.time.Duration;

public class Hooks {
    protected static WebDriver driver;
    public final static int TIMEOUT = 5;

    @Before
    public void setUp() {

        ChromeOptions options = new ChromeOptions();
        options.addArguments("--start-maximized");
        driver = new ChromeDriver(options);
        driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(TIMEOUT));

    }

    @After
    public void tearDown(Scenario scenario) {
        try {
            String screenshotName = scenario.getName();
            if (scenario.isFailed()) {
                TakesScreenshot ts = (TakesScreenshot) driver;
                byte[] screenshot = ts.getScreenshotAs(OutputType.BYTES);
                scenario.attach(screenshot, "img/png", screenshotName);
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
        driver.quit();
    }

}

LoginPageDefinition

package com.example.definitions;


import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;

import org.openqa.selenium.By;
import org.testng.Assert;

public class LoginPageDefinitions {

   Hooks hooks;


    @Given("User is on HRMLogin page {string}")
    public void loginTest(String url) {

        hooks.driver.get(url);

    }

    @When("User enters username as {string} and password as {string}")
    public void goToHomePage(String userName, String passWord) {

        // login to application
        hooks.driver.findElement(By.name("username")).sendKeys(userName);
        hooks.driver.findElement(By.name("password")).sendKeys(passWord);
        hooks.driver.findElement(By.xpath("//*[@class='oxd-form']/div[3]/button")).submit();

        // go the next page
    }

    @Then("User should be able to login successfully and new page open")
    public void verifyLogin() {

        String homePageHeading = hooks.driver.findElement(By.xpath("//*[@class='oxd-topbar-header-breadcrumb']/h6")).getText();

        //Verify new page - HomePage
        Assert.assertEquals(homePageHeading,"Dashboard");

    }

    @Then("User should be able to see error message {string}")
    public void verifyErrorMessage(String expectedErrorMessage) {

        String actualErrorMessage = hooks.driver.findElement(By.xpath("//*[@class='orangehrm-login-error']/div[1]/div[1]/p")).getText();

        // Verify Error Message
        Assert.assertEquals(actualErrorMessage, expectedErrorMessage);

    }

}

Step 6 – Create a TestNG Cucumber Runner class

We need to create a class called Runner class to run the tests. This class will use the TestNG annotation @Test, which tells TestNG what is the test runner class.

package com.example.runner;

import org.testng.annotations.Test;
import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;


@Test
@CucumberOptions(tags = "", features = {"src/test/resources/features"}, glue = {"com.example.definitions"},
        plugin = {"pretty","io.qameta.allure.cucumber7jvm.AllureCucumber7Jvm"})

public class CucumberRunnerTests extends AbstractTestNGCucumberTests{

}

Note:- @Test annotation marks this class as part of the test. So, if we will remove this annotation, the Allure Report executes CucumberRunnerTests as a separate test suite, so there will be duplicate results.

Step 7 – Create testng.xml for the project

<?xml version = "1.0"encoding = "UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name = "Suite1">
    <test name = "Test Demo">
        <classes>
            <class name = "com.example.runner.CucumberRunnerTests"/>
        </classes>
    </test>
</suite>

Step 8 – Run the Test and Generate Allure Report

To run the tests, use the below command

mvn clean test

In the below image, we can see that one test failed and four passed out of five tests.

This will create the allure-results folder with all the test reports within target folder. These files will be used to generate Allure Report.

Use the below command to generate the Allure Report

allure serve

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

Categories in Allure Report

The categories tab gives you a way to create custom defect classifications to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found. Here, we have 2 suits – Feature and Surefire test. Surefire tests are executed from CucumberRunnerTests.

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: status breakdown or severity and duration diagrams.

Timeline in Allure Report

The timeline tab visualizes retrospective test execution, allure adaptors collect precise timings of tests, and here on this tab, they are arranged accordingly to their sequential or parallel timing structure.

Behaviours of Allure Report

This tab groups test results according to Epic, Feature, and Story tags.

Screenshot attached to the failed test case

Packages in Allure Report

The packages tab represents a tree-like layout of test results, grouped by different packages.

When we don’t use @Test in CucumberRunnerTests.java, then as mentioned above the Allure report will have duplicate details.

Congratulations!! We have integrated an allure report with Cucumber, Selenium, and TestNG. I hope this tutorial is useful to you.

Additional Tutorials on Allure Reports

Integration of Allure Report with Selenium and JUnit4
Integration of Allure Report with Selenium and TestNG
Gradle – Allure Report for Selenium and JUnit4
Gradle – Allure Report for Cucumber, Selenium and TestNG
Integration of Allure Report with Rest Assured and JUnit4

QA Automation Expert

Automation solutions to build Test Framework

Allure Report with Cucumber, Selenium and TestNG

Project Structure

Prerequisite

Dependency List

Implementation Steps

Step 1 – Update the Properties section in Maven pom.xml

Step 2 – Add dependencies to pom.xml

Step 3 – Update the Build Section of pom.xml in the Allure Report Project

Step 4 – Create a Feature file

Step 5 – Create the Step Definition class or Glue Code

Step 6 – Create a TestNG Cucumber Runner class

Step 7 – Create testng.xml for the project

Step 8 – Run the Test and Generate Allure Report

Allure Report Dashboard

Categories in Allure Report

Suites in Allure Report

Graphs in Allure Report

Timeline in Allure Report

Behaviours of Allure Report

Packages in Allure Report

One thought on “Allure Report with Cucumber, Selenium and TestNG”

Leave a comment Cancel reply

Project Structure

Prerequisite

Dependency List

Implementation Steps

Step 1 – Update the Properties section in Maven pom.xml

Step 2 – Add dependencies to pom.xml

Step 3 – Update the Build Section of pom.xml in the Allure Report Project

Step 4 – Create a Feature file

Step 5 – Create the Step Definition class or Glue Code

Step 6 – Create a TestNG Cucumber Runner class

Step 7 – Create testng.xml for the project

Step 8 – Run the Test and Generate Allure Report

Allure Report Dashboard

Categories in Allure Report

Suites in Allure Report

Graphs in Allure Report

Timeline in Allure Report

Behaviours of Allure Report

Packages in Allure Report

Share this:

One thought on “Allure Report with Cucumber, Selenium and TestNG”

Leave a comment Cancel reply