vibssingh

How to create Cucumber Report after rerun of failed tests

Last Updated On

HOME

There are multiple reports produced when the Cucumber failed tests are redone. There is no unified test execution status provided in these reports. To generate a consolidated Cucumber report after rerunning any failed tests, follow these steps:

Table of Contents

  1. Set Up a Runner for First Run of Tests
  2. Set Up a Runner for Failed Tests
  3. Mention both Test Runner details in the testng.xml
  4. Add cucumber-reporting Plugin to pom.xml
  5. Run the tests and Generate the Report

Set Up a Runner for First Run of Tests

In the main test runner, configure Cucumber to generate a JSON report for the initial test run and capture any failed tests in a rerun file. We are using rerun plugin to logs the paths of failed scenarios in a text file – rerun.txt that will be created inside the target folder.

import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;

@CucumberOptions(features = "src/test/resources/features/LoginPage.feature",
        glue = "com.example.definitions",
        plugin = {
                "pretty",
                "html:target/cucumber-reports/cucumber-report.html",
                "json:target/cucumber-reports/cucumber-report.json",
                "rerun:target/rerun.txt" // Saves paths of failed scenarios
        }
)
public class RunnerTests extends AbstractTestNGCucumberTests {
}

Set Up a Runner for Failed Tests

Create a second runner that reads from the rerun.txt file and generates a separate JSON report for failed test reruns.

import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;

@CucumberOptions(tags = "",
        features = "@target/rerun.txt",
        glue = "com.example.definitions",
        plugin =  {
                "pretty",
                "html:target/cucumber-reports/cucumber-rerun-report.html",
                "json:target/cucumber-reports/cucumber-rerun-report.json"
        }
)

public class RunnerTestsFailed extends AbstractTestNGCucumberTests {
}

Mention both Test Runner details in the testng.xml

We need to mention both the TestRunner class name in the testng.xml. This will run both the runners sequentially.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
    <test  name="Cucumber with TestNG Test">
        <classes>
            <class name="com.example.runner.RunnerTests"/>
            <class name="com.example.runner.RunnerTestsFailed"/>
        </classes>
    </test> <!-- Test -->
</suite> <!-- Suite -->

Add cucumber-reporting Plugin to pom.xml

Use the cucumber-reporting plugin by Masterthought to combine the JSON files into a single report. Configure the plugin in your pom.xml to read both the main and rerun JSON files.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.example</groupId>
  <artifactId>POM_Cucumber_TestNG_Demo</artifactId>
  <version>1.0-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>POM_Cucumber_TestNG_Demo</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <cucumber.version>7.18.1</cucumber.version>
    <selenium.version>4.25.0</selenium.version>
    <testng.version>7.10.2</testng.version>
    <apache.common.version>2.4</apache.common.version>
    <maven.compiler.plugin.version>3.13.0</maven.compiler.plugin.version>
    <maven.surefire.plugin.version>3.3.1</maven.surefire.plugin.version>
    <maven.compiler.source.version>17</maven.compiler.source.version>
    <maven.compiler.target.version>17</maven.compiler.target.version>
    <maven.cucumber.reporting.version>5.8.2</maven.cucumber.reporting.version>
  </properties>

  <dependencies>

    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-java</artifactId>
      <version>${cucumber.version}</version>
    </dependency>

    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-testng</artifactId>
      <version>${cucumber.version}</version>
      <scope>test</scope>
    </dependency>

    <!-- Selenium -->
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>${selenium.version}</version>
    </dependency>

    <!-- TestNG -->
    <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>${testng.version}</version>
      <scope>test</scope>
    </dependency>

    <!-- Cucumber Reporting -->
    <dependency>
      <groupId>net.masterthought</groupId>
      <artifactId>cucumber-reporting</artifactId>
      <version>${maven.cucumber.reporting.version}</version>
    </dependency>

  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>${maven.compiler.plugin.version}</version>
        <configuration>
          <source>${maven.compiler.source.version}</source>
          <target>${maven.compiler.target.version}</target>
        </configuration>
      </plugin>
      
        <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>${maven.surefire.plugin.version}</version>
        <configuration>
          <testFailureIgnore>true</testFailureIgnore>
          <suiteXmlFiles>
            <suiteXmlFile>testng.xml</suiteXmlFile>
          </suiteXmlFiles>
        </configuration>
      </plugin>

      <plugin>
        <groupId>net.masterthought</groupId>
        <artifactId>maven-cucumber-reporting</artifactId>
        <version>${maven.cucumber.reporting.version}</version>

        <executions>
          <execution>
            <id>generate-cucumber-report</id>
            <phase>verify</phase>
            <goals>
              <goal>generate</goal>
            </goals>
            <configuration>
              <projectName>Cucumber Reporting Example</projectName>
              <outputDirectory>target/cucumber-html-reports</outputDirectory>
              <inputDirectory>target</inputDirectory>
              <jsonFiles>
                <!-- Include both main and rerun JSON reports -->
                <param>target/cucumber-reports/cucumber-report.json</param>
                <param>target/cucumber-reports/cucumber-rerun-report.json</param>
                <param>**/*.json</param>
              </jsonFiles>
            </configuration>
          </execution>
        </executions>
      </plugin>

    </plugins>
  </build>
</project>

Run the tests and Generate the Report

Use the below command to run the tests and generate the consolidated Test Report.

mvn clean verify

RunnerTests class generates cucumber-report.json file whereas RunnerTestsFailed generates cucumber-rerun-report.json files are present in target/cucumber-reports folder.

MasterThought plugin uses these json files to generate a consolidated report in target/cucumber-html-reports.

There are different types of HTML reports gets generated as a part of the test execution cycle.

1. feature-overview – This HTML report gives an overall overview of test execution. Main HTML report which covers all different sections like Features, Tags, Steps, and Failures.

2. failures-overview – This HTML report gives an overview of all failed tests.

3. step-overview – This HTML report shows step statistics for the current cycle.

4. tag-overview – This HTML report shows passing and failing statistics for different tags used in test execution.

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

Posted in CucumberTagged , , , , , , , Leave a comment

Leave a comment