Playwright is integrated with Java, Cucumber, and Junit5. This integration creates a robust and scalable test automation framework for modern web applications. It is a BDD-driven framework.

Table of Contents

What is Playwright?

Page Object Model (POM) is a design pattern used in test automation. It enhances test maintenance and reduces code duplication. In this pattern, web pages are represented as classes. The various elements on the page are defined as variables in the class. All possible user interactions can then be implemented as methods in the class.

What is Cucumber?

Cucumber is Behavior Driven Development (BDD). It is a methodology that integrates non-technical stakeholders, testers, and developers into the same conversation. This is achieved by using clearly defined specifications.

In behavior-driven development with Cucumber, test cases are written in Gherkin syntax. They use a Given-When-Then structure. This structure closely mirrors user stories and business rules.

Benefits of Using Cucumber with Playwright

1. Readable, business-friendly tests: Cucumber’s Gherkin syntax (Given–When–Then) makes test scenarios easy to understand for non-technical stakeholders, bridging the gap between QA, developers, and business teams.

2. Better Test Maintainability: With standardized step definitions and modularized test architecture, scaling up your test suite is effortless. Altering the suite won’t harm the entire framework.

3. Clear reporting and traceability: Cucumber’s reports map test results directly to feature files. They map directly to scenarios. This makes it easier to track coverage and failures against requirements.

4. Cross-browser and cross-platform coverage: When combined with Playwright, Cucumber scenarios can be executed across Chromium, Firefox, and WebKit without changing feature files

5. Scales well in CI/CD pipelines: The combination supports parallel execution. It offers tagging and selective test runs. This makes it suitable for continuous testing in CI environments.

What is JUnit5?

JUnit5 is a test execution engine. It discovers and runs Cucumber tests. It enhances integration testing with advanced annotations, nested tests, and parameterized testing. It simplifies test execution in Maven, Gradle, and Spring Boot with better assertions, Java dependency injection, and parallel execution.

System requirements

The following prerequisites are required to be installed on the machine to begin with a smooth setup and installation.

Java 11 or higher
IntelliJ IDE or any other IDE to create a project
Maven
Browsers on which tests need to be run, like Chrome, Firefox, etc.
Cucumber is installed

High-Level Architecture

Setting Up Cucumber BDD Framework with Playwright and JUnit5

1. Create a new Maven Project

The first step in setup is to create a new Maven project. I will be using IntelliJ in this tutorial. The following steps need to be followed to create a new Maven project :

Open IntelliJ, Navigate to File >> New >> Project

2. In the New Project window, enter the following details:

Name of the Project – Cucumber_Playwright_JUnit5
Location/path where the project needs to be saved – Documents/Playwright (my location)
Select JDK version — I am using JDK 17
Archetype — Search for “quickstart” and select maven-archetype-quickstart from the results

Click on the Create button to create the project.

This will create a a new project in IntelliJ.

2. Set Up the Environment

2.1 Add the Cucumber, Playwright Java, and JUnit5 dependencies

Add the Cucumber, Playwright Java, and JUnit5 dependencies to the pom.xml. The latest dependency can be found here.

<properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <java.version>17</java.version>
        <cucumber.version>7.33.0</cucumber.version>
        <playwright.version>1.57.0</playwright.version>
        <lombok.version>1.18.30</lombok.version>
        <junit.jupiter.engine.version>6.1.0-M1</junit.jupiter.engine.version>
        <junit.jupiter.version>6.1.0-M1</junit.jupiter.version>
        <maven.compiler.plugin.version>3.14.1</maven.compiler.plugin.version>
        <maven.compiler.source.version>17</maven.compiler.source.version>
        <maven.compiler.target.version>17</maven.compiler.target.version>
        <maven.surefire.plugin.version>3.5.4</maven.surefire.plugin.version>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.target>17</maven.compiler.target>
        <maven.site.plugin.version>4.0.0-M16</maven.site.plugin.version>
        <maven.surefire.report.plugin.version>3.5.4</maven.surefire.report.plugin.version>
        <cucumber.junit.platform.engine>7.14.0</cucumber.junit.platform.engine>
    </properties>

    <dependencies>
        <dependency>
            <groupId>com.microsoft.playwright</groupId>
            <artifactId>playwright</artifactId>
            <version>${playwright.version}</version>
        </dependency>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>${junit.jupiter.engine.version}</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>${junit.jupiter.version}</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-params</artifactId>
            <version>${junit.jupiter.version}</version>
            <scope>test</scope>
        </dependency>

        <!-- Cucumber with Java -->
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>${cucumber.version}</version>
            <scope>test</scope>
        </dependency>

        <!-- Cucumber with Junit5 -->
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-junit-platform-engine</artifactId>
            <version>${cucumber.junit.platform.engine}</version>
        </dependency>

        <!-- JUnit Platform -->
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-suite</artifactId>
            <version>${junit.jupiter.version}</version>
            <scope>test</scope>
        </dependency>

    </dependencies>

2.2 Add Maven Compiler Plugin and Surefire Plugin

The compiler plugin is used to compile the source code of a Maven project. This plugin has two goals, which are already bound to specific phases of the default lifecycle:

compile – compile main source files
testCompile – compile test source files

 <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${maven.compiler.plugin.version}</version>
                <configuration>
                    <source>${maven.compiler.source.version}</source>
                    <target>${maven.compiler.target.version}</target>
                </configuration>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>${maven.surefire.plugin.version}</version>
                <configuration>
                    <testFailureIgnore>true</testFailureIgnore>
                </configuration>
            </plugin>

        </plugins>
    </build>

2.3 Add Maven Site Plugin to generate Surefire Report

<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-site-plugin</artifactId>
        <version>${maven.site.plugin.version}</version>
</plugin>

<reporting>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-report-plugin</artifactId>
                <version>${maven.surefire.report.plugin.version}</version>
                <configuration>
                    <outputName>JUnit5 Report</outputName>
                </configuration>
            </plugin>
        </plugins>
    </reporting>

The entire POM.xml file is shown below:-

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>Cucumber_Playwright_JUnit5</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>Cucumber_Playwright_JUnit5</name>
    <url>http://maven.apache.org</url>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <java.version>17</java.version>
        <cucumber.version>7.33.0</cucumber.version>
        <playwright.version>1.57.0</playwright.version>
        <lombok.version>1.18.30</lombok.version>
        <junit.jupiter.engine.version>6.1.0-M1</junit.jupiter.engine.version>
        <junit.jupiter.version>6.1.0-M1</junit.jupiter.version>
        <maven.compiler.plugin.version>3.14.1</maven.compiler.plugin.version>
        <maven.compiler.source.version>17</maven.compiler.source.version>
        <maven.compiler.target.version>17</maven.compiler.target.version>
        <maven.surefire.plugin.version>3.5.4</maven.surefire.plugin.version>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.target>17</maven.compiler.target>
        <maven.site.plugin.version>4.0.0-M16</maven.site.plugin.version>
        <maven.surefire.report.plugin.version>3.5.4</maven.surefire.report.plugin.version>
        <cucumber.junit.platform.engine>7.14.0</cucumber.junit.platform.engine>
    </properties>

    <dependencies>
        <dependency>
            <groupId>com.microsoft.playwright</groupId>
            <artifactId>playwright</artifactId>
            <version>${playwright.version}</version>
        </dependency>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>${junit.jupiter.engine.version}</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>${junit.jupiter.version}</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-params</artifactId>
            <version>${junit.jupiter.version}</version>
            <scope>test</scope>
        </dependency>

        <!-- Cucumber with Java -->
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>${cucumber.version}</version>
            <scope>test</scope>
        </dependency>

        <!-- Cucumber with Junit5 -->
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-junit-platform-engine</artifactId>
            <version>${cucumber.junit.platform.engine}</version>
        </dependency>

        <!-- JUnit Platform -->
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-suite</artifactId>
            <version>${junit.jupiter.version}</version>
            <scope>test</scope>
        </dependency>

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${maven.compiler.plugin.version}</version>
                <configuration>
                    <source>${maven.compiler.source.version}</source>
                    <target>${maven.compiler.target.version}</target>
                </configuration>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>${maven.surefire.plugin.version}</version>
                <configuration>
                    <testFailureIgnore>true</testFailureIgnore>
                </configuration>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-site-plugin</artifactId>
                <version>${maven.site.plugin.version}</version>
            </plugin>
        </plugins>
    </build>

    <reporting>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-report-plugin</artifactId>
                <version>${maven.surefire.report.plugin.version}</version>
                <configuration>
                    <outputName>JUnit5 Report</outputName>
                </configuration>
            </plugin>
        </plugins>
    </reporting>
</project>

3. Define Directory Structure

Creating a well-organized project structure is crucial for maintaining a scalable and efficient automation framework.

1. Maven Configuration (pom.xml): This file manages project dependencies, plugins, and other configurations.

2. Features (src/test/resources/…/features): It contains Gherkin feature files written in plain English.

3. StepDefinition (src/test/java/.../tests/): Here, you write your actual test cases, utilizing the page classes. Cucumber steps are implemented.

4. Runner (src/main/java/…runner/): It connects Cucumber feature files with Junit Platform and triggers execution.

5. Hooks (src/test/java/…/hooks): Hooks manage test lifecycle events like Launch browser before scenario, Close browser after scenario, Capture screenshots on failure

6. Page Classes (src/main/java/.../pages/): Each web page in your application should have a corresponding page class. This class encapsulates all the functionalities of the web page, following the Page Object Model.

7. Steps (src/test/java/…/steps): It is an extra abstraction layer between step definitions and pages. It contains code that is common across scenarios.

8. Utility Classes (src/main/java/.../utils/): These classes can include various utilities like configuration readers, browser factory, helper methods for common tasks, etc.

9. junit-platform.properties: It contains global configuration for Junit Platform.

4. Create a feature file under src/test/resources/features

It is recommended to create a features folder in the src/test/resources directory. Create all the feature files in this features folder. Feature file should be saved as an extension of .feature. The test scenarios in the Feature file are written in Gherkins language. Add the test scenarios in this feature file. I have added sample test scenarios.

Feature: Login to HRM Application

  Background:
    Given User is on HRMLogin page "https://opensource-demo.orangehrmlive.com/"

  @ValidCredentials
  Scenario: Login with valid credentials

    When User enters username as "Admin" and password as "admin123"
    Then User should be able to login successfully and new page opens with heading "Dashboard"

  @InvalidCredentials
  Scenario Outline: Login with invalid credentials

    When User enters username as "<username>" and password as "<password>"
    Then User should be able to see error message "<errorMessage>"

    Examples:
      | username   | password  | errorMessage           |
      | Admin      | admin12$$ | Invalid credentials    |
      | admin$$    | admin123  | Invalid credentials    |
      | abc123     | xyz$$     | Invalid credentials    |

  @MissingUsername 
  Scenario: Login with blank username

    When User enters username as " " and password as "admin123"
    Then User should be able to see a message "Required" below Username

5. Create the utility package in src/test/java

Create a Playwright Factory that includes methods to start and close the browser.

package com.example.utils;

import com.microsoft.playwright.*;

public class PlaywrightFactory {

    private static Playwright playwright;
    private static Browser browser;
    private static BrowserContext context;
    private static Page page;

    public static void initBrowser() {

        playwright = Playwright.create();
        browser = playwright.chromium().launch(new BrowserType.LaunchOptions().setHeadless(false));
        context = browser.newContext();
        page = context.newPage();
    }

    public static Page getPage() {
        return page;
    }

    public static void closeBrowser() {
        if (page != null) page.close();
        if (context != null) context.close();
        if (browser != null) browser.close();
        if (playwright != null) playwright.close();
    }
}

6. Create the hooks package in src/test/java

Create a Hook class that contains the code that runs before or after each scenario. Here, it is responsible for

Starting the Playwright browser before a scenario
Closing the browser after the scenario finishes

Below is the sample code.

package com.example.hooks;

import com.example.utils.PlaywrightFactory;
import com.microsoft.playwright.*;
import io.cucumber.java.After;
import io.cucumber.java.Before;
import io.cucumber.java.Scenario;

public class Hooks {

    @Before
    public void beforeScenario(Scenario scenario) {
        System.out.println("Starting scenario: " + scenario.getName());
        PlaywrightFactory.initBrowser();
    }

    @After
    public void afterScenario(Scenario scenario) {
        Page page = PlaywrightFactory.getPage();
        if (page != null && scenario.isFailed()) {
            scenario.attach(page.screenshot(), "image/png", scenario.getName());
        }
        PlaywrightFactory.closeBrowser();  
        System.out.println("Finished scenario: " + scenario.getName());
    }
}

7. Create the pages package in src/test/java

Each web page in your application should have a corresponding page class. This class encapsulates all the functionalities of the web page, following the Page Object Model.

LoginPage

package com.example.pages;

import com.microsoft.playwright.*;

public class LoginPage {

    private Page page;

    // Locators
    private final Locator usernameLocator;
    private final Locator passwordLocator;
    private final Locator submitLocator;
    private final Locator invalidCredentialsLocator;
    private final Locator missingUsernameLocator;

    public LoginPage(Page page) {
        this.page = page;
        this.usernameLocator = page.locator("input[name='username']");
        this.passwordLocator = page.locator("input[name='password']");
        this.submitLocator = page.locator("button[type='submit']");
        this.invalidCredentialsLocator = page.locator("//p[contains(@class, "oxd-text oxd-text--p oxd-alert-content-text")]");
        this.missingUsernameLocator = page.locator("//span[contains(@class, 'oxd-text oxd-text--span oxd-input-field-error-message oxd-input-group__message')]");
    }

    public void login(String user, String pass){
        usernameLocator.fill(user);
        passwordLocator.fill(pass);
        submitLocator.click();
    }

    public void getUrl(String url){
        page.navigate(url);
    }

    public String getMissingUsernameErrorMessage() {
        return missingUsernameLocator.textContent();
    }

    public String getErrorMessage () {
        return invalidCredentialsLocator.textContent();
    }
}

DashboardPage

package com.example.pages;

import com.microsoft.playwright.Locator;
import com.microsoft.playwright.Page;

public class DashboardPage {

    private Page page;

    private final Locator headingLocator;

    public DashboardPage(Page page){
        this.page = page;
        this.headingLocator = page.locator("//h6[contains(@class, "oxd-topbar-header-breadcrumb-module")]");
    }

    public String getDashboardPageHeading() {
        return headingLocator.textContent();
    }

}

8. Create the step package in src/test/java

package com.example.steps;

import com.example.utils.PlaywrightFactory;
import com.microsoft.playwright.Page;

public class BaseStep {

    protected Page getPage() {
        Page page = PlaywrightFactory.getPage();
        if (page == null) {
            throw new RuntimeException(
                    "Page is NULL. Ensure @Before hook ran and glue includes hooks package."
            );
        }
        return page;
    }
}

9. Create the step definition class in src/test/java

Create the step definition class corresponding to the feature file to test the scenarios in the src/test/java directory. The StepDefinition files should be created in this definitions directory within the folder called definitions.

Below is the step definition of the LoginPage feature file.

package com.example.definitions;

import com.example.pages.DashboardPage;
import com.example.pages.LoginPage;
import com.example.steps.BaseStep;
import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;
import org.junit.jupiter.api.Assertions;


public class LoginPageDefinitions extends BaseStep {

    private LoginPage loginPage;
    private DashboardPage dashboardPage;

    @Given("User is on HRMLogin page {string}")
    public void userIsOnHRMLoginPage(String baseUrl) {
        loginPage = new LoginPage(getPage());
        loginPage.getUrl(baseUrl);
    }

    @When("User enters username as {string} and password as {string}")
    public void userEntersUsernameAsAndPasswordAs(String username, String password) {
        loginPage.login(username, password);
    }

    @Then("User should be able to see error message {string}")
    public void userShouldBeAbleToSeeErrorMessage(String expectedErrorMessage) {
        Assertions.assertEquals(expectedErrorMessage, loginPage.getErrorMessage());
    }

    @Then("User should be able to see a message {string} below Username")
    public void userShouldBeAbleToSeeAMessageBelowUsername(String expectedErrorMessage) {
        Assertions.assertEquals(expectedErrorMessage, loginPage.getMissingUsernameErrorMessage());
    }

    @Then("User should be able to login successfully and new page opens with heading {string}")
    public void userShouldBeAbleToLoginSuccessfullyAndNewPageOpen(String expectedHeading) {
        dashboardPage = new DashboardPage(getPage());
        Assertions.assertEquals(expectedHeading, dashboardPage.getDashboardPageHeading());

    }

}

10. Create a JUnit Platform Cucumber Runner class in src/test/java

We need to create a class called Runner class to run the tests.

This JUnit 5 runner configures and launches Cucumber scenarios by selecting feature files, defining glue code packages, and executing them via the Cucumber engine on the JUnit Platform.

package com.example.runner;

import static io.cucumber.junit.platform.engine.Constants.GLUE_PROPERTY_NAME;
import org.junit.platform.suite.api.ConfigurationParameter;
import org.junit.platform.suite.api.IncludeEngines;
import org.junit.platform.suite.api.SelectClasspathResource;
import org.junit.platform.suite.api.Suite;

@Suite
@IncludeEngines("cucumber")
@SelectClasspathResource("features")
@ConfigurationParameter(
        key = GLUE_PROPERTY_NAME, value = "com.example"
)

public class RunnerTests {
}

1. @Suite – Marks this class as a JUnit 5 test suite. Acts as the entry point for test execution

2. @IncludeEngines(“cucumber”) – Tells JUnit Platform to use the Cucumber engine

3. @SelectClasspathResource(“features”) – Specifies the location of feature file

4. @ConfigurationParameter – Tells Cucumber where to find step definitions and hooks

11. Run the test using the Junit Platform Runner File

Go to the Runner class and right-click and select Run ‘RunnerTests’. The tests will run in IntelliJ.

Below is the test execution result. 1 test is failed intentionally to show the status of test execution in the report.

12. Run the tests from the Command Line

Run the command below in the command prompt to run the tests and to get the test execution report.

mvn clean site test

The execution screen looks like something as shown below.

13. Cucumber Report Generation

Add junit-platform.properties under src/test/resources and add the below instructions in the file.

cucumber.publish.enabled=true

Below is the image of the Cucumber Report generated using the Cucumber Service.

We can see the details of the failed test in the Report.

The highlighted part holds the screenshot of the failed test.

14. Surefire Report Generation

Maven Site Plugin creates a folder – site under the target directory.

Junit also produces an “index.html” report, and it resides under the target folder. The below image shows the index.html report. This is the latest theme of the report.

Recap of Key Points:

Playwright and Java: We explored the powerful combination of Playwright with Java. We highlighted the benefits of this duo in automated testing.
Build: Maven manage the dependencies and execution.
Cucumber (BDD Framework): It enables Behavior-Driven Development (BDD).
Feature: It uses Gherkin feature file to create the scenarios.
StepDefinition: Stepdefinition file contains the test code related to the feature file. When Cucumber executes a Gherkin step in a scenario, it will look for a matching step definition to execute.
Test Runner: It acts as a bridge between the feature and StepDefinition file.
JUnit5: It controls the test execution lifecycle. It integrates with Cucumber via Junit Platform runner.
Hooks: Hooks manage browser setup, teardown, and failure handling

February 13, 2026February 13, 2026 vibssingh

Integration of Playwright with Java, Cucumber and TestNG

Last Updated on

February 13, 2026

HOME

Playwright is integrated with Java, Cucumber, and TestNG. This integration creates a robust and scalable test automation framework for modern web applications. It is a BDD-driven framework.

Table of Contents

What is Playwright?

What is Cucumber?

In behavior-driven development with Cucumber, test cases are written in Gherkin syntax. They use a Given-When-Then structure. This structure closely mirrors user stories and business rules.

Benefits of Using Cucumber with Playwright

4. Cross-browser and cross-platform coverage: When combined with Playwright, Cucumber scenarios can be executed across Chromium, Firefox, and WebKit without changing feature files

5. Scales well in CI/CD pipelines: The combination supports parallel execution. It offers tagging and selective test runs. This makes it suitable for continuous testing in CI environments.

What is TestNG?

TestNG is a testing framework widely used with Selenium for building structured, maintainable, and efficient automated tests. Combining TestNG listeners with Playwright gives testers greater control over test monitoring, logging, and custom actions based on test outcomes. In TestNG you can initialize Playwright and Browser in @BeforeClass method and destroy them in @AfterClass.

System requirements

The following prerequisites are required to be installed on the machine to begin with a smooth setup and installation.

Java 11 or higher
IntelliJ IDE or any other IDE to create a project
Maven
Browsers on which tests need to be run, like Chrome, Firefox, etc.

High-Level Architecture

Setting Up Cucumber BDD Framework with Playwright and TestNG

1. Create a new Maven Project

The first step in setup is to create a new Maven project. I will be using IntelliJ in this tutorial. The following steps need to be followed to create a new Maven project :

Open IntelliJ, Navigate to File >> New >> Project

In the New Project window, enter the following details:

Name of the Project – Cucumber_Playwright_TestNG
Location/path where the project needs to be saved – Documents/Playwright (my location)
Select JDK version — I am using JDK 17
Archetype — Search for “quickstart” and select maven-archetype-quickstart from the results

Click on the Create button to create the project.

This will create a project as shown in IntelliJ.

2. Set Up the Environment

2.1 Add the Cucumber, Playwright Java, and TestNG dependencies

Add the Cucumber, Playwright Java, and TestNG dependencies to the pom.xml. The latest dependency can be found here.

<properties>
    <java.version>17</java.version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <cucumber.version>7.33.0</cucumber.version>
    <testng.version>7.12.0</testng.version>
    <playwright.version>1.57.0</playwright.version>
    <maven.compiler.version>3.14.1</maven.compiler.version>
    <maven.surefire.version>3.5.4</maven.surefire.version>
    <maven.failsafe.version>3.5.4</maven.failsafe.version>
    <lombok.version>1.18.30</lombok.version>
    <maven.compiler.source>17</maven.compiler.source>
    <maven.compiler.target>17</maven.compiler.target>
</properties>

<dependencies>

    <!-- Cucumber with Java -->
    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-java</artifactId>
      <version>${cucumber.version}</version>
      <scope>test</scope>
    </dependency>

    <!-- Cucumber with TestNG -->
    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-testng</artifactId>
      <version>${cucumber.version}</version>
      <scope>test</scope>
    </dependency>

    <!-- TestNG -->
    <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>${testng.version}</version>
      <scope>test</scope>
    </dependency>

    <!-- Playwright -->
    <dependency>
      <groupId>com.microsoft.playwright</groupId>
      <artifactId>playwright</artifactId>
      <version>${playwright.version}</version>
    </dependency>

</dependencies>

2.2 Add Maven Compiler Plugin and SureFire Plugin

The compiler plugin is used to compile the source code of a Maven project. This plugin has two goals, which are already bound to specific phases of the default lifecycle:

compile – compile main source files
testCompile – compile test source files

 <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>${maven.compiler.version}</version>
        <configuration>
          <source>${maven.compiler.source}</source>
          <target>${maven.compiler.target}</target>
        </configuration>
      </plugin>

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>${maven.surefire.version}</version>
        <configuration>
          <testFailureIgnore>true</testFailureIgnore>
          <includes>
            <include>**/*RunnerTests.java</include>
          </includes>
        </configuration>
      </plugin>
      
    </plugins>
  </build>

The entire POM.xml file is shown below:-

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.example</groupId>
  <artifactId>Cucumber_Playwright_TestNG</artifactId>
  <version>1.0-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>Cucumber_Playwright_TestNG</name>
  <url>http://maven.apache.org</url>

  <properties>
    <java.version>17</java.version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <cucumber.version>7.33.0</cucumber.version>
    <testng.version>7.12.0</testng.version>
    <playwright.version>1.57.0</playwright.version>
    <maven.compiler.version>3.14.1</maven.compiler.version>
    <maven.surefire.version>3.5.4</maven.surefire.version>
    <maven.failsafe.version>3.5.4</maven.failsafe.version>
    <lombok.version>1.18.30</lombok.version>
    <maven.compiler.source>17</maven.compiler.source>
    <maven.compiler.target>17</maven.compiler.target>
  </properties>

  <dependencies>

    <!-- Cucumber with Java -->
    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-java</artifactId>
      <version>${cucumber.version}</version>
      <scope>test</scope>
    </dependency>

    <!-- Cucumber with TestNG -->
    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-testng</artifactId>
      <version>${cucumber.version}</version>
      <scope>test</scope>
    </dependency>

    <!-- TestNG -->
    <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>${testng.version}</version>
      <scope>test</scope>
    </dependency>

    <!-- Playwright -->
    <dependency>
      <groupId>com.microsoft.playwright</groupId>
      <artifactId>playwright</artifactId>
      <version>${playwright.version}</version>
    </dependency>

  </dependencies>
 
 <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>${maven.compiler.version}</version>
        <configuration>
          <source>${maven.compiler.source}</source>
          <target>${maven.compiler.target}</target>
        </configuration>
      </plugin>

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>${maven.surefire.version}</version>
        <configuration>
          <testFailureIgnore>true</testFailureIgnore>
          <includes>
            <include>**/*RunnerTests.java</include>
          </includes>
        </configuration>
      </plugin>
      
    </plugins>
  </build>
</project>

3. Define Directory Structure

Creating a well-organized project structure is crucial for maintaining a scalable and efficient automation framework.

1. Maven Configuration (pom.xml): This file manages project dependencies, plugins, and other configurations.

2. Features (src/test/resources/…/features): Contains Gherkin feature files written in plain English.

3. StepDefinition (src/test/java/.../tests/): Here, you write your actual test cases, utilizing the page classes. Cucumber steps are implemented.

4. Runner (src/main/java/…runner/): It connects Cucumber feature files with TestNG and triggers execution.

5. Hooks (src/test/java/…/hooks): Hooks manage test lifecycle events like Launch browser before scenario, Close browser after scenario, Capture screenshots on failure

7. Steps (src/test/java/…/steps): It is an extra abstraction layer between step definitions and pages. It contains code which is common across scenarios.

8. Utility Classes (src/main/java/.../utils/): These classes can include various utilities like configuration readers, browser factory, helper methods for common tasks, etc.

9. Test Data Files (src/test/resources/testdata/): Externalizes test data to avoid hardcoding. This includes items like usernames and passwords. These are stored in external files for easy management.

10. cucumber.properties: It contains global configuration for Cucumber.

11. testng.xml: This file is the central configuration file that defines how the test suite should be executed.

4. Create a feature file under src/test/resources

Feature: Login to HRM Application

  Background:
    Given User is on HRMLogin page "https://opensource-demo.orangehrmlive.com/"

  @ValidCredentials
  Scenario: Login with valid credentials

    When User enters username as "Admin" and password as "admin123"
    Then User should be able to login successfully and new page opens with heading "Dashboard"

  @InvalidCredentials
  Scenario Outline: Login with invalid credentials

    When User enters username as "<username>" and password as "<password>"
    Then User should be able to see error message "<errorMessage>"

    Examples:
      | username   | password  | errorMessage           |
      | Admin      | admin12$$ | Invalid credentials    |
      | admin$$    | admin123  | Invalid credentials    |
      | abc123     | xyz$$     | Invalid credentials    |

  @MissingUsername 
  Scenario: Login with blank username

    When User enters username as " " and password as "admin123"
    Then User should be able to see a message "Required1" below Username

5. Create the utility package in src/test/java

Create a Playwright Factory that contains the methods to start the browser and close the browser.

package com.example.utils;

import com.microsoft.playwright.*;

public class PlaywrightFactory {

    private static Playwright playwright;
    private static Browser browser;
    private static BrowserContext context;
    private static Page page;

    public static void initBrowser() {

        playwright = Playwright.create();
        browser = playwright.chromium().launch(new BrowserType.LaunchOptions().setHeadless(false));
        context = browser.newContext();
        page = context.newPage();
    }

    public static Page getPage() {
        return page;
    }

    public static void closeBrowser() {
        if (page != null) page.close();
        if (context != null) context.close();
        if (browser != null) browser.close();
        if (playwright != null) playwright.close();
    }
}

6. Create the hooks package in src/test/java

Create a Hook class that contain the code that runs before or after each scenario. Here, it is responsible for

Starting the Playwright browser before a scenario
Closing the browser after the scenario finishes

Below is the sample code.

package com.example.hooks;

import com.example.utils.PlaywrightFactory;
import com.microsoft.playwright.*;
import io.cucumber.java.After;
import io.cucumber.java.Before;
import io.cucumber.java.Scenario;

public class Hooks {

    @Before
    public void beforeScenario(Scenario scenario) {
        System.out.println("Starting scenario: " + scenario.getName());
        PlaywrightFactory.initBrowser();
    }

    @After
    public void afterScenario(Scenario scenario) {
        Page page = PlaywrightFactory.getPage();
        if (page != null && scenario.isFailed()) {
            scenario.attach(page.screenshot(), "image/png", scenario.getName());
        }
        PlaywrightFactory.closeBrowser();  
        System.out.println("Finished scenario: " + scenario.getName());
    }
}

7. Create the pages package in src/test/java

Each web page in your application should have a corresponding page class. This class encapsulates all the functionalities of the web page, following the Page Object Model.

LoginPage

package com.example.pages;

import com.microsoft.playwright.*;

public class LoginPage {

    private Page page;

    // Locators
    private final Locator usernameLocator;
    private final Locator passwordLocator;
    private final Locator submitLocator;
    private final Locator invalidCredentialsLocator;
    private final Locator missingUsernameLocator;

    public LoginPage(Page page) {
        this.page = page;
        this.usernameLocator = page.locator("input[name='username']");
        this.passwordLocator = page.locator("input[name='password']");
        this.submitLocator = page.locator("button[type='submit']");
        this.invalidCredentialsLocator = page.locator("//p[contains(@class, \"oxd-text oxd-text--p oxd-alert-content-text\")]");
        this.missingUsernameLocator = page.locator("//span[contains(@class, 'oxd-text oxd-text--span oxd-input-field-error-message oxd-input-group__message')]");
    }

    public void login(String user, String pass){
        usernameLocator.fill(user);
        passwordLocator.fill(pass);
        submitLocator.click();
    }

    public void getUrl(String url){
        page.navigate(url);
    }

    public String getMissingUsernameErrorMessage() {
        return missingUsernameLocator.textContent();
    }

    public String getErrorMessage () {
        return invalidCredentialsLocator.textContent();
    }
}

DashboardPage

package com.example.pages;

import com.microsoft.playwright.Locator;
import com.microsoft.playwright.Page;

public class DashboardPage {

    private Page page;

    private final Locator headingLocator;

    public DashboardPage(Page page){
        this.page = page;
        this.headingLocator = page.locator("//h6[contains(@class, \"oxd-topbar-header-breadcrumb-module\")]");
    }

    public String getDashboardPageHeading() {
        return headingLocator.textContent();
    }

}

8. Create the step package in src/test/java

package com.example.steps;

import com.example.utils.PlaywrightFactory;
import com.microsoft.playwright.Page;

public class BaseStep {

    protected Page getPage() {
        Page page = PlaywrightFactory.getPage();
        if (page == null) {
            throw new RuntimeException(
                    "Page is NULL. Ensure @Before hook ran and glue includes hooks package."
            );
        }
        return page;
    }
}

9. Create the step definition class in src/test/java

Below is the step definition of the LoginPage feature file.

package com.example.definitions;

import com.example.steps.BaseStep;
import com.example.pages.DashboardPage;
import com.example.pages.LoginPage;
import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;
import org.testng.Assert;

public class LoginPageDefinitions extends BaseStep {
    
    private LoginPage loginPage;
    private DashboardPage dashboardPage;

    @Given("User is on HRMLogin page {string}")
    public void userIsOnHRMLoginPage(String baseUrl) {
        loginPage = new LoginPage(getPage());
        loginPage.getUrl(baseUrl);
    }

    @When("User enters username as {string} and password as {string}")
    public void userEntersUsernameAsAndPasswordAs(String username, String password) {
        loginPage.login(username, password);
    }

    @Then("User should be able to see error message {string}")
    public void userShouldBeAbleToSeeErrorMessage(String expectedErrorMessage) {
        Assert.assertEquals(loginPage.getErrorMessage(), expectedErrorMessage);
    }

    @Then("User should be able to see a message {string} below Username")
    public void userShouldBeAbleToSeeAMessageBelowUsername(String expectedErrorMessage) {
        Assert.assertEquals(loginPage.getMissingUsernameErrorMessage(), expectedErrorMessage);
    }

    @Then("User should be able to login successfully and new page opens with heading {string}")
    public void userShouldBeAbleToLoginSuccessfullyAndNewPageOpen(String expectedHeading) {
        dashboardPage = new DashboardPage(getPage());
        Assert.assertEquals(dashboardPage.getDashboardPageHeading(), expectedHeading);

    }

}

10. Create a TestNG Cucumber Runner class in src/test/java

We need to create a class called Runner class to run the tests. This class will use the TestNG annotation @RunWith(), which tells TestNG what is the test runner class. TestRunner should be created under src/test/java within the folder called runner.

AbstractTestNGCucumberTests is a class provided by the Cucumber framework when we are integrating Cucumber with the TestNG test framework. This abstract class helps facilitate the execution of Cucumber tests using TestNG.

package com.example.runner;

import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;

@CucumberOptions(features = "src/test/resources/features/LoginPage.feature",
        glue = {"com.example.hooks","com.example.definitions"},
        plugin = {
                "pretty",
        }
)

public class RunnerTests extends AbstractTestNGCucumberTests {

}

11. Create testng.xml

We need to specify the values of browser in the TestNG XML file that will pass to the test case file.

Let’s create a testng.xml file to run the test using TestNG.

Right click on the project, New >> File

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Playwright test suite" thread-count="3">
    <test name="Chromium Tests">
        <classes>
            <class name="com.example.runner.RunnerTests"/>
        </classes>
    </test>
</suite>

12. Run the test through Runner Class

Go to the Runner class and right-click and select Run ‘RunnerTests’. The tests will run as TestNG tests (in IntelliJ).

Below is the test execution result.

13. Run the tests from TestNG.xml

Execute the tests by right-clicking on testng.xml and select Run ‘…testng.xml’

This will run the tests in chromium as mentioned in the .xml file.

The output of the above program is

14. Run the tests from the Command Line

Run the below command in the command prompt to run the tests and to get the test execution report.

mvn clean test

The execution screen looks like something as shown below.

15. Cucumber Report Generation

Add cucumber.properties under src/test/resources and add the below instructions in the file.

cucumber.publish.enabled=true

Below is the image of the Cucumber Report generated using the Cucumber Service.

The screenshot of the failed test is also attached in the Cucumber Report.

Click on the highlighted part. It contains the screenshot.

16. TestNG Report Generation

TestNG generates various types of reports under test-output folder like emailable-report.html, index.html, testng-results.xml.

We are interested in the emailable-report.html report. Open emailable-report.html, as this is an HTML report, and open it with the browser. The below image shows emailable-report.html.

TestNG also produces an “index.html” report, and it resides under the test-output folder. The below image shows the index.html report. This is the latest theme of the report.

Recap of Key Points:

Playwright and Java: We explored the powerful combination of Playwright with Java. We highlighted the benefits of this duo in automated testing.
Build: Maven manage the dependencies and execution.
Cucumber (BDD Framework): It enables Behavior-Driven Development (BDD).
Feature: It uses Gherkin feature file to create the scenarios.
StepDefinition: StepDefinition file contains the test code related to the feature file. When Cucumber executes a Gherkin step in a scenario, it will look for a matching step definition to execute.
Test Runner: It acts as a bridge between the feature and StepDefinition file.
TestNG: It controls the test execution lifecycle. It integrates with Cucumber via TestNG runner.
Hooks: Hooks manage browser setup, teardown, and failure handling

November 8, 2024November 8, 2024 vibssingh

How to rerun failed tests twice in Cucumber

HOME

The previous tutorial explained the How to rerun failed tests in Cucumber. There are multiple times that we need to rerun the failed tests twice to overcome the intermittent network or environment issues. This can be achieved by creating multiple TestRunner class.

Table of Contents

Set Up a Runner for First Run of Tests

In the main test runner, configure Cucumber to generate a JSON report for the initial test run and capture any failed tests in a rerun file. We are using rerun plugin to logs the paths of failed scenarios in a text file – rerun.txt that will be created inside the target folder.

import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;

@CucumberOptions(features = "src/test/resources/features/LoginPage.feature",
        glue = "com.example.definitions",
        plugin = {
                "pretty",
                "html:target/cucumber-reports/cucumber-report.html",
                "json:target/cucumber-reports/cucumber-report.json",
                "rerun:target/rerun.txt" // Saves paths of failed scenarios
        }
)
public class RunnerTests extends AbstractTestNGCucumberTests {
}

Set Up a Runner for running the failed tests first time

Create a second runner that reads from the rerun.txt file and generates a new rerun1.txt file that contains the path of failed tests and create a separate JSON report for failed test reruns.

import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;

@CucumberOptions(tags = "",
        features = "@target/rerun.txt",
        glue = "com.example.definitions",
        plugin =  {
                "pretty",
                "html:target/cucumber-reports/cucumber-rerun1-report.html",
                "json:target/cucumber-reports/cucumber-rerun1-report.json",
                "rerun:target/rerun1.txt"
        }
)

public class RunnerTestsFailed extends AbstractTestNGCucumberTests {
}

Set Up a Runner for running the failed tests second time

Create a third runner that reads from the rerun1.txt file and generates a separate JSON report for failed test reruns.

import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;

@CucumberOptions(tags = "",
        features = "@target/rerun1.txt",
        glue = "com.example.definitions",
        plugin =  {
                "pretty",
                "html:target/cucumber-reports/cucumber-rerun2-report.html",
                "json:target/cucumber-reports/cucumber-rerun2-report.json"
        }
)

public class RunnerTestsSecondFailed extends AbstractTestNGCucumberTests {
}

Mention all the Test Runner details in the testng.xml

We need to mention all the TestRunner class name in the testng.xml. This will run first runner that will generate a file rerun.txt which in turns contain the path of the failed scenarios. Then, second runner will use this new file rerun.txt as input to the feature file and rerun the failed tests and generate second file – rerun1.txt which is used as input to the third Test Runner file.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
    <test  name="Cucumber with TestNG Test">
        <classes>
            <class name="com.example.runner.RunnerTests"/>
            <class name="com.example.runner.RunnerTestsFailed"/>
            <class name="com.example.runner.RunnerTestsSecondFailed"/>
        </classes>
    </test> <!-- Test -->
</suite> <!-- Suite -->

Add cucumber-reporting Plugin to pom.xml

Use the cucumber-reporting plugin by Masterthought to combine the JSON files into a single report. Configure the plugin in your pom.xml to read both the main and rerun JSON files.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.example</groupId>
  <artifactId>POM_Cucumber_TestNG_Demo</artifactId>
  <version>1.0-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>POM_Cucumber_TestNG_Demo</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <cucumber.version>7.18.1</cucumber.version>
    <selenium.version>4.25.0</selenium.version>
    <testng.version>7.10.2</testng.version>
    <apache.common.version>2.4</apache.common.version>
    <maven.compiler.plugin.version>3.13.0</maven.compiler.plugin.version>
    <maven.surefire.plugin.version>3.3.1</maven.surefire.plugin.version>
    <maven.compiler.source.version>17</maven.compiler.source.version>
    <maven.compiler.target.version>17</maven.compiler.target.version>
    <maven.cucumber.reporting.version>5.8.2</maven.cucumber.reporting.version>
  </properties>

  <dependencies>

    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-java</artifactId>
      <version>${cucumber.version}</version>
    </dependency>

    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-testng</artifactId>
      <version>${cucumber.version}</version>
      <scope>test</scope>
    </dependency>

    <!-- Selenium -->
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>${selenium.version}</version>
    </dependency>

    <!-- TestNG -->
    <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>${testng.version}</version>
      <scope>test</scope>
    </dependency>

    <!-- Cucumber Reporting -->
    <dependency>
      <groupId>net.masterthought</groupId>
      <artifactId>cucumber-reporting</artifactId>
      <version>${maven.cucumber.reporting.version}</version>
    </dependency>

  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>${maven.compiler.plugin.version}</version>
        <configuration>
          <source>${maven.compiler.source.version}</source>
          <target>${maven.compiler.target.version}</target>
        </configuration>
      </plugin>
      
        <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>${maven.surefire.plugin.version}</version>
        <configuration>
          <testFailureIgnore>true</testFailureIgnore>
          <suiteXmlFiles>
            <suiteXmlFile>testng.xml</suiteXmlFile>
          </suiteXmlFiles>
        </configuration>
      </plugin>

      <plugin>
        <groupId>net.masterthought</groupId>
        <artifactId>maven-cucumber-reporting</artifactId>
        <version>${maven.cucumber.reporting.version}</version>

        <executions>
          <execution>
            <id>generate-cucumber-report</id>
            <phase>verify</phase>
            <goals>
              <goal>generate</goal>
            </goals>
            <configuration>
              <projectName>Cucumber Reporting Example</projectName>
              <outputDirectory>target/cucumber-html-reports</outputDirectory>
              <inputDirectory>target</inputDirectory>
              <jsonFiles>
                <!-- Include both main and rerun JSON reports -->
                <param>target/cucumber-reports/cucumber-report.json</param>
                <param>target/cucumber-reports/cucumber-rerun-report.json</param>
                <param>**/*.json</param>
              </jsonFiles>
            </configuration>
          </execution>
        </executions>
      </plugin>

    </plugins>
  </build>
</project>

Run the tests and Generate the Report

Use the below command to run the tests and generate the consolidated Test Report.

mvn clean verify

RunnerTests class generates cucumber-report.json file whereas RunnerTestsFailed generates cucumber-rerun-report.json files are present in target/cucumber-reports folder.

MasterThought plugin uses these json files to generate a consolidated report in target/cucumber-html-reports.

There are different types of HTML reports gets generated as a part of the test execution cycle.

1. feature-overview – This HTML report gives an overall overview of test execution. Main HTML report which covers all different sections like Features, Tags, Steps, and Failures.

2. failures-overview – This HTML report gives an overview of all failed tests.

3. step-overview – This HTML report shows step statistics for the current cycle.

4. tag-overview – This HTML report shows passing and failing statistics for different tags used in test execution.

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

October 29, 2024December 10, 2024 vibssingh

How to create Cucumber Report after rerun of failed tests

Last Updated On

December 10, 2024

HOME

There are multiple reports produced when the Cucumber failed tests are redone. There is no unified test execution status provided in these reports. To generate a consolidated Cucumber report after rerunning any failed tests, follow these steps:

Table of Contents

Set Up a Runner for First Run of Tests

import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;

@CucumberOptions(features = "src/test/resources/features/LoginPage.feature",
        glue = "com.example.definitions",
        plugin = {
                "pretty",
                "html:target/cucumber-reports/cucumber-report.html",
                "json:target/cucumber-reports/cucumber-report.json",
                "rerun:target/rerun.txt" // Saves paths of failed scenarios
        }
)
public class RunnerTests extends AbstractTestNGCucumberTests {
}

Set Up a Runner for Failed Tests

Create a second runner that reads from the rerun.txt file and generates a separate JSON report for failed test reruns.

import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;

@CucumberOptions(tags = "",
        features = "@target/rerun.txt",
        glue = "com.example.definitions",
        plugin =  {
                "pretty",
                "html:target/cucumber-reports/cucumber-rerun-report.html",
                "json:target/cucumber-reports/cucumber-rerun-report.json"
        }
)

public class RunnerTestsFailed extends AbstractTestNGCucumberTests {
}

Mention both Test Runner details in the testng.xml

We need to mention both the TestRunner class name in the testng.xml. This will run both the runners sequentially.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
    <test  name="Cucumber with TestNG Test">
        <classes>
            <class name="com.example.runner.RunnerTests"/>
            <class name="com.example.runner.RunnerTestsFailed"/>
        </classes>
    </test> <!-- Test -->
</suite> <!-- Suite -->

Add cucumber-reporting Plugin to pom.xml

Use the cucumber-reporting plugin by Masterthought to combine the JSON files into a single report. Configure the plugin in your pom.xml to read both the main and rerun JSON files.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.example</groupId>
  <artifactId>POM_Cucumber_TestNG_Demo</artifactId>
  <version>1.0-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>POM_Cucumber_TestNG_Demo</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <cucumber.version>7.18.1</cucumber.version>
    <selenium.version>4.25.0</selenium.version>
    <testng.version>7.10.2</testng.version>
    <apache.common.version>2.4</apache.common.version>
    <maven.compiler.plugin.version>3.13.0</maven.compiler.plugin.version>
    <maven.surefire.plugin.version>3.3.1</maven.surefire.plugin.version>
    <maven.compiler.source.version>17</maven.compiler.source.version>
    <maven.compiler.target.version>17</maven.compiler.target.version>
    <maven.cucumber.reporting.version>5.8.2</maven.cucumber.reporting.version>
  </properties>

  <dependencies>

    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-java</artifactId>
      <version>${cucumber.version}</version>
    </dependency>

    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-testng</artifactId>
      <version>${cucumber.version}</version>
      <scope>test</scope>
    </dependency>

    <!-- Selenium -->
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>${selenium.version}</version>
    </dependency>

    <!-- TestNG -->
    <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>${testng.version}</version>
      <scope>test</scope>
    </dependency>

    <!-- Cucumber Reporting -->
    <dependency>
      <groupId>net.masterthought</groupId>
      <artifactId>cucumber-reporting</artifactId>
      <version>${maven.cucumber.reporting.version}</version>
    </dependency>

  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>${maven.compiler.plugin.version}</version>
        <configuration>
          <source>${maven.compiler.source.version}</source>
          <target>${maven.compiler.target.version}</target>
        </configuration>
      </plugin>
      
        <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>${maven.surefire.plugin.version}</version>
        <configuration>
          <testFailureIgnore>true</testFailureIgnore>
          <suiteXmlFiles>
            <suiteXmlFile>testng.xml</suiteXmlFile>
          </suiteXmlFiles>
        </configuration>
      </plugin>

      <plugin>
        <groupId>net.masterthought</groupId>
        <artifactId>maven-cucumber-reporting</artifactId>
        <version>${maven.cucumber.reporting.version}</version>

        <executions>
          <execution>
            <id>generate-cucumber-report</id>
            <phase>verify</phase>
            <goals>
              <goal>generate</goal>
            </goals>
            <configuration>
              <projectName>Cucumber Reporting Example</projectName>
              <outputDirectory>target/cucumber-html-reports</outputDirectory>
              <inputDirectory>target</inputDirectory>
              <jsonFiles>
                <!-- Include both main and rerun JSON reports -->
                <param>target/cucumber-reports/cucumber-report.json</param>
                <param>target/cucumber-reports/cucumber-rerun-report.json</param>
                <param>**/*.json</param>
              </jsonFiles>
            </configuration>
          </execution>
        </executions>
      </plugin>

    </plugins>
  </build>
</project>

Run the tests and Generate the Report

Use the below command to run the tests and generate the consolidated Test Report.

mvn clean verify

RunnerTests class generates cucumber-report.json file whereas RunnerTestsFailed generates cucumber-rerun-report.json files are present in target/cucumber-reports folder.

MasterThought plugin uses these json files to generate a consolidated report in target/cucumber-html-reports.

There are different types of HTML reports gets generated as a part of the test execution cycle.

1. feature-overview – This HTML report gives an overall overview of test execution. Main HTML report which covers all different sections like Features, Tags, Steps, and Failures.

2. failures-overview – This HTML report gives an overview of all failed tests.

3. step-overview – This HTML report shows step statistics for the current cycle.

4. tag-overview – This HTML report shows passing and failing statistics for different tags used in test execution.

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

October 18, 2024 vibssingh

BDD Multiple Choice Answers – MCQ1

HOME

BDD Quiz – You can test your knowledge with this BDD Quiz.

BDD Multiple Choice Questions – MCQ1

Answers

1) 1) Behavior Driven Development

2) 2) TDD

3) 2) BDD

4) 3) Discovery, Formulation, Automation

5) 2) Discovery, Formulation, Automation

6) 1) Common agreed understanding of the behaviour of the system

7) 2) Documented Examples

8) 1) Implemented Code + Automation Tests

9) 3) To have a different perspective from at least 3 different teams to discuss and create a common understanding.

10) 3) Product Owner, Developer & Tester

11) 2) General description of the software feature or change from an end-user perspective

12) 1) Product Owner

13) 4) All of the above

14) 4) All of the above

15) 3) All of the above

16) 3) Both (1) and (2)

17) 1) False

18) 2) Cucumber and 3) Serenity

19) 4) All of the above

20) 4) Ruby

August 26, 2024August 26, 2024 vibssingh

What is CucumberOptions in Cucumber?

HOME

In Cucumber, “@CucumberOptions“ is an annotation used to specify various options for running Cucumber tests. It allows you to configure different aspects of how your Cucumber tests are executed. This annotation is typically placed above the test runner class, which is responsible for triggering the execution of Cucumber tests.

1.Command-line arguments have the highest precedence. If you specify options through the CLI, they will override any settings provided by “@CucumberOptions“ or properties files. For example, running the tests with specific options from the terminal.

mvn clean test -Dcucumber.options="--tags @smoke --plugin pretty"

2. The “@CucumberOptions” annotation comes next in the precedence order. Settings configured here will override those provided in properties files but will be overridden by CLI arguments. For example, specifying configurations directly in the test runner class.

import org.junit.runner.RunWith;
import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;

@RunWith(Cucumber.class)
@CucumberOptions(
         features = "src/test/resources/features",
         glue = "com.mycompany.stepdefinitions",
         plugin = {"pretty", "html:target/cucumber-reports.html", "json:target/cucumber.json"},
         tags = "@smoke",
         monochrome = true,
         dryRun = false
     )

public class RunCucumberTests {
}

3. Configuration settings in properties files have the lowest precedence. These settings will be used if neither CLI arguments nor “@CucumberOptions“ provide a value for a particular option. For example, Creating a “cucumber.properties“ file to specify default configurations.

cucumber.features=src/test/resources/features
cucumber.glue=com.mycompany.stepdefinitions
cucumber.plugin=pretty, html:target/cucumber-reports.html, json:target/cucumber.json
cucumber.tags=@smoke

Configuration Options in CucumberOptions

Below is an example of CucumberOptions

import org.junit.runner.RunWith;
import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;

@RunWith(Cucumber.class)
@CucumberOptions( features = "src/test/resources/features",
    glue = "com.mycompany.stepdefinitions",
    plugin = {"pretty", "html:target/cucumber-reports.html", "json:target/cucumber.json"},
    tags = "@smoke",
    monochrome = true,
    dryRun = false,
    strict = true,
    name = "Login functionality"  )

public class RunCucumberTest {
}

In this example:

The “features” attribute points to the location of the feature files.
The “glue” attribute points to the package where the step definitions are located.
The “plugin” attribute specifies three plugins: one for pretty output in the console, one for an HTML report, and one for a JSON report.
The “tags” attribute specifies that only scenarios tagged with @smoke will be run.
The “monochrome” attribute is set to true to make the console output more readable.
The “dryRun” attribute is set to false to indicate that the tests should actually be executed.
The “strict” attribute treats undefined and pending steps as errors, ensuring all steps are fully implemented.
The “name” attribute allows running scenarios that match a specific name or pattern using regular expressions.

August 22, 2024May 7, 2025 vibssingh

What is Glue in Cucumber?

Last Updated On

May 7, 2025

HOME

This tutorial explains the Glue code in Cucumber.

Table of Contents

What is Glue code?

The glue option helps Cucumber locate the step definition file. In the glue cucumber option, we specify the package to load the glue code. Where glue code is our step definitions or hooks.

The glue code in Cucumber performs the following tasks:

– Scans the project for step definitions and associates them with the corresponding steps in the feature files.
– Executes the automation code defined in the step definitions when the corresponding step is encountered during test execution.
– Provides a way to configure Cucumber runtime options, such as defining test hooks, plugins, and other settings.

Cucumber Glue Example

import org.junit.runner.RunWith;
import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
 
@RunWith(Cucumber.class)
@CucumberOptions(plugin = "pretty", features = "src/test/resources/features",
glue = "com.example.customer")
 
public class CucumberRunnerTest {
 
}

This option specifies the package where the step definition classes are located. In this example, the step definitions are expected to be present in the “com.example.customer” package.

How Glue Locate Step Definitions

1. By default, Cucumber expects the step definitions to be in the same package as the Test Runner class.
2. If the “glue“ option is not specified in the Cucumber options, Cucumber will automatically assume that the step definitions are located in the same package as the test runner. Therefore, if your test runner class is “com.example.runner“, Cucumber will consider the “com.example.runner“ package as the default package for locating the step definitions.

You can comment or remove the “glue“ line in the Cucumber options if your test runner and step definitions are in the same package.

Multiple StepDefinitions in Glue Code

If the step definitions are present in 2 different packages, you can separate packages using a comma and double quotes.

glue = {"com.example.customer", "com.example.agents"}

Without proper glue settings, Cucumber will not be able to connect the feature files’ scenarios with the corresponding implementation, resulting in unimplemented step errors.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

June 4, 2024June 6, 2024 vibssingh

Cucumber Multiple Choice Answers – MCQ2

Last Updated On

June 6, 2024

HOME

Are you familiar with Cucumber and want to test your understanding? This post contains 25 useful Cucumber multiple-choice questions (quiz) to self-test your knowledge of Cucumber.

You can find the questions for the below-mentioned answers in this post –

Cucumber Quiz: 25 Multiple-Choice Questions with Answers – MCQ2

Answers

1) 4) All of the above

2) 1) Gherkin

3) 3) A table of input parameters

4) 2) Scenario Outline: Login with invalid credentials

5) 2) dryRun

6) 4) –glue

7) 2) –strict

8) 1) mvn test -Dcucumber.filter.tags=”@regression”

9) 1) mvn test -Dcucumber.features=”src/test/resources/features/Login.feature”

10) 3) mvn test -Dcucumber.glue=”com.example.stepdefinitions”

11) 1) mvn test -Dcucumber.plugin=”html:target/cucumber-reports/cucumberReport.html”

12) 1) True

13) 1) True

14) 2) Feature

15) 1)

mvn test -Dcucumber.features="src/test/resources/LoginPage.feature" -Dcucumber.filter.tags="@ValidCredentials"

16) 1) @CucumberOptions

17) 3)

import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
import org.junit.runner.RunWith;
 
@RunWith(Cucumber.class)
@CucumberOptions(plugin = {"pretty"})
public class RunCucumberTest {
}

18) 1) Yes

19) 1) monochrome=true

import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
import org.junit.runner.RunWith;
 
@RunWith(Cucumber.class)
@CucumberOptions(monochrome=true)
public class RunCucumberTest {
}

20) 4) Run the scenario tagged with both smoke and regression

21) 4) All of the above

22) 1) Yes

23) 1) Yes

24) 1) Yes

25) 1) Yes. It is the Test Runner using TestNG.

May 31, 2024January 8, 2025 vibssingh

Cucumber Multiple-Choice Questions with Answers – MCQ2

Last Updated On

January 8, 2025

HOME

Are you familiar with Cucumber and want to test your understanding? This post has 25 useful Cucumber multiple-choice questions (quiz) to self-test your knowledge of Cucumber.

Cucumber Multiple Choice Questions – MCQ1

1. What is the purpose of Cucumber?

Select the best answer

To write tests in a human-readable format
To automate software testing
To improve software quality
All of the above

Answer

2. The __ programming language is used for the creation of these executable test scripts.

Choose one option

Gherkin
Gerkin
Gerin
Gerkn

Answer

3. Which of the following is an example of a Cucumber data table?

Choose one option

A table of test results
A table of test requirements
A table of input parameters
A table of test scenarios

Answer

4. Which of the following is an example of a Cucumber scenario outline?

Choose one option

Scenario: Login with valid credentials
Scenario Outline: Login with invalid credentials
scenario: Add item to shopping cart
scenario outline: Search for product

Answer

5. Which option check if all the steps have the step definition before execute?

Choose one option

monochrome
dryRun
strict
validate

Answer

6. Which of the following is a Cucumber option that can be used to specify the path to the feature files?

–tags
–strict
–format
–glue

Answer

7. Which of the following is a Cucumber option that can be used to enforce stricter syntax checking?

–tags
–strict
–name
–format

Answer

8. Which of the following is a Maven command that can be used to run scenarios that have been tagged as “regression” tests in Cucumber?

mvn test -Dcucumber.filter.tags="@regression"

mvn test -Dcucumber.tags="@regression"

mvn test -cucumber.filter.tags="@regression"

Answer

9. Which of the following is a Maven command that can be used to run a feature file in Cucumber

Choose one option

mvn test -Dcucumber.features="src/test/resources/features/Login.feature"

mvn test -Dcucumber.features="src/test/resources/features/Login"

mvn test -cucumber.feature="src/test/resources/features/Login.feature"

Answer

10. Which of the following is a Maven command that can be used to pass the glue code in Cucumber

Choose one option

mvn test -Dcucumber.gluecode="com.example.stepdefinitions"

mvn test -Dcucumber.glue_code="com.example.stepdefinitions"

mvn test -Dcucumber.glue="com.example.stepdefinitions"

Answer

11. Which of the following is a Maven command that can be used to generate Cucumber Report

Choose one option

mvn test -Dcucumber.plugin="html:target/cucumber-reports/cucumberReport.html"

mvn test -cucumber.plugin="html:target/cucumber-reports/cucumberReport.html"

mvn test -Dcucumber.plugins="target/cucumber-reports/cucumberReport"

Answer

12. Tags that are placed above a Scenario Outline will be inherited by Examples

Choose one option

True
False

Answer

13. Below is a valid example of Scenario Outline

Scenario Outline: Login to Home Page
  Given user is logged in
  When user clicks <link>
  Then user will be logged out

  @mobile
  Examples:
    | link                   |
    |  mobile logout |

  @desktop
  Examples:
    | link                    |
    | desktop logout |

Choose one option

True
False

Answer

14. Tags that are placed above a __ will be inherited by Scenario, Scenario Outline, or Examples.

Choose one option

Scenario Outline
Feature
Background
None

Answer

15. Select a valid example of running multiple parameters in the Cucumber

mvn test -Dcucumber.features="src/test/resources/LoginPage.feature" -Dcucumber.filter.tags="@ValidCredentials"

mvn test -Dcucumber.features="src/test/resources/LoginPage.feature" and -Dcucumber.filter.tags="@ValidCredentials"

mvn test -Dcucumber.features="src/test/resources/LoginPage.feature" or -Dcucumber.filter.tags="@ValidCredentials"

Answer

16. The __ can be used to provide additional configuration to the runner.

Choose one option

@CucumberOptions
@Cucumber
@Options
None

Answer

17. Which of the following is an example of Cucumber TestRunner?

import io.cucumber.junit.CucumberOptions;
import org.junit.runner.RunWith;

@RunWith
@CucumberOptions(plugin = {"pretty"})
public class RunCucumberTest {
}

import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;

@CucumberOptions(plugin = {"pretty"})
public class RunCucumberTest {
}

import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
import org.junit.runner.RunWith;

@RunWith(Cucumber.class)
@CucumberOptions(plugin = {"pretty"})
public class RunCucumberTest {
}

Answer

18. Can we run the tests in Cucumber parallelly?

Choose one option

Answer

19. How to print console output from Cucumber in a readable format?

import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
import org.junit.runner.RunWith;

@RunWith(Cucumber.class)
@CucumberOptions(monochrome=true)
public class RunCucumberTest {
}

import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
import org.junit.runner.RunWith;

@RunWith(Cucumber.class)
@CucumberOptions(plugin = {"pretty", "summary"})
public class RunCucumberTest {
}

import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
import org.junit.runner.RunWith;

@RunWith(Cucumber.class)
@CucumberOptions(snippets = SnippetType.CAMELCASE)
public class RunCucumberTest {
}

import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
import org.junit.runner.RunWith;

@RunWith(Cucumber.class)
@CucumberOptions(dryRun=true)
public class RunCucumberTest {
}

Answer

20. Which of the following is an example of a Cucumber plugin?

mvn test -Dcucumber.tags="@smoke and @regression"

Choose one option

Run all the scenarios in the feature file
Run the scenario tagged with smoke
Run the scenario tagged with regression
Run the scenario tagged with both smoke and regression

Answer

21. What are the different types of hooks

Choose one option

Scenario Hook
Step Hook
Conditional Hook
All of the above

Answer

22. Can we integrate Cucumber with TestNG?

Choose one option

Answer

23. Can we integrate Cucumber with JUnit?

Choose one option

Answer

24. Can we execute the Cucumber tests in CI/CD like GitHub pipeline, Jenkins and so

Choose one option

Answer

25. Below is the valid syntax for the Test Runner class?

import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;
 
@CucumberOptions(features = "src/test/resources/Features")
 public class CucumberRunnerTests extends AbstractTestNGCucumberTests {
 
}

Choose one option

Answer

====================================================================

May 29, 2024April 24, 2025 vibssingh

API Automation with REST Assured, Cucumber and TestNG

Last Updated On

April 24, 2025

HOME

Cucumber is not an API automation tool, but it works well with other API automation tools.

There are 2 most commonly used Automation Tools for JVM to test API – Rest-Assured and Karate. In this tutorial, I will use RestAssured with Cucumber and TestNG for API Testing.

Table of Contents

What is Rest Assured?

REST Assured is a Java library that provides a domain-specific language (DSL) for writing powerful, maintainable tests for RESTful APIs. REST Assured can be used easily in combination with existing unit testing frameworks, such as JUnit and TestNG. Rest assured, no matter how complex the JSON structures are, Rest Assured has methods to retrieve data from almost every part of the request and response.

What is Cucumber?

Cucumber is one such open-source tool, which supports Behaviour Driven Development(BDD). In simple words, Cucumber can be defined as a testing framework, driven by plain English. It serves as documentation, automated tests, and development aid – all in one.

Each scenario is a set of steps that the Cucumber must complete. Cucumber validates the software’s compliance with the specification and generates a report indicating success or failure for each scenario.

The cucumber must adhere to some basic syntax rules known as Gherkin to comprehend the scenarios.

In this tutorial, I will explain creating a framework for the testing of Rest API in Cucumber BDD.

Dependency List

Cucumber – 7.18.0
Java 17
TestNG – 7.10.2
Maven – 3.9.6
Rest Assured – 5.4.0
Maven Compiler – 3.13.0
Maven Surefire – 3.2.5

Project Structure

Implementation Steps

Step 1 – Download and Install Java

Cucumber and Rest-Assured need Java to be installed on the system to run the tests. Click here to learn How to install Java.

Step 2 – Download and setup Eclipse IDE on the system

The Eclipse IDE (integrated development environment) provides strong support for Java developers. Click here to learn How to install Eclipse.

Step 3 – Setup Maven

To build a test framework, we need to add several dependencies to the project. Click here to learn How to install Maven.

Step 4 – Create a new Maven Project

File -> New Project-> Maven-> Maven project-> Next -> Enter Group ID & Artifact ID -> Finish

Click here to learn How to create a Maven project

Step 5 – Install the Cucumber Eclipse plugin for the Eclipse project (Eclipse Only)

The Cucumber plugin is an Eclipse plugin that allows Eclipse to understand the Gherkin syntax. Cucumber Eclipse Plugin highlights the keywords present in the Feature File. To install Cucumber Eclipse Plugin, please refer to this tutorial – How to install Cucumber Eclipse Plugin.

Step 6 – Create source folder src/test/resources

Create source folder src/test/resources to create test scenarios in the Feature file.

A new Maven Project is created with 2 folders – src/main/java and src/test/java. To create test scenarios, we need a new source folder called – src/test/resources. To create this folder, right-click on your Maven project ->select New ->Java, and then Source Folder.

Mention the source folder name as src/test/resources and click the Next button. This will create a source folder under your new Maven project.

Step 7 – Add dependencies to the project

Add Rest Assured, Cucumber, and TestNG dependencies in the pom.xml/build.gradle.
REST Assured includes JsonPath and XmlPath as transitive dependencies.

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <rest.assured.version>5.4.0</rest.assured.version>
    <cucumber.version>7.18.0</cucumber.version>
    <testng.version>7.10.2</testng.version>
    <maven.compiler.plugin.version>3.13.0</maven.compiler.plugin.version>
    <maven.compiler.source>17</maven.compiler.source>
    <maven.compiler.target>17</maven.compiler.target>
    <maven.surefire.plugin.version>3.2.5</maven.surefire.plugin.version>
  </properties>

  <dependencies>

    <!-- Rest Assured -->
    <dependency>
      <groupId>io.rest-assured</groupId>
      <artifactId>rest-assured</artifactId>
      <version>${rest.assured.version}</version>
      <scope>test</scope>
    </dependency>

    <!-- Cucumber with TestNG -->
    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-testng</artifactId>
      <version>${cucumber.version}</version>
    </dependency>

    <!-- Cucumber with Java -->
    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-java</artifactId>
      <version>${cucumber.version}</version>
    </dependency>

    <!-- TestNG -->
    <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>${testng.version}</version>
      <scope>test</scope>
    </dependency>

  </dependencies>

Step 8 – Add Maven Compiler Plugin and Surefire Plugin

The compiler plugin is used to compile the source code of a Maven project. This plugin has two goals, which are already bound to specific phases of the default lifecycle:

compile – compile main source files
testCompile – compile test source files

 <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>${maven.compiler.plugin.version}</version>
        <configuration>
          <source>${maven.compiler.source}</source>
          <target>${maven.compiler.target}</target>
        </configuration>
      </plugin>

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>${maven.surefire.plugin.version}</version>
        <configuration>
          <suiteXmlFiles>
            <suiteXmlFile>testng.xml</suiteXmlFile>
          </suiteXmlFiles>
        </configuration>
      </plugin>
    </plugins>
  </build>

The complete POM.xml will look like something below:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.example</groupId>
  <artifactId>RestAPI_Cucumber_TestNG_Demo</artifactId>
  <version>1.0-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>RestAPI_Cucumber_TestNG_Demo</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <rest.assured.version>5.4.0</rest.assured.version>
    <cucumber.version>7.18.0</cucumber.version>
    <testng.version>7.10.2</testng.version>
    <maven.compiler.plugin.version>3.13.0</maven.compiler.plugin.version>
    <maven.compiler.source>17</maven.compiler.source>
    <maven.compiler.target>17</maven.compiler.target>
    <maven.surefire.plugin.version>3.2.5</maven.surefire.plugin.version>
  </properties>

  <dependencies>

    <!-- Rest Assured -->
    <dependency>
      <groupId>io.rest-assured</groupId>
      <artifactId>rest-assured</artifactId>
      <version>${rest.assured.version}</version>
      <scope>test</scope>
    </dependency>

    <!-- Cucumber with TestNG -->
    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-testng</artifactId>
      <version>${cucumber.version}</version>
    </dependency>

    <!-- Cucumber with Java -->
    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-java</artifactId>
      <version>${cucumber.version}</version>
    </dependency>

    <!-- TestNG -->
    <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>${testng.version}</version>
      <scope>test</scope>
    </dependency>

  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>${maven.compiler.plugin.version}</version>
        <configuration>
          <source>${maven.compiler.source}</source>
          <target>${maven.compiler.target}</target>
        </configuration>
      </plugin>

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>${maven.surefire.plugin.version}</version>
        <configuration>
          <suiteXmlFiles>
            <suiteXmlFile>testng.xml</suiteXmlFile>
          </suiteXmlFiles>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

Step 9 – Create a feature file under src/test/resources

Create a folder with name features. Now, create the feature file in this folder. The feature file should be saved with the extension .feature. This feature file contains the test scenarios created to test the application. The Test Scenarios are written in Gherkins language in the format of Given, When, Then, And, But.

Below is an example of a Test Scenario where we are using the GET method to get the information from the API.

Feature: Validation of get method

  @GetUserDetails
  Scenario Outline: Send a valid Request to get user details

    Given I send a request to the URL to get user details
    Then the response will return status <statusCode> and id <id> and email "<employee_email>" and first name "<employee_firstname>" and last name "<employee_lastname>"

    Examples:
    | statusCode | id  | employee_email          | employee_firstname | employee_lastname |
    | 200        | 2   | janet.weaver@reqres.in  | Janet              | Weaver            |

Step 10 – Create the Step Definition class or Glue Code

StepDefinition acts as an intermediate to your runner and feature file. It stores the mapping between each step of the scenario in the Feature file. So when you run the scenario, it will scan the step definition file to check the matched glue or test code.

import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.restassured.http.ContentType;
import io.restassured.response.ValidatableResponse;

import static io.restassured.RestAssured.given;
import static org.hamcrest.Matchers.equalTo;

public class APIDemoDefinitions {

    private ValidatableResponse validatableResponse;
    private String endpoint = "https://reqres.in/api/users/2";

    @Given("I send a request to the URL to get user details")
    public void sendRequest(){
        validatableResponse = given().contentType(ContentType.JSON)
                .when().get(endpoint).then();

        System.out.println("Response :"+validatableResponse.extract().asPrettyString());
    }

    @Then("the response will return status {int} and id {int} and email {string} and first name {string} and last name {string}")
    public void verifyStatus(int expectedStatusCode, int expectedId, String expectedEmail, String expectedFirstName, String expectedLastName){

        validatableResponse.assertThat().statusCode(expectedStatusCode).body("data.id",equalTo(expectedId)).and()
                .body("data.email",equalTo(expectedEmail)).body("data.first_name",equalTo(expectedFirstName))
                .body("data.last_name",equalTo(expectedLastName));

    }
}

To use REST assured effectively it’s recommended to statically import methods from the following classes:

io.restassured.RestAssured.*
io.restassured.matcher.RestAssuredMatchers.*
org.hamcrest.Matchers.*

Step 11 – Create a TestNG Cucumber Runner class

A runner will help us run the feature file and act as an interlink between the feature file and the StepDefinition Class.

import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;

@CucumberOptions(tags = "", features = {"src/test/resources/features"}, glue = {"com.example.stepdefinitions"},
        plugin = {})
public class CucumberRunnerTests extends AbstractTestNGCucumberTests {

}

Note:- The name of the Runner class should end with Test otherwise we can’t run the tests using Command-Line.

Step 12 – Create a testng.xml file

Create a testng.xml at the root of the project.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
    <test  name="Rest Assured, Cucumber with TestNG Test">
        <classes>
            <class name="com.example.runner.CucumberRunnerTests"/>
        </classes>
    </test> <!-- Test -->
</suite> <!-- Suite -->

Step 13 – Run the tests from TestNG

You can execute the test script by right-clicking on TestRunner class -> Run As TestNG (Eclipse).

You can execute the test script by right-clicking on TestRunner class -> Run CucumberRunnerTests (IntelliJ).

Step 14 – Run the tests from the testng.xml

Right-click on the testng.xml. Click on Run’...\testng.xml’.

Step 15 – TestNG Report Generation

TestNG generates various types of reports under the test-output or target folder like emailable-report.html, index.html, and testng-results.xml.

We are interested in the ‘emailable-report.html’ report. Open “emailable-report.html“, as this is an HTML report, and open it with the browser. The below image shows emailable-report.html.

emailable-report.html

Index.html

TestNG also produces “index.html” report, and it resides under the test-output folder. The below image shows the index.html report.

Step 16 – Run the tests from the Command Line

Run the below command in the command prompt to run the tests and to get the test execution report.

mvn clean test

The output of the above program is

Step 17 – Cucumber Report Generation

To get Cucumber Test Reports, add cucumber.properties under src/test/resources and add the below instruction in the file.

cucumber.publish.enabled=true

The complete code can be found here – vibssingh/RestAPI_Cucumber_TestNG_Demo.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!