The previous tutorial explained the generation of Allure Report with Cucumber5, Selenium and JUnit4 in a Maven project. In this tutorial, I will explain the steps to create an Allure Report with Cucumber7, Selenium, and JUnit5 in a Maven project.

Pre Requisite:

Java 11 or above installed
Eclipse or IntelliJ IDE installed
Maven Installed
Environment variables JAVA_HOME and ALLURE_HOME are correctly configured

In this tutorial, I’ll create an Allure Report for the testing of web applications using Cucumber7, and Selenium 4 with JUnit5.

This framework consists of:-

Cucumber Java – 7.6.0
Cucumber JUnit Platform Engine – 7.6.0
Java 11
Maven – 3.8.1
Selenium – 4.3.0
Allure JUnit5 – 2.21.0
AspectJ Weaver – 1.9.7

Implementation Steps

Add Cucumber, Selenium, JUnit5, and Allure-JUnit5 dependencies in pom.xml
Create Pages and Test Code for the pages
Execute the Tests
Generate the Allure Report

Project Structure

Step 1 – Add Cucumber, Selenium, JUnit5, and Allure-JUnit5 dependencies in pom.xml

The Cucumber, Selenium, JUnit5, WebDriverMananger, and Allure-JUnit5 dependencies are added to the pom.xml.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
 
    <groupId>com.example</groupId>
    <artifactId>Cucumber7Junit5</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    <packaging>jar</packaging>
 
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <cucumber.version>7.6.0</cucumber.version>
        <selenium.version>4.3.0</selenium.version>
        <webdrivermanager.version>5.2.1</webdrivermanager.version>
        <junit.jupiter.version>5.9.0</junit.jupiter.version>
        <apache.common.version>2.4</apache.common.version>
        <projectlombok.version>1.18.24</projectlombok.version>
        <maven.compiler.plugin.version>3.10.1</maven.compiler.plugin.version>
        <maven.surefire.plugin.version>3.0.0-M7</maven.surefire.plugin.version>
        <maven.compiler.source.version>11</maven.compiler.source.version>
        <maven.compiler.target.version>11</maven.compiler.target.version>
		<allure.junit5.version>2.21.0</allure.junit5.version>
		<allure.version>2.19.0</allure.version>
		<allure.maven.version>2.11.2</allure.maven.version>
		<aspectj.version>1.9.9.1</aspectj.version>
    </properties>
 
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>io.cucumber</groupId>
                <artifactId>cucumber-bom</artifactId>
                <version>${cucumber.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
            <dependency>
                <groupId>org.junit</groupId>
                <artifactId>junit-bom</artifactId>
                <version>${junit.jupiter.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>
 
    <dependencies>
 
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-java</artifactId>
            <scope>test</scope>
        </dependency>
 
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-junit-platform-engine</artifactId>
            <scope>test</scope>
        </dependency>
 
        <!-- JUnit Platform -->
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-suite</artifactId>
            <scope>test</scope>
        </dependency>
 
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <scope>test</scope>
        </dependency>
 
        <!-- Selenium -->
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>${selenium.version}</version>
        </dependency>
 
        <!-- Web Driver Manager -->
        <dependency>
            <groupId>io.github.bonigarcia</groupId>
            <artifactId>webdrivermanager</artifactId>
            <version>${webdrivermanager.version}</version>
        </dependency>
 
        <!-- Apache Common -->
        <dependency>
            <groupId>org.apache.directory.studio</groupId>
            <artifactId>org.apache.commons.io</artifactId>
            <version>${apache.common.version}</version>
        </dependency>
 
 <!--Allure Reporting Dependency-->
		<dependency>
			<groupId>io.qameta.allure</groupId>
			<artifactId>allure-junit5</artifactId>
			<version>${allure.junit5.version}</version>
			<scope>test</scope>
		</dependency>
		
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>${projectlombok.version}</version>
            <scope>provided</scope>
        </dependency>
		
    </dependencies>
 
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${maven.compiler.plugin.version}</version>
                <configuration>
                    <source>${maven.compiler.source.version}</source>
                    <target>${maven.compiler.target.version}</target>
                </configuration>
            </plugin>
           
           <plugin>

				<artifactId>maven-surefire-plugin</artifactId>
				<version>${maven.surefire.plugin.version}</version>
				<configuration>
					<properties>
						<property>
							<name>listener</name>
							<value>io.qameta.allure.junit5.AllureJunit5</value>
						</property>
					</properties>
					<argLine>
                            -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                            -Dcucumber.options="--plugin io.qameta.allure.cucumber7jvm.AllureCucumber7Jvm"
                        </argLine>
					<systemProperties>
						<property>
							<name>allure.results.directory</name>
							<value>${project.build.directory}/allure-results</value>
						</property>
						<property>
							<name>junit.jupiter.extensions.autodetection.enabled</name>
							<value>true</value>
						</property>
					</systemProperties>
				</configuration>
				<dependencies>
					<dependency>
						<groupId>org.aspectj</groupId>
						<artifactId>aspectjweaver</artifactId>
						<version>${aspectj.version}</version>
					</dependency>
					<dependency>
						<groupId>io.cucumber</groupId>
						<artifactId>cucumber-junit-platform-engine</artifactId>
						<version>${cucumber.version}</version>
					</dependency>
				</dependencies>
			</plugin>
			
            <plugin>
				<groupId>io.qameta.allure</groupId>
				<artifactId>allure-maven</artifactId>
				<version>${allure.maven.version}</version>
				<configuration>
					<reportVersion>2.4.1</reportVersion>
				</configuration>
			</plugin>
		</plugins>
	</build>
</project>

Step 2 – Create Locator and Action classes and Step Definition corresponding to the feature file and Test Runner Class

There is another tutorial that explains the project structure as well as the feature file and corresponding Step Definitions, please refer to this tutorial – Integration of Cucumber7 with Selenium and JUnit5.

Step 3 – Execute the Tests

Use the below command to run the tests

mvn clean test

The output of the above program is

Step 4 – Generate the Allure Report

Once the test execution is finished, a folder named allure-results will be generated in the target folder.

To generate Allure Report, first, go to the target folder.

cd target

Now, use the below command to generate the Allure Report

allure serve

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

The overview page hosts several default widgets representing the basic characteristics of your project and test environment.

Statistics – overall report statistics.
Launches – if this report represents several test launches, statistics per launch will be shown here.
Behaviors – information on results aggregated according to stories and features.
Executors – information on test executors that were used to run the tests.
History Trend – if tests accumulated some historical data, its trend will be calculated and shown on the graph.
Environment – information on the test environment.

Categories in Allure Report

The categories tab gives you a way to create custom defect classifications to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: status breakdown or severity and duration diagrams.

Timeline in Allure Report

The timeline tab visualizes retrospective test execution, allure adaptors collect precise timings of tests, and here on this tab, they are arranged accordingly to their sequential or parallel timing structure.

Behaviors of Allure Report

This tab groups test results according to Epic, Feature, Story, Test Severity, Test Description, Test Steps, and so on.

Packages in Allure Report

The packages tab represents a tree-like layout of test results, grouped by different packages.

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

December 2, 2022March 28, 2023 vibssingh

Integration of Allure Report with Jenkins

HOME

In this tutorial, we are going to learn how we can integrate Allure Report with Jenkins.

Before proceeding, I strongly advise you to read this blog to learn How to Setup Allure Report with Selenium TestNG.

Implementation Steps

Download the Allure Jenkins Plugin
Configure Allure
Create a new Maven Jenkins job

Download Allure Jenkins Plugin

To generate Allure Report in Jenkins, we need to download Allure Plugin. Please refer to this tutorial to install the plugin – How to install Plugins in Jenkins

Configure Allure

Go back to the Manage Jenkins link as shown below:

When we click on the “Manage Jenkins” link, we are redirected to the Manage Jenkins page, where we can see various types of options, including the “Global Tool Configuration” option.

We need to set the Allure Commandline in Jenkins as shown below.

Click on the Allure Command line installations button. By default, “Install Automatically” will be checked, so since we are going to use the Allure installed on our local machine, “Install automatically” will install the latest version of Allure.

Provide the Name as ALLURE_HOME because that is what is currently installed on my machine, and also provide the path of Allure in the ALLURE_HOME textbox.

Click on the Apply and Save buttons.

Create a new Maven Jenkins job

Step 1: Create a new project using the Maven project plugin

Give the Name of the project.
Click on the Maven project.
Click on the OK button.

In the General section, enter the project description in the Description box.

Select Source Code Management as None if the project is locally present on the machine

Step 2: Build Management

Go to the Build section of the new job.

In the Root POM textbox, enter pom.xml
In the Goals and options section, enter “clean test”

Click on the Advanced button.

Step 3: Select a custom workspace

Mention the full path of the project in the directory.

Step 4: Select “Allure Reports” from “Post Build Actions“

Scroll down to ‘Post Build Actions’ and click on the ‘Add Post Build Actions’ drop-down list. Select “Allure Report“.

Enter the Result Path as “allure-results” and click on the “Save” button.

Click on the Apply and Save buttons.

We have created a new Maven project “AllureReportWithSelenium_Demo” with the configuration to run the Selenium with TestNG Tests and also to generate Allure Report after execution using Jenkins.

Step 5: Execute the tests

Let’s execute it now by clicking on the “Build Now” button.

Right-click on Build Number (here in my case it is #1).

Click on Console Output to see the result.

Step 6: View the Allure Report

Once the execution is completed, we could see a link to view the ‘Allure Report’.

Click on the Allure Report. It displays the summary of the tests.

There is another way to create Allure Report in Jenkins, which is by using the Jenkins pipeline. To know more about this, please refer to this tutorial – How to create Jenkins pipeline for Allure Report.

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

Additional Tutorials

*Integration Of Jenkins With Selenium WebDriver*
*How to install Maven Plugin in Jenkins*
*Integrate Gradle project with Jenkins*
*Jenkins GitLab Integration*
*How to generate HTML Reports in Jenkins*
*Integration of Cucumber Report with TestNG in Jenkins*
*Serenity with Jenkins*

November 10, 2022March 13, 2023 vibssingh

Allure Reports

HOME

Allure Framework is a lightweight, flexible multi-language test report tool that not only displays a very concise representation of what has been tested in a neat web report form, but also allows everyone involved in the development process to extract the most useful information from everyday test execution.

Allure Report for Maven Projects

Chapter 1 What is Allure Report?
Chapter 2 Integration of Allure Report with Selenium and JUnit4
Chapter 3 Integration of Allure Report with Selenium and JUnit5
Chapter 4 Integration of Allure Report with Selenium and TestNG
Chapter 5 *Allure Report with Cucumber5, Selenium and JUnit4*
Chapter 6 *Allure Report with Cucumber5, Selenium and TestNG*
Chapter 7 Integration of Allure Report with Rest Assured and JUnit4
Chapter 8 Integration of Allure Report with Rest Assured and TestNG
Chapter 9 Allure Report for Cucumber7, Selenium, and JUnit5
Chapter 10 Integration of Allure Report with Jenkins

Allure Report for Gradle Projects

Chapter 1 Gradle – Allure Report for Selenium and TestNG
Chapter 2 Gradle – Allure Report for Selenium and JUnit4
Chapter 3 Gradle – Allure Report for Cucumber, Selenium and TestNG

November 2, 2022March 25, 2023 vibssingh

Gradle – Allure Report for Selenium and JUnit5

HOME

The previous tutorial explained the Integration of Selenium and JUnit5 in a Gradle project. In this tutorial, I will explain the steps to create an Allure Report with Selenium and JUnit5 in a Gradle project.

Pre Requisite:

Java 8 or above installed
Eclipse or IntelliJ IDE installed
Gradle Installed
Environment variables JAVA_HOME, ALLURE_HOME and GRADLE_HOME correctly configured

In this tutorial, I’ll create a BDD Framework for creating an Allure Report for Selenium WebDriver with JUnit5. This framework consists of:-

Java 11
JUnit– 5.8.2
Gradle – 7.3.3
Selenium – 4.3.0
Allure Report – 2.19
AspectJ Weaver – 1.9.7

Project Structure

Implementation Steps

Add Selenium, JUnit5, and Allure-JUnit5 dependencies in build.gradle
Create Pages and Test Code for the pages
Execute the Tests
Generate Allure Report

There is another tutorial that explains the steps to create a Gradle Project with Selenium and JUnit5. Please refer to this tutorial – Gradle – Integration of Selenium and JUnit5.

Step 1 – Add Selenium, JUnit5, and Allure-JUnit5 dependencies in build.gradle

/*
 * This file was generated by the Gradle 'init' task.
 *
 */

plugins {
    // Apply the application plugin to add support for building a CLI application in Java.
    id 'application'
    id 'io.qameta.allure' version '2.11.0'
}

repositories {
    // Use Maven Central for resolving dependencies.
    mavenCentral()
}

java {
    sourceCompatibility = 11
    targetCompatibility = 11
}

dependencies {
    // Use JUnit Jupiter for testing.
    testImplementation 'org.junit.jupiter:junit-jupiter:5.8.2'
    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.8.2'
    
     // Allure 
     implementation 'io.qameta.allure:allure-junit5:2.19.0'
     runtimeOnly 'org.aspectj:aspectjweaver:1.9.7'

    // This dependency is used by the application.
    implementation 'com.google.guava:guava:30.1.1-jre'
    implementation 'org.seleniumhq.selenium:selenium-java:4.4.0'
    implementation 'io.github.bonigarcia:webdrivermanager:5.3.0'
}

application {
    // Define the main class for the application.
    mainClass = 'com.example.App'
}

tasks.named('test') {
    // Use JUnit Platform for unit tests.
    useJUnitPlatform()  {
    }
}

Step 2 – Create Pages and Test Code for the pages

As mentioned above, there is another tutorial that explains the project structure as well as the feature file and corresponding Step Definitions, please refer to this tutorial – Gradle Project with Selenium and JUnit5.

Below is the sample test code. I have added features of the allure report like @Severity, @Description.

I have used Parameterized Tests here, to know more about that, please refer to this tutorial – How to run parameterized Selenium test using JUnit5.

import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Disabled;
import io.qameta.allure.Description;
import io.qameta.allure.Severity;
import io.qameta.allure.SeverityLevel;

public class LoginPageTests extends BaseTests{
	 
	@Severity(SeverityLevel.NORMAL)
    @ParameterizedTest
    @CsvSource({
            "admin$$,admin123",
            "Admin,admin123!!",
            "admin123,Admin",
            "%%%%%,$$$$$$"})
	@Description("Test Description : Login Test with invalid credentials")
    public void invalidCredentials(String username, String password) {
   
	    LoginPage objLoginPage = new LoginPage(driver);
    	objLoginPage.login(username, password);
    	 
    	// Verify Error Message
    	 assertEquals("Invalid credentials",objLoginPage.getErrorMessage());
    
    }
    
	@Severity(SeverityLevel.BLOCKER)
    @Test
	@Description("Test Description : Login Test with valid credentials")
    public void validLogin() {
   
	    LoginPage objLoginPage = new LoginPage(driver);
    	objLoginPage.login("Admin", "admin123");
    	 
    	HomePage objHomePage = new HomePage(driver);
    	
    	// Verify Home Page
    	 assertEquals("Employee Information",objHomePage.getHomePageText());  
    }
    
	@Severity(SeverityLevel.NORMAL)
    @Test 
	@Description("Test Description : Login Test with missing username - Failed Test")
    public void missingUsername() {
   
	    LoginPage objLoginPage = new LoginPage(driver);
    	objLoginPage.login("", "admin123");
    	     	
    	// Verify Error Message
   	     assertEquals("Invalid credentials",objLoginPage.getMissingUsernameText());
   	        
    }
	
	@Severity(SeverityLevel.NORMAL)
    @Test @Disabled
	@Description("Test Description : Login Test with missing password")
    public void missingPassword() {
  
	    LoginPage objLoginPage = new LoginPage(driver);
    	objLoginPage.login("admin", "");
    	    	
    	// Verify Error Message
   	     assertEquals("Invalid credentials",objLoginPage.getMissingPasswordText());
    
    }      
}

Below is the BaseTest class, where I have shown the use of @Step of Allure Report.

import java.time.Duration;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import io.github.bonigarcia.wdm.WebDriverManager;
import io.qameta.allure.Step;

public class BaseTests {
	
	public WebDriver driver;
	public final static int TIMEOUT = 10;    
 
	@BeforeEach
    @Step("Start the application")
    public void setup() {
    	WebDriverManager.chromedriver().setup();
	    driver = new ChromeDriver();
	    driver.manage().window().maximize();
	    driver.get("https://opensource-demo.orangehrmlive.com/");	    
	    driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(TIMEOUT));

    }
   
    @AfterEach
    @Step("Stop the application")
    public void tearDown() {
        driver.quit();
    }
    
}

Step 3 – Execute the Tests

Go to the app project and run the tests, using the below command

gradle clean test

The output of the test execution is

Step 4 – Generate the Allure Report

Once the test execution is finished, a folder named allure-results will be generated in the build folder.

Note:- Make sure that you move to folder app, because the build folder is present in the app folder.

To generate an Allure Report, use the below command.

allure serve build/allure-results

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

The overview page hosts several default widgets representing the basic characteristics of your project and test environment.

Statistics – overall report statistics.
Launches – if this report represents several test launches, statistics per launch will be shown here.
Behaviors – information on results aggregated according to stories and features.
Executors – information on test executors that were used to run the tests.
History Trend – if tests accumulated some historical data, its trend will be calculated and shown on the graph.
Environment – information on the test environment.

Categories in Allure Report

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes, can be found.

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: status breakdown or severity and duration diagrams.

Timeline in Allure Report

Behaviors of Allure Report

This tab groups test results according to Epic, Feature, Story, Test Severity, Test Description, Test Steps, and so on.

Packages in Allure Report

The packages tab represents a tree-like layout of test results, grouped by different packages.

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

October 17, 2022May 11, 2023 vibssingh

Gradle – Allure Report for Cucumber7, Selenium and JUnit4

HOME

The previous tutorial explained the generation of Allure Report with Cucumber5, Selenium and JUnit4 in a Maven project. In this tutorial, I will explain the steps to create an Allure Report with Cucumber, Selenium, and JUnit4 in a Gradle project.

Prerequisite:

Java 8 or above installed
Eclipse or IntelliJ IDE installed
Gradle Installed
Environment variables JAVA_HOME, ALLURE_HOME and GRADLE_HOME are correctly configured

In this tutorial, I’ll create a BDD Framework for the testing of web applications using Cucumber7, and Selenium 4 with JUnit4. This framework consists of:-

Cucumber Java- 7.6.0
Cucumber JUnit4 – 7.6.0
Java 11
JUnit4 – 4.13.2
Gradle – 7.3.3
Selenium – 4.3.0
Allure Cucumber – 2.19.0
AspectJ Weaver – 1.9.7

Project Structure

Implementation Steps

Add Cucumber, Selenium, TestNG, and Allure-JUnit dependencies in build.gradle
Create Locator and Action classes and Step Definition corresponding to the feature file and Test Runner Class
Execute the Tests
Generate Allure Report

There is a tutorial that explains the Integration of Cucumber, Selenium, and JUnit4 in a Gradle project. Please refer to this tutorial – Gradle Project with Cucumber, Selenium, and JUnit4.

Step 1 – Add Cucumber, Selenium, TestNG, and Allure-JUnit4 dependencies in build.gradle

/*
 * This file was generated by the Gradle 'init' task.
 *
 */

plugins {
    // Apply the application plugin to add support for building a CLI application in Java.
    id 'application'
    id 'io.qameta.allure' version '2.11.0'
}

repositories {
    // Use Maven Central for resolving dependencies.
    mavenCentral()
}

java {
    sourceCompatibility = 11
    targetCompatibility = 11
}
 
dependencies {

    testImplementation 'io.cucumber:cucumber-java:7.6.0'
    testImplementation 'io.cucumber:cucumber-junit:7.6.0'
    
    // Use JUnit test framework.
    testImplementation 'junit:junit:4.13.2'
    
    // Allure 
    implementation 'io.qameta.allure:allure-cucumber7-jvm:2.19.0'
    runtimeOnly 'org.aspectj:aspectjweaver:1.9.7'  

    // This dependency is used by the application.
    implementation 'com.google.guava:guava:30.1.1-jre'
    implementation 'org.seleniumhq.selenium:selenium-java:4.4.0'
    implementation 'io.github.bonigarcia:webdrivermanager:5.3.0'
}

application {
    // Define the main class for the application.
    mainClass = 'com.example.App'
}

configurations {
    cucumberRuntime {
        extendsFrom testImplementation
    }
}

task cucumber() {
    dependsOn assemble, testClasses
    doLast {
        javaexec {
        
           systemProperty("allure.results.directory", "build/allure-results")  
            main = "io.cucumber.core.cli.Main"
            classpath = configurations.cucumberRuntime + sourceSets.main.output + sourceSets.test.output
            args = ['--plugin', 'pretty',         
            '--glue', 'com.example.definitions', 'src/test/resources']
        }
    }
}

Step 2 – Create Locator and Action classes and Step Definition corresponding to the feature file and Test Runner Class

Step 3 – Execute the Tests

Go to the app project and run the tests, using the below command

gradle cucumber

The output of the test execution is

Step 4 – Generate the Allure Report

Once the test execution is finished, a folder named allure-results will be generated in the build folder.

Note:- Make sure that you move to the folder app because the build folder is present in the app folder.

allure serve build/allure-results

Allure Report Dashboard

The overview page hosts several default widgets representing the basic characteristics of your project and test environment.

Statistics – overall report statistics.
Launches – if this report represents several test launches, statistics per launch will be shown here.
Behaviors – information on results aggregated according to stories and features.
Executors – information on test executors that were used to run the tests.
History Trend – if tests accumulated some historical data, its trend will be calculated and shown on the graph.
Environment – information on the test environment.

Categories in Allure Report

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: status breakdown or severity and duration diagrams.

Timeline in Allure Report

Behaviors of Allure Report

This tab groups test results according to Epic, Feature, Story, Test Severity, Test Description, Test Steps, and so on.

Packages in Allure Report

The packages tab represents a tree-like layout of test results, grouped by different packages.

BDD Features

The feature’s description appears in every scenario.

All scenario steps are automatically translated into allure steps.

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

October 5, 2022 vibssingh

Gradle – Allure Report for Cucumber, Selenium and TestNG

HOME

The previous tutorial explained generation of Allure Report with Cucumber5, Selenium and TestNG in a Maven project. In this tutorial, I will explain the steps to create an Allure Report with Cucumber, Selenium and TestNG in a Gradle project.

Pre Requisite:

Java 8 or above installed
Eclipse or IntelliJ IDE installed
Gradle Installed
Environment variables JAVA_HOME and GRADLE_HOME correctly configured

In this tutorial, I’ll create a BDD Framework for the testing of web applications using Cucumber, Selenium WebDriver with TestNG. This framework consists of:-

Cucumber Java- 7.6.0
Cucumber TestNG – 7.6.0
Java 11
TestNG – 7.6.0
Gradle – 7.5.1
Selenium – 4.3.0
AspectJ Weaver – 1.9.7

Project Structure

Implementation Steps

Add Cucumber, Selenium, TestNG, and Allure-TestNG dependencies in build.gradle
Create Locator and Action classes and Step Definition corresponding to the feature file and Test Runner Class
Execute the Tests
Generate Allure Report

There is a tutorial that explains the Integration of Cucumber, Selenium and TestNG in a Gradle project. Please refer to this tutorial – Gradle Project with Cucumber, Selenium and TestNG.

Step 1 – Add Cucumber, Selenium, TestNG, and Allure-TestNG dependencies in build.gradle

/*
 * This file was generated by the Gradle 'init' task.
 *
 */

plugins {
    // Apply the application plugin to add support for building a CLI application in Java.
    id 'application'
    id 'io.qameta.allure' version '2.11.0'
}

repositories {
    // Use Maven Central for resolving dependencies.
    mavenCentral()
}

java {
    sourceCompatibility = 11
    targetCompatibility = 11
}

dependencies {
   
    // Use TestNG framework, also requires calling test.useTestNG() below
    testImplementation 'io.cucumber:cucumber-java:7.6.0'
    testImplementation 'io.cucumber:cucumber-testng:7.6.0'
          
     // Allure 
     implementation 'io.qameta.allure:allure-cucumber7-jvm:2.19.0'
     runtimeOnly 'org.aspectj:aspectjweaver:1.9.7'
     
     //TestNG  
      testImplementation 'org.testng:testng:7.6.0'
      
     //Others  
     implementation 'com.google.guava:guava:31.0.1-jre'
     implementation 'org.seleniumhq.selenium:selenium-java:4.4.0'
     implementation 'io.github.bonigarcia:webdrivermanager:5.3.0'

}

application {
    // Define the main class for the application.
    mainClass = 'com.example.App'
}

tasks.named('test') {
    // Use TestNG for unit tests.
    useTestNG()
}

configurations {
    cucumberRuntime {
        extendsFrom testImplementation
    }
}

task cucumber() {
    dependsOn assemble, compileTestJava
    doLast {
        javaexec {
         systemProperty("allure.results.directory", "build/allure-results")
         
            main = "io.cucumber.core.cli.Main"
            classpath = configurations.cucumberRuntime + sourceSets.main.output + sourceSets.test.output
            args = ['--plugin', 'pretty',
            '--glue', 'com.example.definitions', 'src/test/resources']
        }
    }
}

Step 2 – Create Locator and Action classes and Step Definition corresponding to the feature file and Test Runner Class

Step 3 – Execute the Tests

Go to the app project and run the tests, using the below command

gradle cucumber

The output of the test execution is

Step 4 – Generate the Allure Report

Once the test execution is finished, a folder named allure-results will be generated in the build folder.

Note:- Make sure that you move to folder app, because build folder is present in app folder.

To generate Allure Report, use the below command

allure serve build/allure-results

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

The overview page hosts several default widgets representing basic characteristics of your project and test environment.

Statistics – overall report statistics.
Launches – if this report represents several test launches, statistics per launch will be shown here.
Behaviors – information on results aggregated according to stories and features.
Executors – information on test executors that were used to run the tests.
History Trend – if tests accumulated some historical data, its trend will be calculated and shown on the graph.
Environment – information on the test environment.

Categories in Allure Report

The categories tab gives you the way to create custom defects classification to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: status breakdown or severity and duration diagrams.

Timeline in Allure Report

Behaviors of Allure Report

This tab groups test results according to Epic, Feature, Story, Test Severity, Test Description, Test Steps, and so on.

Packages in Allure Report

The packages tab represents a tree-like layout of test results, grouped by different packages.

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

September 30, 2022March 16, 2023 vibssingh

Gradle – Allure Report for Selenium and JUnit4

HOME

The previous tutorial explained the generation of Allure Report for Selenium and JUnit4 in a Maven Project. This tutorial explains the generation of Allure Report for Selenium and JUnit4 in a Gradle project.

Pre-Requisite:

Java 11 installed
Gradle installed
Eclipse or IntelliJ installed
Allure installed
Environment variables JAVA_HOME , GRADLE_HOME and ALLURE_HOME correctly configured

This framework consists of:

Selenium – 4.4.0
Java 11
JUnit – 4.13.2
Gradle – 7.5.1
Allure Report – 2.11.0
Allure JUnit4 – 2.19.0

Project Structure

Implementation Steps

Add Selenium, JUnit4 and Allure-JUnit4 dependencies in build.gradle
Create Pages and Test Code for the pages
Execute the Tests through Command Line
Generate Allure Report

There is another tutorial that explain the steps to create a Gradle Project with JUnit4 – please refer to this tutorial – How to create Gradle project with Selenium and JUnit4

Step 1 – Add Selenium, JUnit4, and Allure-JUnit4 dependencies in build.gradle

plugins {
    // Apply the application plugin to add support for building a CLI application in Java.
    id 'application'
    id 'io.qameta.allure' version '2.11.0'
}

repositories {
    // Use Maven Central for resolving dependencies.
    mavenCentral()
}

java {
    sourceCompatibility = 11
    targetCompatibility = 11
}

dependencies {
    // Use JUnit test framework.
    testImplementation 'junit:junit:4.13.2'

    // This dependency is used by the application.
    implementation 'com.google.guava:guava:30.1.1-jre'
    implementation 'org.seleniumhq.selenium:selenium-java:4.4.0'
    implementation 'io.github.bonigarcia:webdrivermanager:5.3.0'
    implementation 'io.qameta.allure:allure-junit4:2.19.0'
}

application {
    // Define the main class for the application.
    mainClass = 'com.example.App'
}

test {
    useJUnit {   
    }

    testLogging {
        events "passed", "skipped", "failed"
        showStandardStreams = true
    }

    systemProperties System.properties
}

Step 2 – Create Pages and Test Code for the pages

Below is the sample project which uses Selenium and TestNG which is used to generate an Allure Report.

We have used PageFactory model to build the tests. I have created a package named pages and created the page classes in that folder. Page class contains the locators of each web element present on that particular page along with the methods of performing actions using these web elements.

This is the BaseClass that contains the PageFactory.initElements. The initElements is a static method of PageFactory class that is used to initialize all the web elements located by @FindBy annotation.

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.support.PageFactory;
 
public class BasePage { 
 
      public WebDriver driver;
 
      public BasePage(WebDriver driver) {
          this.driver = driver;
          PageFactory.initElements(driver,this);
    }
 
}

Below is the code for LoginPage and HomePage.

LoginPage

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;
 
public class LoginPage extends BasePage{
     
    public LoginPage(WebDriver driver) {
         super(driver);
          
    }
      
    @FindBy(name = "username")
    public WebElement userName;
   
    @FindBy(name = "password")
    public WebElement password;
      
    @FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/form/div[1]/div/span")
    public WebElement missingUsernameErrorMessage;
      
    @FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/form/div[1]/div/span")
    public WebElement missingPasswordErrorMessage;
   
    @FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/form/div[3]/button")
    public WebElement login;
   
    @FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/div/div[1]/div[1]/p")
    public  WebElement errorMessage;
         
   // Get the error message when password is blank
    public String getMissingPasswordText() {
        return missingPasswordErrorMessage.getText();
    }
 
   // Get the error message when username is blank
   public String getMissingUsernameText() {
        return missingUsernameErrorMessage.getText();
    }
        
    // Get the Error Message
    public String getErrorMessage() {
        return errorMessage.getText();
    }
      
    public void login(String strUserName, String strPassword) {
   
        userName.sendKeys(strUserName);
        password.sendKeys(strPassword);
        login.click();
    }
  
}

HomePage

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;
 
public class HomePage extends BasePage {
 
    public HomePage(WebDriver driver) {
        super(driver);
 
    }
 
     @FindBy(xpath = "//*[@id='app']/div[1]/div[2]/div[2]/div/div[1]/div[1]/div[1]/h5")
      public  WebElement homePageUserName;
 
      // Get the User name from Home Page
        public String getHomePageText() {
           return homePageUserName.getText();
   }
 
}

Here, we have BaseTests Class also which contains the common methods needed by other test pages.

import java.time.Duration;
import org.junit.After;
import org.junit.Before;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import io.github.bonigarcia.wdm.WebDriverManager;
import io.qameta.allure.Step;

public class BaseTests {
	
	public WebDriver driver;
	public final static int TIMEOUT = 10;
    
 
	@Before
    @Step("Start the application")
    public void setup() {
    	WebDriverManager.chromedriver().setup();
	    driver = new ChromeDriver();
	    driver.manage().window().maximize();
	    driver.get("https://opensource-demo.orangehrmlive.com/");	    
	    driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(TIMEOUT));

    }
 
    @Step("Stop the application")
    @After
    public void tearDown() {
        driver.quit();
    }   
}

LoginTests

import org.junit.Assert;
import org.junit.Ignore;
import org.junit.Test;
import io.qameta.allure.Description;
import io.qameta.allure.Severity;
import io.qameta.allure.SeverityLevel;

public class LoginTests extends BaseTests{
	 
	@Severity(SeverityLevel.NORMAL)
    @Test
	@Description("Test Description : Login Test with invalid credentials")
    public void invalidCredentials() {
   
	    LoginPage objLoginPage = new LoginPage(driver);
    	objLoginPage.login("Admin", "admin123$$");
    	 
    	// Verify Error Message
    	 Assert.assertEquals("Invalid credentials",objLoginPage.getErrorMessage());
    
    }
    
	@Severity(SeverityLevel.BLOCKER)
    @Test
	@Description("Test Description : Login Test with valid credentials")
    public void gotoHomePage() {
   
	    LoginPage objLoginPage = new LoginPage(driver);
    	objLoginPage.login("Admin", "admin123");
    	 
    	HomePage objHomePage = new HomePage(driver);
    	
    	// Verify Home Page
    	 Assert.assertEquals("Employee Information",objHomePage.getHomePageText());
    
    }
    
	@Severity(SeverityLevel.NORMAL)
    @Test
	@Description("Test Description : Login Test with missing username")
    public void missingUsername() {
   
	    LoginPage objLoginPage = new LoginPage(driver);
    	objLoginPage.login("", "admin123");
    	     	
    	// Verify Error Message
   	     Assert.assertEquals("Invalid credentials",objLoginPage.getMissingUsernameText());
   	     
    
    }
	
	@Severity(SeverityLevel.NORMAL)
    @Test @Ignore
	@Description("Test Description : Login Test with missing password")
    public void missingPassword() {
   
	    LoginPage objLoginPage = new LoginPage(driver);
    	objLoginPage.login("admin", "");
    	    	
    	// Verify Error Message
   	     Assert.assertEquals("Invalid credentials",objLoginPage.getMissingPasswordText());
    
    }    
   
}

Step 3 – Execute the Tests through Command Line

Note:- As you can see my project has two parts – app and GradleSeleniumJUnit4Demo.

Go to the app project and run the tests, using the below command

gradle clean test

The output of the test execution is

Step 4 – Generate the Allure Report

Once the test execution is finished, a folder named allure-results will be generated in the build folder.

To generate Allure Report, use the below command

allure serve build/allure-results

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

The overview page hosts several default widgets representing basic characteristics of your project and test environment.

Statistics – overall report statistics.
Launches – if this report represents several test launches, statistics per launch will be shown here.
Behaviors – information on results aggregated according to stories and features.
Executors – information on test executors that were used to run the tests.
History Trend – if tests accumulated some historical data, its trend will be calculated and shown on the graph.
Environment – information on the test environment.

Categories in Allure Report

Categories tab gives you the way to create custom defects classification to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: statuses breakdown or severity and duration diagrams.

Timeline in Allure Report

Timeline tab visualizes retrospective of tests execution, allure adaptors collect precise timings of tests, and here on this tab they are arranged accordingly to their sequential or parallel timing structure.

Behaviors of Allure Report

This tab groups test results according to Epic, Feature and Story tags.

Packages in Allure Report

Packages tab represents a tree-like layout of test results, grouped by different packages.

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

September 22, 2022March 16, 2023 vibssingh

Gradle – Allure Report for Selenium and TestNG

HOME

The previous tutorial explained the generation of Allure Report for Selenium and TestNG in a Maven Project. This tutorial explains the generation of Allure Report for Selenium and TestNG in a Gradle project.

Pre-Requisite:

Java 8 or higher installed
Gradle installed
Eclipse or IntelliJ installed

This framework consists of:

Selenium – 4.3
Java 11
TestNG – 7.6.1
Gradle – 7.5.1
Allure Maven Plugin – 2.11.0
Allure TestNG – 2.19.1

Project Structure

To create a Gradle project from the command line, please refer to this tutorial – How to create a Java Gradle project using Command Line.

Implementation Steps

Add Selenium, TestNG, and Allure-TestNG dependencies in build.gradle
Create Pages and Test Code for the pages
Create testng.xml
Execute the Tests
Generate Allure Report

Step 1 – Add Selenium, TestNG, and Allure-TestNG dependencies in the build.gradle

plugins {
    // Apply the application plugin to add support for building a CLI application in Java.
    id 'application'
    id 'io.qameta.allure' version '2.11.0'
}

repositories {
    // Use Maven Central for resolving dependencies.
    mavenCentral()
}

java {
    sourceCompatibility = 11
    targetCompatibility = 11
}

dependencies {

    
    // Use TestNG framework, also requires calling test.useTestNG() below
    testImplementation 'org.testng:testng:7.6.1'

    // This dependency is used by the application.
    implementation 'com.google.guava:guava:31.0.1-jre'
    implementation 'org.seleniumhq.selenium:selenium-java:4.4.0'
    implementation 'io.github.bonigarcia:webdrivermanager:5.3.0'
    implementation 'io.qameta.allure:allure-testng:2.19.0'
}

application {
    // Define the main class for the application.
    mainClass = 'com.example.App'
}

tasks.named('test') {
    // Use TestNG for unit tests.
    useTestNG() {
    
    useDefaultListeners = true
    suites "./testng.xml"
 
   }
   
    testLogging {
        events "PASSED", "FAILED", "SKIPPED"
        exceptionFormat = 'full'
    }
}

Step 2 – Create Pages and Test Code for the pages

Below is the sample project which uses Selenium and TestNG which is used to generate an Allure Report.

We have used the PageFactory model to build the tests. I have created a package named pages and created the page classes in that folder. Page class contains the locators of each web element present on that particular page along with the methods of performing actions using these web elements.

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.support.PageFactory;

public class BasePage {	

	  public WebDriver driver;

	  public BasePage(WebDriver driver) {
		  this.driver = driver;
		  PageFactory.initElements(driver,this);
	}

}

Below is the code for LoginPage and HomePage

LoginPage

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;

public class LoginPage extends BasePage{
	
	 public LoginPage(WebDriver driver) {
		 super(driver);
		
    }
	
	@FindBy(name = "username")
    public WebElement userName;
 
    @FindBy(name = "password")
    public WebElement password;
    
    @FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/form/div[1]/div/span")
    public WebElement missingUsernameErrorMessage;
    
    @FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/form/div[1]/div/span")
    public WebElement missingPasswordErrorMessage;
 
    @FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/form/div[3]/button")
    public WebElement login;
 
    @FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/div/div[1]/div[1]/p")
    public  WebElement errorMessage;
          
   // Get the error message when password is blank
    public String getMissingPasswordText() {
        return missingPasswordErrorMessage.getText();
    }
      
    // Get the Error Message
    public String getErrorMessage() {
        return errorMessage.getText();
    }
    
    public void login(String strUserName, String strPassword) {
 
    	userName.sendKeys(strUserName);
    	password.sendKeys(strPassword);
    	login.click();
    }

}

HomePage

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;

public class HomePage extends BasePage {

	public HomePage(WebDriver driver) {
		super(driver);

	}

	 @FindBy(xpath = "//*[@id='app']/div[1]/div[2]/div[2]/div/div[1]/div[1]/div[1]/h5")
	  public  WebElement homePageUserName;

	  // Get the User name from Home Page
	    public String getHomePageText() {
	       return homePageUserName.getText();
   }

}

Here, we have BaseTests Class also which contains the common methods needed by other test pages.

import java.time.Duration;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import io.github.bonigarcia.wdm.WebDriverManager;
import io.qameta.allure.Step;

public class BaseTests {
	
	public WebDriver driver;
	public final static int TIMEOUT = 30;
    
	@BeforeMethod
    @Step("Start the application")
    public void setup() {
    	WebDriverManager.chromedriver().setup();
	    driver = new ChromeDriver();
	    driver.manage().window().maximize();
	    driver.get("https://opensource-demo.orangehrmlive.com/");	    
	    driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(TIMEOUT));

    }
 
    @Step("Stop the application")
    @AfterMethod
    public void tearDown() {
        driver.quit();
    }
    
}

LoginTests

import org.testng.Assert;
import org.testng.annotations.Test;
import io.qameta.allure.Description;
import io.qameta.allure.Severity;
import io.qameta.allure.SeverityLevel;

public class LoginTests extends BaseTests{
	 
	@Severity(SeverityLevel.NORMAL)
    @Test(description = "This test validates error message when credentials are incorrect", priority = 0)
	@Description("Test Description : Login Test with invalid credentials")
    public void invalidCredentials() {
   
	    LoginPage objLoginPage = new LoginPage(driver);
    	objLoginPage.login("Admin", "admin123$$");
    	 
    	// Verify Error Message
    	 Assert.assertEquals(objLoginPage.getErrorMessage(),"Invalid credentials");
    
    }
    
	@Severity(SeverityLevel.BLOCKER)
    @Test(description = "This test validates login to the application", priority = 1)
	@Description("Test Description : Login Test with valid credentials")
    public void gotoHomePage() {
   
	    LoginPage objLoginPage = new LoginPage(driver);
    	objLoginPage.login("Admin", "admin123");
    	 
    	HomePage objHomePage = new HomePage(driver);
    	
    	// Verify Home Page
    	 Assert.assertEquals(objHomePage.getHomePageText(),"Employee Information");
    
    }
    
	@Severity(SeverityLevel.NORMAL)
    @Test(description = "This test will fail", priority = 2)
	@Description("Test Description : Login Test with missing username")
    public void missingUsername() {
   
	    LoginPage objLoginPage = new LoginPage(driver);
    	objLoginPage.login("", "admin123");
    	    	
    	// Verify Error Message
   	        	 Assert.assertEquals(objLoginPage.getMissingUsernameText(),"Invalid credentials");
    
    }
	
	@Severity(SeverityLevel.NORMAL)
    @Test(description = "This test will skip", priority = 3, enabled = false)
	@Description("Test Description : Login Test with missing password")
    public void missingPassword() {
   
	    LoginPage objLoginPage = new LoginPage(driver);
    	objLoginPage.login("admin", "");
    	    	
    	// Verify Error Message
   	     Assert.assertEquals(objLoginPage.getErrorMessage(),"Invalid credentials");
    
    }
     
}

Step 3 – Create testng.xml

Right-click on the project and select TestNG -> Convert to TestNG.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test name="Gradle - Selenium with TestNG Tests">
    <classes>
      <class name="com.example.LoginTests"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

Step 4 – Execute the Tests

Note:- As you can see my project has two parts – the app and GradleSeleniumTestNG.

Go to the app project and run the tests, using the below command

gradle clean test

The output of the test execution is

Step 5 – Generate the Allure Report

Once the test execution is finished, a folder named allure-results will be generated in the build folder.

To generate Allure Report, use the below command

allure serve build/allure-results

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

The overview page hosts several default widgets representing the basic characteristics of your project and test environment.

Statistics – overall report statistics.
Launches – if this report represents several test launches, statistics per launch will be shown here.
Behaviors – information on results aggregated according to stories and features.
Executors – information on test executors that were used to run the tests.
History Trend – if tests accumulated some historical data, its trend will be calculated and shown on the graph.
Environment – information on the test environment.

Categories in Allure Report

The categories tab gives you a way to create custom defects classifications to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.

Here, I have used TestNG, so to skip the tests have used enabled. But, in the Allure Report, it is marked as unknown (pink color).

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: status breakdown or severity and duration diagrams.

Timeline in Allure Report

Behaviors of Allure Report

This tab groups test results according to Epic, Feature, Story, Test Severity, Test Description, Test Steps, and so on.

Packages in Allure Report

The packages tab represents a tree-like layout of test results, grouped by different packages.

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

September 10, 2021January 17, 2023 vibssingh

Integration of Allure Report with Rest Assured and TestNG

HOME

In the previous tutorial, I explained the Integration of the Allure Report with Rest Assured with JUnit4. In this tutorial, I will explain how to Integrate Allure Report with Rest Assured and TestNG.

The below example covers the implementation of Allure Report for Rest API using Rest Assured, TestNG, Java, and Maven.

Pre-Requisite

Java 11 installed
Maven installed
Eclipse or IntelliJ installed

This framework consists of:

Java 11
Maven – 3.8.1
Allure Report – 2.14.0
Rest Assured – 4.4.0
Allure Rest Assured – 2.14.0
Allure TestNG – 2.14.0
Aspectj – 1.9.6

Implementation Steps

Update Properties section in Maven pom.xml
Add Rest Assured, Allure-Rest Assured and Allure-TetNG dependencies in POM.xml
Update Build Section of pom.xml in Allure Report Project.
Create the Test Code for the testing of REST API under src/test/java
Create TestNG.xml for the project
Run the Tests and Generate Allure Report

Step 1 – Update Properties section in Maven pom.xml

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <rest-assured.version>4.4.0</rest-assured.version>
    <allure.testng.version>2.14.0</allure.testng.version>
    <allure.rest-assured.version>2.14.0</allure.rest-assured.version>
    <jackson.version>2.12.3</jackson.version>
    <json.version>20210307</json.version>
    <maven.compiler.plugin.version>3.5.1</maven.compiler.plugin.version>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <aspectj.version>1.9.6</aspectj.version>
    <maven-surefire-plugin-version>3.0.0-M5</maven-surefire-plugin-version>
  </properties>

Step 2 – Add the Allure-Rest Assured dependency

 <!--Allure Reporting Dependency-->   
   <dependency>
      <groupId>io.qameta.allure</groupId>
      <artifactId>allure-rest-assured</artifactId>
      <version>${allure.rest-assured.version}</version>
   </dependency>

Add other dependencies like Rest Assured and Allure-TetNG dependencies in POM.xml

<dependencies>

     <!-- Allure TestNG Dependency -->
     <dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-testng</artifactId>
        <version>${allure.testng.version}</version>
        <scope>test</scope>
    </dependency>
    
   <!-- Rest Assured Dependency -->
    <dependency>
       <groupId>io.rest-assured</groupId>
       <artifactId>rest-assured</artifactId>
       <version>${rest-assured.version}</version>
       <scope>test</scope>
    </dependency>

    <!-- Jackson Dependency -->
    <dependency>
       <groupId>com.fasterxml.jackson.core</groupId>
       <artifactId>jackson-databind</artifactId>
       <version>${jackson.version}</version>
    </dependency>

    <!-- JSON Dependency -->
    <dependency>
       <groupId>org.json</groupId>
       <artifactId>json</artifactId>
       <version>${json.version}</version>
    </dependency>  
    
  </dependencies>

Step 3 – Update the Build Section of pom.xml in Allure Report Project

 <build>
   <plugins>
         <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
               <version>${maven.compiler.plugin.version}</version>
                <configuration>
                    <source>${maven.compiler.source}</source> <!--For JAVA 8 use 1.8-->
                    <target>${maven.compiler.target}</target> <!--For JAVA 8 use 1.8-->
                </configuration>
            </plugin>

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>${maven-surefire-plugin-version}</version>
           <configuration>
                    <suiteXmlFiles>
                        <suiteXmlFile>TestNG.xml</suiteXmlFile>
                    </suiteXmlFiles>
                 <argLine>
                    -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                 </argLine>
             </configuration>          
             <dependencies>
             
            <!-- https://mvnrepository.com/artifact/org.aspectj/aspectjweaver -->
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjweaver</artifactId>
                    <version>${aspectj.version}</version>
                </dependency>
            </dependencies>
        </plugin>
      </plugins>
  </build>

Step 4 – Create the Test Code for the testing of REST API under src/test/java

To see our request and response in more detail using Rest Assured, we need to add a line to our Rest Assured tests. This will provide the request and response details in the report.

 .filter(new AllureRestAssured())

@Epic("REST API Regression Testing using TestNG")
@Feature("Verify CRUID Operations on Employee module")
public class EmployeeDetailsTest {

	String BaseURL = "http://dummy.restapiexample.com/api";

	@Test(description = "GET Request Operation")
	@Story("GET Request with Valid User")
	@Severity(SeverityLevel.NORMAL)
	@Description("Test Description : Verify the details of employee of id-2")
	public void verifyUser() {

		// GIVEN
		given()
               .filter(new AllureRestAssured())

		// WHEN
		.when()
               .get(BaseURL + "/v1/employee/2")
				
        // THEN
		  .then()
                .statusCode(200)
                .statusLine("HTTP/1.1 200 OK")
				
                 // To verify booking id at index 2
				.body("data.employee_name", equalTo("Garrett Winters"))
				.body("message", equalTo("Successfully! Record has been fetched."));
	}

	@Test(description = "GET Request Operation")
	@Story("GET Request with Invalid User")
	@Severity(SeverityLevel.NORMAL)
	@Description("Test Description : Verify the details of employee of id-99999")
	public void verifyInvalidUser() {

		// Given
		given()
              .filter(new AllureRestAssured())

		// WHEN
		.when()
              .get(BaseURL + "/v1/employee/99999")
				
        // THEN
		.then()
               .statusCode(200)
               .statusLine("HTTP/1.1 200 OK")
			   .body("message", equalTo("Successfully! Record has not been fetched."));
	}

	@Test(description = "POST Request Operation")
	@Story("POST Request")
	@Severity(SeverityLevel.NORMAL)
	@Description("Test Description : Verify the creation of a new employee")
	public void createUser() {

		JSONObject data = new JSONObject();

		data.put("employee_name", "APITest");
		data.put("employee_salary", "99999");
		data.put("employee_age", "30");

        //GIVEN 
		given()
              .filter(new AllureRestAssured())
              .contentType(ContentType.JSON)
              .body(data.toString())

	    // WHEN
		.when()
               .post(BaseURL + "/v1/create")

		// THEN
		.then()
               .statusCode(200)
               .body("data.employee_name", equalTo("APITest"))
			   .body("message", equalTo("Successfully! Record has been added."));

	}

}

Step 5 – Create testng.xml for the project

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test thread-count="5" name="Test">
    <classes>
      <class name="com.example.RestAssuredTestNGAllureReport.EmployeeDetailsTest"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

Step 6 – Run the Test and Generate Allure Report

To run the tests, use the below command

mvn clean test

In the below image, we can see that two tests failed and one passed out of three tests.

This will create the allure-results folder with all the test reports. These files will be used to generate the Allure Report.

To create Allure Report, use the below command

allure serve

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

The overview page hosts several default widgets representing the basic characteristics of your project and test environment.

Categories in Allure Report

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes, can be found.

View test history

Each time you run the report from the command line with the mvn clean test command, a new result JSON file will get added to the allure-results folder. Allure can use those files to include a historical view of your tests. Let’s give that a try.

To get started, run mvn clean test a few times and watch how the number of files in the allure-reports folder grows.

Now go back to view your report. Select Suites from the left nav, select one of your tests and click Retries in the right pane. You should see the history of test runs for that test:

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: status breakdown or severity and duration diagrams.

Timeline in Allure Report

Timeline tab visualizes retrospective of tests execution, allure adaptors collect precise timings of tests, and here on this tab, they are arranged accordingly to their sequential or parallel timing structure.

Behaviors of Allure Report

This tab groups test results according to Epic, Feature, and Story tags.

Packages in Allure Report

The packages tab represents a tree-like layout of test results, grouped by different packages.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

September 1, 2021March 25, 2023 vibssingh

Integration of Allure Report with Rest Assured and JUnit4

HOME

In the previous tutorial, I explained the Integration of the Allure Report with Cucumber5 and JUnit4. In this tutorial, I will explain how to Integrate Allure Report with Rest Assured and JUnit4.

The below example covers the implementation of Allure Report for Rest API using Rest Assured, JUnit4, Java, and Maven.

Pre-Requisite

Java 11 installed
Maven installed
Eclipse or IntelliJ installed

This framework consists of:

Java 11
Maven – 3.8.1
Allure Report – 2.14.0
Rest Assured – 4.4.0
Allure Rest Assured – 2.14.0
Allure JUnit4 – 2.14.0
Aspectj – 1.9.6

Implementation Steps

Update Properties section in Maven pom.xml
Add Rest Assured, Allure-Rest Assured, and Allure-JUnit4 dependencies in POM.xml
Update Build Section of pom.xml in Allure Report Project.
Create the Test Code for the testing of REST API under src/test/java
Run the Tests and Generate Allure Report

Step 1 – Update Properties section in Maven pom.xml

 <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <rest-assured.version>4.4.0</rest-assured.version>
    <allure.junit4.version>2.14.0</allure.junit4.version>
    <allure.rest-assured.version>2.14.0</allure.rest-assured.version>
    <jackson.version>2.12.3</jackson.version>
    <json.version>20210307</json.version>
    <maven.compiler.plugin.version>3.5.1</maven.compiler.plugin.version>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <aspectj.version>1.9.6</aspectj.version>
    <maven-surefire-plugin-version>3.0.0-M5</maven-surefire-plugin-version>
    <allure.maven.version>2.10.0</allure.maven.version>
  </properties>

Step 2 – Add the Allure-Rest Assured dependency

<!--Allure Reporting Dependency-->   
<dependency>
    <groupId>io.qameta.allure</groupId>
    <artifactId>allure-rest-assured</artifactId>
    <version>${allure.rest-assured.version}</version>
</dependency>

Add other dependencies like Rest Assured and Allure-JUnit4 dependencies in POM.xml

<dependencies>
    
    <dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-junit4</artifactId>
        <version>${allure.junit4.version}</version>
        <scope>test</scope>
    </dependency>   
    
   <!-- Rest Assured Dependency -->
    <dependency>
       <groupId>io.rest-assured</groupId>
       <artifactId>rest-assured</artifactId>
       <version>${rest-assured.version}</version>
       <scope>test</scope>
    </dependency>

    <!-- Jackson Dependency -->
    <dependency>
       <groupId>com.fasterxml.jackson.core</groupId>
       <artifactId>jackson-databind</artifactId>
       <version>${jackson.version}</version>
    </dependency>

    <!-- JSON Dependency -->
    <dependency>
       <groupId>org.json</groupId>
       <artifactId>json</artifactId>
       <version>${json.version}</version>
    </dependency>  

</dependencies>

Step 3 – Update the Build Section of pom.xml in Allure Report Project

<build>
   <plugins>
         <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
               <version>${maven.compiler.plugin.version}</version>
                <configuration>
                    <source>${maven.compiler.source}</source> <!--For JAVA 8 use 1.8-->
                    <target>${maven.compiler.target}</target> <!--For JAVA 8 use 1.8-->
                </configuration>
            </plugin>

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>${maven-surefire-plugin-version}</version>
            <configuration>
                <testFailureIgnore>false</testFailureIgnore>
                <argLine>
                    -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                </argLine>
                <properties>
                    <property>
                        <name>listener</name>
                        <value>io.qameta.allure.junit4.AllureJunit4</value>
                    </property>
                </properties>
            </configuration>
            <dependencies>
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjweaver</artifactId>
                    <version>${aspectj.version}</version>
                </dependency>
            </dependencies>
        </plugin>
        
        <plugin>
                <groupId>io.qameta.allure</groupId>
                <artifactId>allure-maven</artifactId>
                <version>${allure.maven.version}</version>
                 <configuration>
                    <reportVersion>2.4.1</reportVersion>
                </configuration>
            </plugin>
            
      </plugins>
  </build>
</project>

Step 4 – Create a Test Code for the testing of REST API under src/test/java

Rest Assured and Allure Report are two popular tools for testing. Rest Assured is used for API testing and Allure Report is used for creating detailed reports about tests. To see our request and response in more detail using these tools, we need to add a line to our Rest Assured tests. This will provide the request and response details in the report.

.filter(new AllureRestAssured())

@Epic("REST API Regression Testing using JUnit4")
@Feature("Verify CRUID Operations on Employee module")
public class EmployeeDetailsTest {

	String BaseURL = "http://dummy.restapiexample.com/api";

	@Test
	@Story("GET Request")
	@Severity(SeverityLevel.NORMAL)
	@Description("Test Description : Verify the details of employee of id-2")
	public void verifyUser() {

		// Given
		given()
              .filter(new AllureRestAssured())
				
      // WHEN
	  .when()
             .get(BaseURL + "/v1/employee/2")
				
      // THEN
	  .then()
             .statusCode(200)
             .statusLine("HTTP/1.1 200 OK")
			// To verify booking id at index 2
			 .body("data.employee_name", equalTo("Garrett Winters"))
			 .body("message", equalTo("Successfully! Record has been fetched."));
	}

	@Test
	@Story("POST Request")
	@Severity(SeverityLevel.NORMAL)
	@Description("Test Description : Verify the creation of a new employee")
	public void createUser() {

		JSONObject data = new JSONObject();

		// Map<String, String> map = new HashMap<String, String>();

		data.put("employee_name", "APITest");
		data.put("employee_salary", "99999");
		data.put("employee_age", "30");

		// GIVEN
		given()
               .filter(new AllureRestAssured())
               .contentType(ContentType.JSON)
               .body(data.toString())

	    // WHEN
		.when()
              .post(BaseURL + "/v1/create")

		// THEN
		.then()
              .statusCode(200)
              .body("data.employee_name", equalTo("APITest"))
			 .body("message", equalTo("Successfully! Record has been added."));

	}

}

Step 5 – Run the Test and Generate Allure Report

To run the tests, use the below command

mvn clean test

The output of the above program is

This will create allure-results folder with all the test reports. These files will be used to generate Allure Report.

To create Allure Report, use the below command

allure serve

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

Categories in Allure Report

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.

View test history

To get started, run mvn clean test a few times and watch how the number of files in the allure-reports folder grows.

Now go back to view your report. Select Suites from the left nav, select one of your tests and click Retries in the right pane. You should see the history of test runs for that test:

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: status breakdown or severity and duration diagrams.

Timeline in Allure Report

Behaviors of Allure Report

This tab groups test results according to Epic, Feature, and Story tags.

The below image shows the request body sent and the status code of the response, its body, and header provided by API.

Packages in Allure Report

The packages tab represents a tree-like layout of test results, grouped by different packages.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!