Allure Report

vibssingh

Integration of Allure Report with Rest Assured and TestNG

HOME

In the previous tutorial, I have explained the Integration of the Allure Report with Rest Assured with JUnit4 . In this tutorial, I will explain how to Integrate Allure Report with Rest Assured and TestNG.

Below example covers the implementation of Allure Report for Rest API using Rest Assured ,TestNG, Java and Maven.

Pre-Requisite

  1. Java 11 installed
  2. Maven installed
  3. Eclipse or IntelliJ installed

This framework consists of:

  1. Java 11
  2. Maven – 3.8.1
  3. Allure Report – 2.14.0
  4. Rest Assured – 4.4.0
  5. Allure Rest Assured – 2.14.0
  6. Allure TestNG – 2.14.0
  7. Aspectj – 1.9.6

Implementation Steps

  1. Update Properties section in Maven pom.xml
  2. Add Rest Assured, Allure-Rest Assured and Allure-TetNG dependencies in POM.xml
  3. Update Build Section of pom.xml in Allure Report Project.
  4. Create the Test Code for the testing of REST API under src/test/java
  5. Create TestNG.xml for the project
  6. Run the Tests and Generate Allure Report

Step 1 – Update Properties section in Maven pom.xml

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <rest-assured.version>4.4.0</rest-assured.version>
    <allure.testng.version>2.14.0</allure.testng.version>
    <allure.rest-assured.version>2.14.0</allure.rest-assured.version>
    <jackson.version>2.12.3</jackson.version>
    <json.version>20210307</json.version>
    <maven.compiler.plugin.version>3.5.1</maven.compiler.plugin.version>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <aspectj.version>1.9.6</aspectj.version>
    <maven-surefire-plugin-version>3.0.0-M5</maven-surefire-plugin-version>
  </properties>

Step 2 – Add the Allure-Rest Assured dependency 

 <!--Allure Reporting Dependency-->   
   <dependency>
      <groupId>io.qameta.allure</groupId>
      <artifactId>allure-rest-assured</artifactId>
      <version>${allure.rest-assured.version}</version>
   </dependency>

Add other dependencies like Rest Assured and Allure-TetNG dependencies in POM.xml

<dependencies>

     <!-- Allure TestNG Dependency -->
     <dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-testng</artifactId>
        <version>${allure.testng.version}</version>
        <scope>test</scope>
    </dependency>
    
   <!-- Rest Assured Dependency -->
    <dependency>
       <groupId>io.rest-assured</groupId>
       <artifactId>rest-assured</artifactId>
       <version>${rest-assured.version}</version>
       <scope>test</scope>
    </dependency>

    <!-- Jackson Dependency -->
    <dependency>
       <groupId>com.fasterxml.jackson.core</groupId>
       <artifactId>jackson-databind</artifactId>
       <version>${jackson.version}</version>
    </dependency>

    <!-- JSON Dependency -->
    <dependency>
       <groupId>org.json</groupId>
       <artifactId>json</artifactId>
       <version>${json.version}</version>
    </dependency>  
    
  </dependencies>

Step 3 – Update Build Section of pom.xml in Allure Report Project

 <build>
   <plugins>
         <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
               <version>${maven.compiler.plugin.version}</version>
                <configuration>
                    <source>${maven.compiler.source}</source> <!--For JAVA 8 use 1.8-->
                    <target>${maven.compiler.target}</target> <!--For JAVA 8 use 1.8-->
                </configuration>
            </plugin>

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>${maven-surefire-plugin-version}</version>
           <configuration>
                    <suiteXmlFiles>
                        <suiteXmlFile>TestNG.xml</suiteXmlFile>
                    </suiteXmlFiles>
                 <argLine>
                    -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                 </argLine>
             </configuration>          
             <dependencies>
             
            <!-- https://mvnrepository.com/artifact/org.aspectj/aspectjweaver -->
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjweaver</artifactId>
                    <version>${aspectj.version}</version>
                </dependency>
            </dependencies>
        </plugin>
      </plugins>
  </build>

Step 4 – Create the Test Code for the testing of REST API under src/test/java

To see our request and response more detailed using Rest Assured we need to add a line to our Rest Assured tests.This will provide the request and response details in the report. 

 .filter(new AllureRestAssured())

@Epic("REST API Regression Testing using TestNG")
@Feature("Verify CRUID Operations on Employee module")
public class EmployeeDetailsTest {

	String BaseURL = "http://dummy.restapiexample.com/api";

	@Test(description = "GET Request Operation")
	@Story("GET Request with Valid User")
	@Severity(SeverityLevel.NORMAL)
	@Description("Test Description : Verify the details of employee of id-2")
	public void verifyUser() {

		// GIVEN
		given()
               .filter(new AllureRestAssured())

		// WHEN
		.when()
               .get(BaseURL + "/v1/employee/2")
				
        // THEN
		  .then()
                .statusCode(200)
                .statusLine("HTTP/1.1 200 OK")
				
                 // To verify booking id at index 2
				.body("data.employee_name", equalTo("Garrett Winters"))
				.body("message", equalTo("Successfully! Record has been fetched."));
	}

	@Test(description = "GET Request Operation")
	@Story("GET Request with Invalid User")
	@Severity(SeverityLevel.NORMAL)
	@Description("Test Description : Verify the details of employee of id-99999")
	public void verifyInvalidUser() {

		// Given
		given()
              .filter(new AllureRestAssured())

		// WHEN
		.when()
              .get(BaseURL + "/v1/employee/99999")
				
        // THEN
		.then()
               .statusCode(200)
               .statusLine("HTTP/1.1 200 OK")
			   .body("message", equalTo("Successfully! Record has not been fetched."));
	}

	@Test(description = "POST Request Operation")
	@Story("POST Request")
	@Severity(SeverityLevel.NORMAL)
	@Description("Test Description : Verify the creation of a new employee")
	public void createUser() {

		JSONObject data = new JSONObject();

		data.put("employee_name", "APITest");
		data.put("employee_salary", "99999");
		data.put("employee_age", "30");

        //GIVEN 
		given()
              .filter(new AllureRestAssured())
              .contentType(ContentType.JSON)
              .body(data.toString())

	    // WHEN
		.when()
               .post(BaseURL + "/v1/create")

		// THEN
		.then()
               .statusCode(200)
               .body("data.employee_name", equalTo("APITest"))
			   .body("message", equalTo("Successfully! Record has been added."));

	}

}

Step 5 – Create testng.xml for the project

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test thread-count="5" name="Test">
    <classes>
      <class name="com.example.RestAssuredTestNGAllureReport.EmployeeDetailsTest"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

Step 6 – Run the Test and Generate Allure Report

To run the tests, use the below command

mvn clean test

In the below image, we can see that two tesst are failed and one passed out of three tests.

This will create allure-results folder with all the test report. These files will be use to generate Allure Report.

To create Allure Report, use the below command

allure serve

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

Overview page hosts several default widgets representing basic characteristics of your project and test environment.

Categories in Allure Report

Categories tab gives you the way to create custom defects classification to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.

View test history

Each time you run the report from the command line with the mvn clean test command, new result json file will get added in the allure-results folder. Allure can use those files to include a historical view of your tests. Let’s give that a try.

To get started run mvn clean test a few time and watch how the number of file in the allure-reports folder grows.

Now go back to view your report. Select Suites from the left nav, select one of your tests, and cick Retries in the right pane. You should see history of test runs for that test:

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: statuses breakdown or severity and duration diagrams.

Timeline in Allure Report

Timeline tab visualizes retrospective of tests execution, allure adaptors collect precise timings of tests, and here on this tab they are arranged accordingly to their sequential or parallel timing structure.

Behaviors of Allure Report

This tab groups test results according to Epic, Feature and Story tags.

Packages in Allure Report

Packages tab represents a tree-like layout of test results, grouped by different packages.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

vibssingh

Integration of Allure Report with Rest Assured and JUnit4

HOME

In the previous tutorial, I have explained the Integration of the Allure Report with Cucumber5 and JUnit4. In this tutorial, I will explain how to Integrate Allure Report with Rest Assured and JUnit4.

Below example covers the implementation of Allure Report for Rest API using Rest Assured ,JUnit4, Java and Maven.

Pre-Requisite

  1. Java 11 installed
  2. Maven installed
  3. Eclipse or IntelliJ installed

This framework consists of:

  1. Java 11
  2. Maven – 3.8.1
  3. Allure Report – 2.14.0
  4. Rest Assured – 4.4.0
  5. Allure Rest Assured – 2.14.0
  6. Allure JUnit4 – 2.14.0
  7. Aspectj – 1.9.6

Implementation Steps

  1. Update Properties section in Maven pom.xml
  2. Add Rest Assured, Allure-Rest Assured and Allure-JUnit4 dependencies in POM.xml
  3. Update Build Section of pom.xml in Allure Report Project.
  4. Create the Test Code for the testing of REST API under src/test/java
  5. Run the Tests and Generate Allure Report

Step 1 – Update Properties section in Maven pom.xml

 <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <rest-assured.version>4.4.0</rest-assured.version>
    <allure.junit4.version>2.14.0</allure.junit4.version>
    <allure.rest-assured.version>2.14.0</allure.rest-assured.version>
    <jackson.version>2.12.3</jackson.version>
    <json.version>20210307</json.version>
    <maven.compiler.plugin.version>3.5.1</maven.compiler.plugin.version>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <aspectj.version>1.9.6</aspectj.version>
    <maven-surefire-plugin-version>3.0.0-M5</maven-surefire-plugin-version>
    <allure.maven.version>2.10.0</allure.maven.version>
  </properties>

Step 2 – Add the Allure-Rest Assured dependency

<!--Allure Reporting Dependency-->   
<dependency>
    <groupId>io.qameta.allure</groupId>
    <artifactId>allure-rest-assured</artifactId>
    <version>${allure.rest-assured.version}</version>
</dependency>

Add other dependencies like Rest Assured and Allure-JUnit4 dependencies in POM.xml

<dependencies>
    
    <dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-junit4</artifactId>
        <version>${allure.junit4.version}</version>
        <scope>test</scope>
    </dependency>   
    
   <!-- Rest Assured Dependency -->
    <dependency>
       <groupId>io.rest-assured</groupId>
       <artifactId>rest-assured</artifactId>
       <version>${rest-assured.version}</version>
       <scope>test</scope>
    </dependency>

    <!-- Jackson Dependency -->
    <dependency>
       <groupId>com.fasterxml.jackson.core</groupId>
       <artifactId>jackson-databind</artifactId>
       <version>${jackson.version}</version>
    </dependency>

    <!-- JSON Dependency -->
    <dependency>
       <groupId>org.json</groupId>
       <artifactId>json</artifactId>
       <version>${json.version}</version>
    </dependency>  

</dependencies>

Step 3 – Update Build Section of pom.xml in Allure Report Project

<build>
   <plugins>
         <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
               <version>${maven.compiler.plugin.version}</version>
                <configuration>
                    <source>${maven.compiler.source}</source> <!--For JAVA 8 use 1.8-->
                    <target>${maven.compiler.target}</target> <!--For JAVA 8 use 1.8-->
                </configuration>
            </plugin>

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>${maven-surefire-plugin-version}</version>
            <configuration>
                <testFailureIgnore>false</testFailureIgnore>
                <argLine>
                    -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                </argLine>
                <properties>
                    <property>
                        <name>listener</name>
                        <value>io.qameta.allure.junit4.AllureJunit4</value>
                    </property>
                </properties>
            </configuration>
            <dependencies>
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjweaver</artifactId>
                    <version>${aspectj.version}</version>
                </dependency>
            </dependencies>
        </plugin>
        
        <plugin>
                <groupId>io.qameta.allure</groupId>
                <artifactId>allure-maven</artifactId>
                <version>${allure.maven.version}</version>
                 <configuration>
                    <reportVersion>2.4.1</reportVersion>
                </configuration>
            </plugin>
            
      </plugins>
  </build>
</project>

Step 4 – Create Test Code for the testing of REST API under src/test/java

Rest Assured and Allure Report is two popular tool for testing.Rest Assured is using for API testing and Allure Report is using for create detailed reports about tests.To see our request and response more detailed using this tools we need to add a line to our Rest Assured tests.This will provide the request and response details in the report.

.filter(new AllureRestAssured())

@Epic("REST API Regression Testing using JUnit4")
@Feature("Verify CRUID Operations on Employee module")
public class EmployeeDetailsTest {

	String BaseURL = "http://dummy.restapiexample.com/api";

	@Test
	@Story("GET Request")
	@Severity(SeverityLevel.NORMAL)
	@Description("Test Description : Verify the details of employee of id-2")
	public void verifyUser() {

		// Given
		given()
              .filter(new AllureRestAssured())
				
      // WHEN
	  .when()
             .get(BaseURL + "/v1/employee/2")
				
      // THEN
	  .then()
             .statusCode(200)
             .statusLine("HTTP/1.1 200 OK")
			// To verify booking id at index 2
			 .body("data.employee_name", equalTo("Garrett Winters"))
			 .body("message", equalTo("Successfully! Record has been fetched."));
	}

	@Test
	@Story("POST Request")
	@Severity(SeverityLevel.NORMAL)
	@Description("Test Description : Verify the creation of a new employee")
	public void createUser() {

		JSONObject data = new JSONObject();

		// Map<String, String> map = new HashMap<String, String>();

		data.put("employee_name", "APITest");
		data.put("employee_salary", "99999");
		data.put("employee_age", "30");

		// GIVEN
		given()
               .filter(new AllureRestAssured())
               .contentType(ContentType.JSON)
               .body(data.toString())

	    // WHEN
		.when()
              .post(BaseURL + "/v1/create")

		// THEN
		.then()
              .statusCode(200)
              .body("data.employee_name", equalTo("APITest"))
			 .body("message", equalTo("Successfully! Record has been added."));

	}

}

Step 5 – Run the Test and Generate Allure Report

To run the tests, use the below command

mvn clean test

This will create allure-results folder with all the test report. These files will be use to generate Allure Report.

To create Allure Report, use the below command

allure serve

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

Categories in Allure Report

Categories tab gives you the way to create custom defects classification to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.

View test history

Each time you run the report from the command line with the mvn clean test command, new result json file will get added in the allure-results folder. Allure can use those files to include a historical view of your tests. Let’s give that a try.

To get started run mvn clean test a few time and watch how the number of file in the allure-reports folder grows.

Now go back to view your report. Select Suites from the left nav, select one of your tests, and cick Retries in the right pane. You should see history of test runs for that test:

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: statuses breakdown or severity and duration diagrams.

Timeline in Allure Report

Timeline tab visualizes retrospective of tests execution, allure adaptors collect precise timings of tests, and here on this tab they are arranged accordingly to their sequential or parallel timing structure.

Behaviors of Allure Report

This tab groups test results according to Epic, Feature and Story tags.

Below image show the request body sent and the status code of response, its body and header provided by API.

Packages in Allure Report

Packages tab represents a tree-like layout of test results, grouped by different packages.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

vibssingh

Allure Report with Cucumber5 and TestNG

HOME

In the previous tutorial, I have explained the Integration of the Allure Report with Selenium and TestNG. In this tutorial, I will explain how to Integrate Allure Report with Cucumber5 and TestNG.

Below example covers the implementation of Allure Reports with Cucumber5, Selenium, TestNG, Java and Maven.

Pre-Requisite

  1. Java 11 installed
  2. Maven installed
  3. Eclipse or IntelliJ installed

This framework consists of:

  1. Selenium – 3.141.59
  2. Java 11
  3. Cucumber 5 – 5.7.0
  4. Maven – 3.8.1
  5. Allure Report – 2.14.0
  6. Allure Cucumber5 – 2.14.0
  7. Aspectj – 1.9.6

Implementation Steps

  1. Update Properties section in Maven pom.xml
  2. Add Cucumber5, Selenium, TestNG and Allure Cucumber5 dependencies in POM.xml
  3. Update Build Section of pom.xml in Allure Report Project.
  4. Create source folder – src/test/resources and features folder within src/test/resources to create test scenarios in Feature file
  5. Create the Step Definition class or Glue Code
  6. Create a Cucumber Runner class
  7. Run the Test and Generate Allure Report

Step 1 – Update Properties section in Maven pom.xml

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <selenium.version>3.141.59</selenium.version>
    <cucumber.version>5.7.0</cucumber.version>
    <testng.version>7.4.0</testng.version>
    <allure.cucumber5.version>2.14.0</allure.cucumber5.version>
    <maven.compiler.plugin.version>3.5.1</maven.compiler.plugin.version>
    <maven.surefire.plugin.version>3.0.0-M5</maven.surefire.plugin.version>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <aspectj.version>1.9.6</aspectj.version>
    <allure.maven.version>2.10.0</allure.maven.version>
  </properties>

Step 2 – Add Cucumber5, Selenium, TestNG, Allure-Cucumber5 and Allure-TestNG dependencies in POM.xml

<dependencies>
  <!--Cucumber Dependencies-->  
    <dependency>
       <groupId>io.cucumber</groupId>
       <artifactId>cucumber-java</artifactId>
       <version>${cucumber.version}</version>
    </dependency>
 
  <!--Selenium Dependency-->
    <dependency>
       <groupId>org.seleniumhq.selenium</groupId>
       <artifactId>selenium-java</artifactId>
       <version>${selenium.version}</version>
    </dependency>
  <!--Cucumber TestNG Dependency-->
    <dependency>
       <groupId>io.cucumber</groupId>
       <artifactId>cucumber-testng</artifactId>
       <version>${cucumber.version}</version>
        <scope>test</scope>
    </dependency>
  <!--Hamcrest Dependency-->
    <dependency>
       <groupId>org.hamcrest</groupId>
       <artifactId>hamcrest</artifactId>
       <version>2.2</version>
       <scope>test</scope>
    </dependency>
    
  <!--Allure Cucumber Dependency-->   
      <dependency>
         <groupId>io.qameta.allure</groupId>
         <artifactId>allure-cucumber5-jvm</artifactId>
         <version>${allure.cucumber5.version}</version>
    </dependency>  
</dependencies>

Step 3 – Update Build Section of pom.xml in Allure Report Project

<build>
    <plugins>   
      <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${maven.compiler.plugin.version}</version>
                <configuration>
                    <source>${maven.compiler.source}</source> 
                    <target>${maven.compiler.target}</target> 
                </configuration>
            </plugin>
     
     <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
             <version>${maven.surefire.plugin.version}</version>
            <configuration>
                 <suiteXmlFiles>
                        <suiteXmlFile>TestNG.xml</suiteXmlFile>
                    </suiteXmlFiles>
                <argLine>
                    -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                    -Dcucumber.options="--plugin io.qameta.allure.cucumber5jvm.AllureCucumber5Jvm"
                </argLine>
                        
            </configuration>
            <dependencies>
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjweaver</artifactId>
                    <version>${aspectj.version}</version>
                </dependency>
            </dependencies>
        </plugin>
        
         <plugin>
                <groupId>io.qameta.allure</groupId>
                <artifactId>allure-maven</artifactId>
                <version>${allure.maven.version}</version>
                <configuration>
                    <reportVersion>${allure.version}</reportVersion>
                </configuration>
            </plugin>
      </plugins>      
   </build>

Step 4 – Create source folder – src/test/resources and features folder within src/test/resources to create test scenarios in Feature file

Feature file should be saved as an extension of .feature. Add the test scenarios in this feature file. I have added sample test scenarios. In this feature file. The test scnerios are written in Gherkins language.

@LoginPage @TestNG
@severity=blocker
Feature: Feature - Login to HRM Application 
 
   @ValidCredentials
   Scenario: Scenario - Login with valid credentials 
     
    Given User is on Home page
    When User enters username as "Admin"
    And User enters password as "admin123"
    Then User should be able to login sucessfully
    
   @InvalidCredentials
   Scenario Outline: Scenario -Login with invalid credentials
   
    Given User is on Home page
    When User enters username as "<username>"
    And User enters password as "<password>"
    Then Error message "<message>" should be displayed
    
    Examples:
    |username  |password  |message                         |
    |admin     |admin     |Invalid credentials             |
    |          |admin123  |Username can be empty           | 
    |Admin     |          |Password can be empty           |
    |          |          |Username cannot be empty        |

Step 5 – Create the Step Definition class or Glue Code

CommonDefinitions.java

package com.example.CucumberAllureReportDemo.definitions;
import java.util.concurrent.TimeUnit;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import io.cucumber.java.After;
import io.cucumber.java.Before;
import io.cucumber.java.Scenario;
public class CommonDefinitions {
	protected static WebDriver driver;
	@Before
	public void setup() {
		System.setProperty("webdriver.gecko.driver",
				"C:\\Users\\Vibha\\Software\\geckodriver-v0.26.0-win64\\geckodriver.exe");
		driver = new FirefoxDriver();
		driver.manage().window().maximize();
		driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
	}
	@After
	public void tearDown(Scenario scenario) {
		try {
			String screenshotName = scenario.getName().replaceAll("", "_");
			if (scenario.isFailed()) {
				TakesScreenshot ts = (TakesScreenshot) driver;
				byte[] screenshot = ts.getScreenshotAs(OutputType.BYTES);
				scenario.attach(screenshot, "img/png", screenshotName);
			}
		} catch (Exception e) {
			e.printStackTrace();
		}
		driver.quit();
	}
}

LoginPageDefinition.java

public class LoginDefinition {
	@Given("User is on Home page")
	public void userOnHomePage() {
		CommonDefinitions.driver.get("https://opensource-demo.orangehrmlive.com/");
	}
	@When("User enters username as {string}")
	public void entersUsername(String userName) throws InterruptedException {
		System.out.println("Username Entered");
		CommonDefinitions.driver.findElement(By.name("txtUsername")).sendKeys(userName);
	}
	@When("User enters password as {string}")
	public void entersPassword(String passWord) throws InterruptedException {
		System.out.println("Password Entered");
		CommonDefinitions.driver.findElement(By.name("txtPassword")).sendKeys(passWord);
		CommonDefinitions.driver.findElement(By.id("btnLogin")).submit();
	}
	@Then("User should be able to login sucessfully")
	public void sucessfulLogin() throws InterruptedException {
		String newPageText = CommonDefinitions.driver.findElement(By.id("welcome")).getText();
		System.out.println("newPageText :" + newPageText);
		assertThat(newPageText, containsString("Welcome"));
	}
	@Then("Error message {string} should be displayed")
	public void unsucessfulLogin(String message) throws InterruptedException {
		String errorMessage = CommonDefinitions.driver.findElement(By.id("spanMessage")).getText();
		System.out.println("Error Message :" + errorMessage);
		Assert.assertEquals(errorMessage, message);
	}
}

Step 6 – Create a Cucumber Runner class

We need to create a class called Runner class to run the tests. This class will use the JUnit annotation @RunWith(), which tells JUnit what is the test runner class.

import org.testng.annotations.Test;
import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;
@Test
@CucumberOptions(plugin = { "pretty }, tags = "", features = "src/test/resources/features/Login.feature", glue = "com.example.CucumberAllureReportDemo.definitions")
public class CucumberRunnerTests extends AbstractTestNGCucumberTests {
}

Note:- @Test annotation marks this class as part of the test. So, if we will remove this annotation, the Allure Report execute CucumberRunnerTests as a seperate test suite, so there will be duplicate results.

Step 7 – Create testng.xml for the project

<?xml version = "1.0"encoding = "UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name = "Suite1">
  <test name = "Test Demo">
    <classes>
          <class name = "com.example.CucumberAllureReportDemo.runner.CucumberRunnerTests"/>
     </classes>  
   </test>
</suite>

Step 8 – Run the Test and Generate Allure Report

To run the tests, use the below command

mvn clean test

In the below image, we can see that two tests are failed and three passed out of five tests.

This will create allure-results folder with all the test report. These files will be use to generate Allure Report.

Use the below comand to generate the Allure Report

allure serve

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

Categories in Allure Report

Categories tab gives you the way to create custom defects classification to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found. Here, we have 2 suits – Feature and Surefire test. Surefire test are executed from CucumberRunnerTests.

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: statuses breakdown or severity and duration diagrams.

Timeline in Allure Report

Timeline tab visualizes retrospective of tests execution, allure adaptors collect precise timings of tests, and here on this tab they are arranged accordingly to their sequential or parallel timing structure.

Behaviors of Allure Report

This tab groups test results according to Epic, Feature and Story tags.

Packages in Allure Report

Packages tab represents a tree-like layout of test results, grouped by different packages.

When we don’t use @Test in CucumberRunnerTests.java, then as mentioned above the Allure report will have duplicate details as shown below:-

vibssingh

Allure Report with Cucumber5 and JUnit4

HOME

In the previous tutorial, I have explained the Integration of the Allure Report with Selenium and JUnit4. In this tutorial, I will explain how to Integrate Allure Report with Cucumber5 and JUnit4.

Below example covers the implementation of Allure Reports in Selenium using JUnit4, Java and Maven.

Pre-Requisite

  1. Java 11 installed
  2. Maven installed
  3. Eclipse or IntelliJ installed

This framework consists of:

  1. Selenium – 3.141.59
  2. Java 11
  3. Cucumber 5 – 5.7.0
  4. Maven – 3.8.1
  5. Allure Report – 2.14.0
  6. Allure Cucumber5 – 2.14.0
  7. Allure JUnit4 – 2.14.0
  8. Aspectj – 1.9.6

Implementation Steps

  1. Update Properties section in Maven pom.xml
  2. Add Cucumber5, Selenium, JUnit4, Allure Cucumber5 and Allure-JUnit4 dependencies in POM.xml
  3. Update Build Section of pom.xml in Allure Report Project.
  4. Create source folder – src/test/resources and features folder within src/test/resources to create test scenarios in Feature file
  5. Create the Step Definition class or Glue Code
  6. Create a Cucumber Runner class
  7. Run the Test and Generate Allure Report

Step 1 – Update Properties section in Maven pom.xml

 <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <selenium.version>3.141.59</selenium.version> 
    <cucumber.version>5.7.0</cucumber.version>
    <allure.cucumber5.version>2.14.0</allure.cucumber5.version>
    <allure.junit4.version>2.14.0</allure.junit4.version>
    <maven.compiler.plugin.version>3.5.1</maven.compiler.plugin.version>
    <maven.surefire.plugin.version>3.0.0-M5</maven.surefire.plugin.version>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <aspectj.version>1.9.6</aspectj.version>
    <allure.version>2.14</allure.version>
  </properties>

Step 2 – Add Cucumber5, Selenium, JUnit4, Allure-Cucumber5 and Allure-JUnit4 dependencies in POM.xml

 <dependencies>
     
     <!--Cucumber Dependencies-->     
    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-java</artifactId>
      <version>${cucumber.version}</version>
    </dependency>
    
    <dependency>
      <groupId>io.cucumber</groupId>
      <artifactId>cucumber-junit</artifactId>
      <version>${cucumber.version}</version>
       <scope>test</scope>
    </dependency>
 
   <!--Selenium Dependency-->
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>${selenium.version}</version>
    </dependency>   

   <!--Hamcrest Dependency-->
    <dependency>
      <groupId>org.hamcrest</groupId>
      <artifactId>hamcrest</artifactId>
      <version>2.2</version>
      <scope>test</scope>
    </dependency>
    
   <!--Allure Cucumber Dependency-->     
      <dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-cucumber5-jvm</artifactId>
        <version>${allure.cucumber5.version}</version>
    </dependency>
    
     <!--Allure Reporting Dependency-->     
    <dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-junit4</artifactId>
        <version>${allure.junit4.version}</version>
        <scope>test</scope>
    </dependency>

  </dependencies>

Step 3 – Update Build Section of pom.xml in Allure Report Project

<build>
   <plugins>
  
       <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${maven.compiler.plugin.version}</version>
                <configuration>
                    <source>${maven.compiler.source}</source> 
                    <target>${maven.compiler.target}</target>
                </configuration>
         </plugin>
     
       <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
             <version>${maven.surefire.plugin.version}</version>
            <configuration>
                 <systemPropertyVariables>
                        <allure.results.directory>${project.build.directory}/allure-results</allure.results.directory>
                    </systemPropertyVariables>
                <argLine>
                    -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                    -Dcucumber.options="--plugin io.qameta.allure.cucumber5jvm.AllureCucumber5Jvm"
                </argLine>
                        
            </configuration>
            <dependencies>
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjweaver</artifactId>
                    <version>${aspectj.version}</version>
                </dependency>
            </dependencies>
         </plugin>
       
     </plugins> 
  </build>

Step 4 – Create source folder – src/test/resources and features folder within src/test/resources to create test scenarios in Feature file

Feature file should be saved as an extension of .feature. Add the test scenarios in this feature file. I have added sample test scenarios. In this feature file, I have created a scenario for successful login and one Scenario Outline for failed login. The test scenarios are written in Gherkins language.

@LoginPage @Junit4

@severity=blocker

Feature: Feature - Login to HRM Application 
 
   @ValidCredentials
   Scenario: Scenario - Login with valid credentials 
     
    Given User is on Home page
    When User enters username as "Admin"
    And User enters password as "admin123"
    Then User should be able to login sucessfully
    
   @InvalidCredentials
   Scenario Outline: Scenario -Login with invalid credentials
   
    Given User is on Home page
    When User enters username as "<username>"
    And User enters password as "<password>"
    Then Error message "<message>" should be displayed
    
    Examples:
    |username  |password  |message                         |
    |admin     |admin     |Invalid credentials             |
    |          |admin123  |Username cannot be empty        | 
    |Admin     |          |Password cannot be empty        |
    |          |          |Username can be empty           |

Step 5 – Create the Step Definition class or Glue Code

public class LoginDefinition {

	@Given("User is on Home page")
	public void userOnHomePage() {

		CommonDefinitions.driver.get("https://opensource-demo.orangehrmlive.com/");
	}

	@When("User enters username as {string}")
	public void entersUsername(String userName) throws InterruptedException {

		System.out.println("Username Entered");
		CommonDefinitions.driver.findElement(By.name("txtUsername")).sendKeys(userName);

	}

	@When("User enters password as {string}")
	public void entersPassword(String passWord) throws InterruptedException {

		System.out.println("Password Entered");
		CommonDefinitions.driver.findElement(By.name("txtPassword")).sendKeys(passWord);

		CommonDefinitions.driver.findElement(By.id("btnLogin")).submit();
	}

	@Then("User should be able to login sucessfully")
	public void sucessfulLogin() throws InterruptedException {

		String newPageText = CommonDefinitions.driver.findElement(By.id("welcome")).getText();
		System.out.println("newPageText :" + newPageText);
		assertThat(newPageText, containsString("Welcome"));

	}

	@Then("Error message {string} should be displayed")
	public void unsucessfulLogin(String message) throws InterruptedException {

		String errorMessage = CommonDefinitions.driver.findElement(By.id("spanMessage")).getText();
		System.out.println("Error Message :" + errorMessage);
		Assert.assertEquals(errorMessage, message);

	}

}

CommonDefinitions.java

public class CommonDefinitions {

	protected static WebDriver driver;

	@Before
	public void setup() {
		System.setProperty("webdriver.gecko.driver",
				"C:\\Users\\Vibha\\Software\\geckodriver-v0.26.0-win64\\geckodriver.exe");

		driver = new FirefoxDriver();

		driver.manage().window().maximize();
		driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
	}

	@After
	public void tearDown(Scenario scenario) {
		try {
			String screenshotName = scenario.getName().replaceAll("", "_");
			if (scenario.isFailed()) {
				TakesScreenshot ts = (TakesScreenshot) driver;
				byte[] screenshot = ts.getScreenshotAs(OutputType.BYTES);
				scenario.attach(screenshot, "img/png", screenshotName);
			}
		} catch (Exception e) {
			e.printStackTrace();
		}
		driver.quit();
	}

}

Step 6 – Create a Cucumber Runner class

We need to create a class called Runner class to run the tests. This class will use the JUnit annotation @RunWith(), which tells JUnit what is the test runner class.

import org.junit.runner.RunWith;

import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;

@RunWith(Cucumber.class)
@CucumberOptions(features = {
		"src/test/resources/features/Login.feature" }, glue = "com.example.CucumberJunitAllureReportDemo.definitions", plugin = {
				"pretty", "html:test-output", "json:target/cucumber-report/cucumber.json" })

public class CucumberRunnerTests {

}

Step 7 – Run the Test and Generate Allure Report

To run the tests, use the below command

mvn clean test

In the below image, we can see that one test is failed and four passed out of five tests.

This will create allure-results folder with all the test report. These files will be use to generate Allure Report.

Change current directory to target directory and then the below comand to generate the Allure Report

allure serve

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

It shows detail of all the test steps and the screenshot of the failed test step also as shown below.

Categories in Allure Report

Categories tab gives you the way to create custom defects classification to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: statuses breakdown or severity and duration diagrams.

Timeline in Allure Report

Timeline tab visualizes retrospective of tests execution, allure adaptors collect precise timings of tests, and here on this tab they are arranged accordingly to their sequential or parallel timing structure.

Behaviors of Allure Report

This tab groups test results according to Epic, Feature and Story tags.

Packages in Allure Report

Packages tab represents a tree-like layout of test results, grouped by different packages.

vibssingh

Allure Report with Selenium and JUnit5

HOME

In the previous tutorial, I have explained the Integration of the Allure Report with Selenium and JUnit4. In this tutorial, I will explain how to Integrate Allure Report with Selenium and JUnit5.

Pre-Requisite

  1. Java 11 installed
  2. Maven installed
  3. Eclipse or IntelliJ installed

This framework consists of:

  1. Selenium – 3.141.59
  2. Java 11
  3. JUnit – 4.13.2
  4. Maven – 3.8.1
  5. Allure Report – 2.14.0
  6. Allure JUnit4 – 2.14.0

Implementation Steps

  1. Update Properties section in Maven pom.xml
  2. Add Selenium, JUnit5 and Allure-JUnit5 dependencies in POM.xml
  3. Update Build Section of pom.xml in Allure Report Project.
  4. Create Pages and Test Code for the pages
  5. Run the Test and Generate Allure Report

Structure of Project

Step 1 – Update Properties section in Maven pom.xml

 <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <java.version>11</java.version>
    <selenium.version>3.141.59</selenium.version> 
    <junit.jupiter.version>5.8.0-M1</junit.jupiter.version>
    <junit.platform.version>1.8.0-M1</junit.platform.version>
    <allure.maven.version>2.10.0</allure.maven.version>
    <allure.junit5.version>2.14.0</allure.junit5.version>
    <maven.surefire.plugin.version>3.0.0-M3</maven.surefire.plugin.version>
    <maven.compiler.plugin.version>3.8.1</maven.compiler.plugin.version>
    <aspectj.version>1.9.6</aspectj.version>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
  </properties>

Step 2 – Add Selenium, JUnit5 and Allure-JUnit5 dependencies in POM.xml

<dependencies>
  
      <!--Selenium Dependency-->
      <dependency>
          <groupId>org.seleniumhq.selenium</groupId>
          <artifactId>selenium-java</artifactId>
          <version>${selenium.version}</version>
       </dependency>
  
     <!--JUNIT 5 Dependencies-->
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>${junit.jupiter.version}</version>
        </dependency>
        
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>${junit.jupiter.version}</version>
        </dependency>
        
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-engine</artifactId>
            <version>${junit.platform.version}</version>
        </dependency>
        
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-launcher</artifactId>
            <version>${junit.platform.version}</version>
        </dependency>
        
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-runner</artifactId>
            <version>${junit.platform.version}</version>
        </dependency>
        
        <!--Allure Reporting Dependencies-->
        <dependency>
            <groupId>io.qameta.allure</groupId>
            <artifactId>allure-junit5</artifactId>
            <version>${allure.junit5.version}</version>
        </dependency>
        
  </dependencies>

Step 3 – Update Build Section of pom.xml in Allure Report Project

<build>
        <plugins>
        <plugin>
               
                <artifactId>maven-surefire-plugin</artifactId>
                <version>${maven.surefire.plugin.version}</version>
                <configuration>
                <testFailureIgnore>false</testFailureIgnore>
                        <argLine>
                            -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                        </argLine>
                    <systemProperties>
                        <property>
                            <name>junit.jupiter.extensions.autodetection.enabled</name>
                            <value>true</value>
                        </property>
                    </systemProperties>
                </configuration>
                
                <dependencies>
                   
                    <dependency>
                        <groupId>org.aspectj</groupId>
                        <artifactId>aspectjweaver</artifactId>
                        <version>${aspectj.version}</version>
                    </dependency>
                </dependencies>
                
            </plugin>
            <plugin>
                <groupId>io.qameta.allure</groupId>
                <artifactId>allure-maven</artifactId>
                <version>${allure.maven.version}</version>
                <configuration>
                    <reportVersion>2.4.1</reportVersion>
                </configuration>
            </plugin>
      </plugins>
  </build>

Step 4 – Create Pages and Test Code for the pages

Below is the sample project which uses Selenium and JUnit4 which is used to generate an Allure Report.

We have 2 pages. Below is the code for Login Page which contains all the web elements and methods related to that web elements.

public class LoginPage {

	WebDriver driver;

	By userName = By.name("txtUsername");

	By password = By.name("txtPassword");

	By titleText = By.id("logInPanelHeading");

	By login = By.id("btnLogin");

	By errorMessage = By.id("spanMessage");

	public LoginPage(WebDriver driver) {
		this.driver = driver;
	}

	// Set user name in textbox
	public void setUserName(String strUserName) {
		driver.findElement(userName).sendKeys(strUserName);
	}

	// Set password in password textbox
	public void setPassword(String strPassword) {
		driver.findElement(password).sendKeys(strPassword);
	}

	// Click on login button
	public void clickLogin() {
		driver.findElement(login).click();
	}

	@Step("Verify title of Login Page")
	public void verifyPageTitle() {
		String loginPageTitle = driver.findElement(titleText).getText();
		assertTrue(loginPageTitle.contains("LOGIN Panel"));
	}

	/* Failed Test */
	@Step("Verify error message when invalid credentail is provided")
	public void verifyErrorMessage() {
		String invalidCredentialErrorMessage = driver.findElement(errorMessage).getText();
		assertTrue(invalidCredentialErrorMessage.contains("Incorrect Credentials"));
	}

	@Step("Enter username and password")
	public void login(String strUserName, String strPasword) {

		// Fill user name
		this.setUserName(strUserName);

		// Fill password
		this.setPassword(strPasword);

		// Click Login button
		this.clickLogin();

	}
}

assertTrue() is imported from below JUnit package for assertion.

import static org.junit.jupiter.api.Assertions.assertTrue;

DashboardPage.java

public class DashboardPage {

	WebDriver driver;

	By dashboardPageTitle = By.id("welcome");

	By assignLeaveOption = By.cssSelector(
			"#dashboard-quick-launch-panel-menu_holder > table > tbody > tr > td:nth-child(1) > div > a > span");

	By leaveListOption = By.cssSelector(
			"#dashboard-quick-launch-panel-menu_holder > table > tbody > tr > td:nth-child(2) > div > a > span");

	By timesheetsOption = By.cssSelector(
			"#dashboard-quick-launch-panel-menu_holder > table > tbody > tr > td:nth-child(3) > div > a > span");

	By applyLeaveOption = By.cssSelector(
			"#dashboard-quick-launch-panel-menu_holder > table > tbody > tr > td:nth-child(4) > div > a > span");

	public DashboardPage(WebDriver driver) {
		this.driver = driver;

	}

	@Step("Verify title of Dashboard page")
	public void verifyDashboardPageTitle() {
		String DashboardPageTitle = driver.findElement(dashboardPageTitle).getText();
		assertTrue(DashboardPageTitle.contains("Welcome"));
	}

	@Step("Verify Assign Leave Quick Launch Options on Dashboard page")
	public void verifyAssignLeaveOption() {
		String QuickLaunchOptions = driver.findElement(assignLeaveOption).getText();
		assertTrue(QuickLaunchOptions.contains("Assign Leave"));
	}

	@Step("Verify Leave List Quick Launch Options on Dashboard page")
	public void verifyLeaveListOption() {
		String LeaveListQuickLaunchOption = driver.findElement(leaveListOption).getText();
		assertTrue(LeaveListQuickLaunchOption.contains("Leave List"));
	}

	@Step("Verify Assign Leave Quick Launch Options on Dashboard page")
	public void verifytimesheetsOption() {
		String timesheetsOptionQuickLaunchOption = driver.findElement(timesheetsOption).getText();
		assertTrue(timesheetsOptionQuickLaunchOption.contains("Timesheets"));
	}

	@Step("Verify Leave List Quick Launch Options on Dashboard page")
	public void verifyApplyLeaveOption() {
		String applyLeaveQuickLaunchOptions = driver.findElement(applyLeaveOption).getText();
		assertTrue(applyLeaveQuickLaunchOptions.contains("Apply Leave"));
	}

}

Test Classes related to various Pages

BaseTest.java

public class BaseTest {

	public static WebDriver driver;
	LoginPage objLogin;
	DashboardPage objDashboardPage;

	@Step("Start the application")
	@BeforeEach
	public void setup() {
		System.setProperty("webdriver.gecko.driver",
				"C:\\Users\\Vibha\\Software\\geckodriver-v0.26.0-win64\\geckodriver.exe");
		driver = new FirefoxDriver();
		driver.manage().window().maximize();
		driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
		driver.get("https://opensource-demo.orangehrmlive.com/");
	}

	@Step("Stop the application")
	@AfterEach
	public void close() {
		driver.close();
	}
}

@BeforeEach is used to signal that the annotated method should beexecuted before each @Test, @RepeatedTest, @ParameterizedTest, @TestFactory,and @TestTemplate method in the current test class. It is imported from:-

import org.junit.jupiter.api.BeforeEach;

AfterEach is used to signal that the annotated method should beexecuted after each @Test, @RepeatedTest, @ParameterizedTest, @TestFactory,and @TestTemplate method in the current test class. It is imported from:- 

import org.junit.jupiter.api.AfterEach;

LoginTests.java

@Epic("Web Application Regression Testing using JUnit5")
@Feature("Login Page Tests")
public class LoginTests extends BaseTest {

	LoginPage objLogin;
	DashboardPage objDashboardPage;

	@Severity(SeverityLevel.NORMAL)
	@Test
	@Description("Test Description : Verify the title of Login Page")
	@Story("Title of Login Page")
	public void verifyLoginPage() {

		// Create Login Page object
		objLogin = new LoginPage(driver);

		// Verify login page text
		objLogin.verifyPageTitle();
	}

	@Severity(SeverityLevel.BLOCKER)
	@Test
	@Description("Test Description : Login Test with invalid credentials")
	@Story("Unsuccessful Login to Application")
	public void invalidCredentialTest() {

		// Create Login Page object
		objLogin = new LoginPage(driver);
		objLogin.login("test", "test123");

		// Verify login page text
		objLogin.verifyErrorMessage();

	}

}

DashboardTests.java

package com.example.Junit5AllureReportDemo.tests;

import org.junit.jupiter.api.Test;

import com.example.Junit5AllureReportDemo.pages.DashboardPage;
import com.example.Junit5AllureReportDemo.pages.LoginPage;

import io.qameta.allure.Description;
import io.qameta.allure.Epic;
import io.qameta.allure.Feature;
import io.qameta.allure.Severity;
import io.qameta.allure.SeverityLevel;
import io.qameta.allure.Story;

@Epic("Web Application Regression Testing using JUnit5")
@Feature("Dashboard Page Tests")
public class DashboardTests extends BaseTest {

	LoginPage objLogin;
	DashboardPage objDashboardPage;

	@Severity(SeverityLevel.BLOCKER)
	@Test
	@Description("Test Description : Verify title of Dashboard page")
	@Story("Title of Dashboard Page")
	public void dashboardTitleTest() {

		objLogin = new LoginPage(driver);

		// login to application
		objLogin.login("Admin", "admin123");

		// go the dashboard page
		objDashboardPage = new DashboardPage(driver);

		objDashboardPage.verifyDashboardPageTitle();

	}

	@Severity(SeverityLevel.BLOCKER)
	@Test
	@Description("Test Description : Verify Assign Leave Option in Quick Link Menu")
	@Story("Validation of Assign Leave Option")
	public void assignLeaveOptionTest() {

		objLogin = new LoginPage(driver);

		// login to application
		objLogin.login("Admin", "admin123");

		// go the dashboard page
		objDashboardPage = new DashboardPage(driver);

		objDashboardPage.verifyAssignLeaveOption();

	}

	@Severity(SeverityLevel.BLOCKER)
	@Test
	@Description("Test Description : Verify Apply Leave Option in Quick Link Menu")
	@Story("Validation of Apply Leave Option")
	public void applyLeaveOptionTest() {

		objLogin = new LoginPage(driver);

		// login to application
		objLogin.login("Admin", "admin123");

		// go the dashboard page
		objDashboardPage = new DashboardPage(driver);

		objDashboardPage.verifyApplyLeaveOption();

	}

	@Severity(SeverityLevel.BLOCKER)
	@Test
	@Description("Test Description : Verify Leave List Option in Quick Link Menu")
	@Story("Validation of Leave List Option")
	public void leaveListOptionTest() {

		objLogin = new LoginPage(driver);

		// login to application
		objLogin.login("Admin", "admin123");

		// go the dashboard page
		objDashboardPage = new DashboardPage(driver);

		objDashboardPage.verifyLeaveListOption();

	}

	@Severity(SeverityLevel.BLOCKER)
	@Test
	@Description("Test Description : Verify Timesheets Option in Quick Link Menu")
	@Story("Validation of Timesheets Option")
	public void timesheetsOptionTest() {

		objLogin = new LoginPage(driver);

		// login to application
		objLogin.login("Admin", "admin123");

		// go the dashboard page
		objDashboardPage = new DashboardPage(driver);

		objDashboardPage.verifyTimesheetsOption();

	}

}

Step 5 – Run the Test and Generate Allure Report

To run the tests, use the below command

mvn clean test

In the below image, we can see that one test is failed and six passed out of seven tests.

This will create allure-results folder with all the test report. These files will be use to generate Allure Report.

To create Allure Report, use the below command

allure serve

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

Categories in Allure Report

Categories tab gives you the way to create custom defects classification to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: statuses breakdown or severity and duration diagrams.

Timeline in Allure Report

Timeline tab visualizes retrospective of tests execution, allure adaptors collect precise timings of tests, and here on this tab they are arranged accordingly to their sequential or parallel timing structure.

Behaviors of Allure Report

This tab groups test results according to Epic, Feature and Story tags.

Packages in Allure Report

Packages tab represents a tree-like layout of test results, grouped by different packages.

vibssingh

Integration of Allure Report with Selenium and JUnit4

HOME

In the previous tutorial, I have explained the Integration of the Allure Report with Selenium and TestNG. In this tutorial, I will explain how to Integrate Allure Report with Selenium and JUnit4.

What is Allure Framework?

Allure is an open-source framework designed to create interactive and comprehensive test report by Yandex QA Team.

Below example covers the implementation of Allure Reports in Selenium using JUnit4, Java and Maven.

Pre-Requisite

  1. Java 11 installed
  2. Maven installed
  3. Eclipse or IntelliJ installed

This framework consists of:

  1. Selenium – 3.141.59
  2. Java 11
  3. JUnit – 4.13.2
  4. Maven – 3.8.1
  5. Allure Report – 2.14.0
  6. Allure JUnit4 – 2.14.0

Implementation Steps

  1. Update Properties section in Maven pom.xml
  2. Add Selenium, JUnit4 and Allure-JUnit4 dependencies in POM.xml
  3. Update Build Section of pom.xml in Allure Report Project.
  4. Create Pages and Test Code for the pages
  5. Run the Test and Generate Allure Report

Structure of Project

Step 1 – Update Properties section in Maven pom.xml

 <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <selenium.version>3.141.59</selenium.version>
    <junit.version>4.13.2</junit.version>
    <allure.junit4.version>2.14.0</allure.junit4.version>
    <maven.compiler.plugin.version>3.5.1</maven.compiler.plugin.version>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <aspectj.version>1.9.6</aspectj.version>
    <maven-surefire-plugin-version>3.0.0-M5</maven-surefire-plugin-version>
  </properties>

Step 2 – Add Selenium, JUnit4 and Allure-JUnit4 dependencies in POM.xml

<dependencies>
   <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>${selenium.version}</version>
    </dependency>
    
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>${junit.version}</version>
      <scope>test</scope>
    </dependency>
    
     <dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-junit4</artifactId>
        <version>${allure.junit4.version}</version>
        <scope>test</scope>
    </dependency>   
  </dependencies>

Step 3 – Update Build Section of pom.xml in Allure Report Project

<build>
       
       <plugins>
   <!-- Compiler plug-in -->
  
           <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${maven.compiler.plugin.version}</version>
                <configuration>
                    <source>${maven.compiler.source}</source> <!--For JAVA 8 use 1.8-->
                    <target>${maven.compiler.target}</target> <!--For JAVA 8 use 1.8-->
                </configuration>
            </plugin>
            
     <!-- Added Surefire Plugin configuration to execute tests -->       
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>${maven-surefire-plugin-version}</version>
            <configuration>
                <testFailureIgnore>false</testFailureIgnore>
                <argLine>
                    -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                </argLine>
                <properties>
                    <property>
                        <name>listener</name>
                        <value>io.qameta.allure.junit4.AllureJunit4</value>
                    </property>
                </properties>
            </configuration>
            <dependencies>
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjweaver</artifactId>
                    <version>${aspectj.version}</version>
                </dependency>
            </dependencies>
        </plugin>
      </plugins>
  </build>

Step 4 – Create Pages and Test Code for the pages

Below is the sample project which uses Selenium and JUnit4 which is used to generate an Allure Report.

We have 2 pages. Below is the code for Login Page which contains all the web elements and methods related to that web elements.

public class LoginPage {

	WebDriver driver;

	By userName = By.name("txtUsername");

	By password = By.name("txtPassword");

	By titleText = By.id("logInPanelHeading");

	By login = By.id("btnLogin");

	By errorMessage = By.id("spanMessage");

	public LoginPage(WebDriver driver) {
		this.driver = driver;
	}

	// Set user name in textbox
	public void setUserName(String strUserName) {
		driver.findElement(userName).sendKeys(strUserName);
	}

	// Set password in password textbox
	public void setPassword(String strPassword) {
		driver.findElement(password).sendKeys(strPassword);
	}

	// Click on login button
	public void clickLogin() {
		driver.findElement(login).click();
	}

	@Step("Verify title of Login Page")
	public void verifyPageTitle() {
		String loginPageTitle = driver.findElement(titleText).getText();
		Assert.assertTrue(loginPageTitle.contains("LOGIN Panel"));
	}

	/* Failed Test */
	@Step("Verify error message when invalid credentail is provided")
	public void verifyErrorMessage() {
		String invalidCredentialErrorMessage = driver.findElement(errorMessage).getText();
		Assert.assertTrue(invalidCredentialErrorMessage.contains("Incorrect Credentials"));
	}

	@Step("Enter username and password")
	public void login(String strUserName, String strPasword) {

		// Fill user name
		this.setUserName(strUserName);

		// Fill password
		this.setPassword(strPasword);

		// Click Login button
		this.clickLogin();

	}
}

DashboardPage.java

public class DashboardPage {

	WebDriver driver;

	By dashboardPageTitle = By.id("welcome");

	By options = By.cssSelector(
			"#dashboard-quick-launch-panel-menu_holder > table > tbody > tr > td:nth-child(1) > div > a > span");

	public DashboardPage(WebDriver driver) {
		this.driver = driver;

	}

	@Step("Verify title of Dashboard page")
	public void verifyDashboardPageTitle() {
		String DashboardPageTitle = driver.findElement(dashboardPageTitle).getText();
		Assert.assertTrue(DashboardPageTitle.contains("Welcome"));
	}

	@Step("Verify Quick Launch Options on Dashboard page")
	public void verifyQuickLaunchOptions() {
		String QuickLaunchOptions = driver.findElement(options).getText();
		Assert.assertTrue(QuickLaunchOptions.contains("Assign Leave"));
	}

}

Test Classes related to various Pages

BaseTest.java

public class BaseTest {

	public static WebDriver driver;
	LoginPage objLogin;
	DashboardPage objDashboardPage;

	@Step("Start the application")
	@Before
	public void setup() {
		System.setProperty("webdriver.gecko.driver",
				"C:\\Users\\Vibha\\Software\\geckodriver-v0.26.0-win64\\geckodriver.exe");
		driver = new FirefoxDriver();
		driver.manage().window().maximize();
		driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
		driver.get("https://opensource-demo.orangehrmlive.com/");
	}

	@Step("Stop the application")
	@After
	public void close() {
		driver.close();
	}
}

LoginTests.java

@Epic("Web Application Regression Testing using JUnit4")
@Feature("Login Page Tests")
@Listeners(TestExecutionListener.class)
public class LoginTests extends BaseTest {

	LoginPage objLogin;
	DashboardPage objDashboardPage;

	@Severity(SeverityLevel.NORMAL)
	@Test(priority = 0, description = "Verify Login Page")
	@Description("Test Description : Verify the title of Login Page")
	@Story("Title of Login Page")
	public void verifyLoginPage() {

		// Create Login Page object
		objLogin = new LoginPage(driver);

		// Verify login page text
		objLogin.verifyPageTitle();
	}

	@Severity(SeverityLevel.BLOCKER)
	@Test(priority = 1, description = "Login with invalid username and password")
	@Description("Test Description : Login Test with invalid credentials")
	@Story("Unsuccessful Login to Application")
	public void invalidCredentialTest() {

		// Create Login Page object
		objLogin = new LoginPage(driver);
		objLogin.login("test", "test123");

		// Verify login page text
		objLogin.verifyErrorMessage();

	}

}

DashboardTests.java

@Epic("Web Application Regression Testing using JUnit4")
@Feature("Dashboard Page Tests")
public class DashboardTests extends BaseTest {

	LoginPage objLogin;
	DashboardPage objDashboardPage;

	@Severity(SeverityLevel.BLOCKER)
	@Test
	@Description("Test Description : After successful login to application opens Dashboard page")
	@Story("Successful login of application opens Dashboard Page")

	public void DasboardTest() {

		objLogin = new LoginPage(driver);

		// login to application
		objLogin.login("Admin", "admin123");

		// go the dashboard page
		objDashboardPage = new DashboardPage(driver);

		// Verify dashboard page
		objDashboardPage.verifyQuickLaunchOptions();

	}

}

Step 5 – Run the Test and Generate Allure Report

To run the tests, use the below command 

mvn clean test

In the below image, we can see that one test is failed and two passed out of three tests.

This will create allure-results folder with all the test report. These files will be use to generate Allure Report.

To create Allure Report, use the below command

allure serve

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

Overview page hosts several default widgets representing basic characteristics of your project and test environment.

  1. Statistics – overall report statistics.
  2. Launches – if this report represents several test launches, statistics per launch will be shown here.
  3. Behaviors – information on results aggregated according to stories and features.
  4. Executors – information on test executors that were used to run the tests.
  5. History Trend – if tests accumulated some historical data, it’s trend will be calculated and shown on the graph.
  6. Environment – information on test environment.

Categories in Allure Report

Categories tab gives you the way to create custom defects classification to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: statuses breakdown or severity and duration diagrams.

Timeline in Allure Report

Timeline tab visualizes retrospective of tests execution, allure adaptors collect precise timings of tests, and here on this tab they are arranged accordingly to their sequential or parallel timing structure.

Behaviors of Allure Report

This tab groups test results according to Epic, Feature and Story tags.

Packages in Allure Report

Packages tab represents a tree-like layout of test results, grouped by different packages.

vibssingh

Integration of Allure Report with Selenium and TestNG

HOME

In this tutorial, I will explain how to integrate Allure Report (one of the very famous Report) with Selenium and TestNG.

What is Allure Framework?

Allure is an open-source framework designed to create interactive and comprehensive test report by Yandex QA Team.

Below example covers the implementation of Allure Reports in Selenium using TestNG, Java and Maven.

Pre-Requisite

  1. Java 11 installed
  2. Maven installed
  3. Eclipse or IntelliJ installed

This framework consists of:

  1. Selenium – 3.141.59
  2. Java 11
  3. TestNG – 7.4.0
  4. Maven – 3.8.1
  5. Allure Report – 2.14.0
  6. Allure TestNG – 2.14.0

Implementation Steps

  1. Update Properties section in Maven pom.xml
  2. Add Selenium, TestNG and Allure TestNG dependencies in POM.xml
  3. Update Build Section of pom.xml in Allure Report Project.
  4. Create Pages and Test Code for the pages
  5. Create testng.xml for the project
  6. Run the Test and Generate Allure Report

Step 1 – Update Properties section

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <selenium.version>3.141.59</selenium.version>
    <testng.version>7.4.0</testng.version>
    <allure.testng.version>2.14.0</allure.testng.version>
    <maven.compiler.plugin.version>3.5.1</maven.compiler.plugin.version>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <aspectj.version>1.9.6</aspectj.version>
    <maven.surefire.plugin.version>3.0.0-M5</maven.surefire.plugin.version>
  </properties>

Step 2 – Add Selenium, TestNG and Allure TestNG dependencies in POM.xml

<dependencies>
    
     <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-java -->
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>${selenium.version}</version>
    </dependency>
    
    <!-- https://mvnrepository.com/artifact/org.testng/testng -->
    <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>${testng.version}</version>
      <scope>test</scope>
    </dependency>
    
    <!-- https://mvnrepository.com/artifact/io.qameta.allure/allure-testng -->
    <dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-testng</artifactId>
        <version>${allure.testng.version}</version>
        <scope>test</scope>
    </dependency>
  </dependencies>

Step 3 – Update Build Section of pom.xml in Allure Report Project

<build>
       
       <plugins>
   <!-- Compiler plug-in -->
  
           <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${maven.compiler.plugin.version}</version>
                <configuration>
                    <source>${maven.compiler.source}</source> <!--For JAVA 8 use 1.8-->
                    <target>${maven.compiler.target}</target> <!--For JAVA 8 use 1.8-->
                </configuration>
            </plugin>
            
     <!-- Added Surefire Plugin configuration to execute tests -->       
          <plugin>
              <groupId>org.apache.maven.plugins</groupId>
              <artifactId>maven-surefire-plugin</artifactId>
              <version>${maven.surefire.plugin.version}</version>
              <configuration>
                    <suiteXmlFiles>
                        <suiteXmlFile>TestNG.xml</suiteXmlFile>
                    </suiteXmlFiles>
                 <argLine>
                    -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                 </argLine>
             </configuration>          
             <dependencies>
            
            <!-- https://mvnrepository.com/artifact/org.aspectj/aspectjweaver -->
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjweaver</artifactId>
                    <version>${aspectj.version}</version>
                </dependency>
            </dependencies>
        </plugin>
      </plugins>
  </build>

Step 4 – Create Pages and Test Code for the pages

Below is the sample project which uses Selenium and TestNG which is used to generate an Allure Report.

We have 2 pages. Below is the code for Login Page which contains all the web elements and methods related to that web elements.

LoginPage.java

public class LoginPage {

	WebDriver driver;

	By userName = By.name("txtUsername");

	By password = By.name("txtPassword");

	By titleText = By.id("logInPanelHeading");

	By login = By.id("btnLogin");

	By errorMessage = By.id("spanMessage");

	public LoginPage(WebDriver driver) {
		this.driver = driver;
	}

	// Set user name in textbox
	public void setUserName(String strUserName) {
		driver.findElement(userName).sendKeys(strUserName);
	}

	// Set password in password textbox
	public void setPassword(String strPassword) {
		driver.findElement(password).sendKeys(strPassword);
	}

	// Click on login button
	public void clickLogin() {
		driver.findElement(login).click();
	}

	@Step("Verify title of Login Page")
	public void verifyPageTitle() {
		String loginPageTitle = driver.findElement(titleText).getText();
		Assert.assertTrue(loginPageTitle.contains("LOGIN Panel"));
	}

    /* Failed Test */
	@Step("Verify error message when invalid credentail is provided")
	public void verifyErrorMessage() {
		String invalidCredentialErrorMessage = driver.findElement(errorMessage).getText();
		Assert.assertTrue(invalidCredentialErrorMessage.contains("Incorrect Credentials"));
	}

	@Step("Enter username and password")
	public void login(String strUserName, String strPasword) {

		// Fill user name
		this.setUserName(strUserName);

		// Fill password
		this.setPassword(strPasword);

		// Click Login button
		this.clickLogin();

	}
}

Dashboard.java

public class DashboardPage {

	WebDriver driver;

	By dashboardPageTitle = By.id("welcome");

	By options = By.cssSelector(
			"#dashboard-quick-launch-panel-menu_holder > table > tbody > tr > td:nth-child(1) > div > a > span");

	public DashboardPage(WebDriver driver) {
		this.driver = driver;

	}

	@Step("Verify title of Dashboard page")
	public void verifyDashboardPageTitle() {
		String DashboardPageTitle = driver.findElement(dashboardPageTitle).getText();
		Assert.assertTrue(DashboardPageTitle.contains("Welcome"));
	}

	@Step("Verify Quick Launch Options on Dashboard page")
	public void verifyQuickLaunchOptions() {
		String QuickLaunchOptions = driver.findElement(options).getText();
		Assert.assertTrue(QuickLaunchOptions.contains("Assign Leave"));
	}

}

Below are the Test clases for Login Page and Dasboard Page. Here, we have BaseTest Class also which contains the common methods needed by other test pages.

BaseTest.java

public class BaseTest {

	public static WebDriver driver;
	LoginPage objLogin;
	DashboardPage objDashboardPage;

	@Step("Start the application")
	@BeforeMethod
	public void setup() {
		System.setProperty("webdriver.gecko.driver",
				"C:\\Users\\Vibha\\Software\\geckodriver-v0.26.0-win64\\geckodriver.exe");
		driver = new FirefoxDriver();
		driver.manage().window().maximize();
		driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
		driver.get("https://opensource-demo.orangehrmlive.com/");
	}

	@Step("Stop the application")
	@AfterMethod
	public void close() {
		driver.close();
	}
}

LoginTests.java

@Epic("Web Application Regression Testing")
@Feature("Login Page Tests")
@Listeners(TestExecutionListener.class)
public class LoginTests extends BaseTest {

	LoginPage objLogin;
	DashboardPage objDashboardPage;

	@Severity(SeverityLevel.NORMAL)
	@Test(priority = 0, description = "Verify Login Page")
	@Description("Test Description : Verify the title of Login Page")
	@Story("Title of Login Page")
	public void verifyLoginPage() {

		// Create Login Page object
		objLogin = new LoginPage(driver);

		// Verify login page text
		objLogin.verifyPageTitle();
	}

   /* Failed Test */
	@Severity(SeverityLevel.BLOCKER)
	@Test(priority = 1, description = "Login with invalid username and password")
	@Description("Test Description : Login Test with invalid credentials")
	@Story("Unsuccessful Login to Application")
	public void invalidCredentialTest() {

		// Create Login Page object
		objLogin = new LoginPage(driver);
		objLogin.login("test", "test123");

		// Verify login page text
		objLogin.verifyErrorMessage();

	}

}

We can order tests by severity by using @Severity annotation. Click here to know more about other Allure annotations.

DashboardTests.java

@Epic("Web Application Regression Testing")
@Feature("Dashboard Page Tests")
@Listeners(TestExecutionListener.class)
public class DashboardTests extends BaseTest {

	LoginPage objLogin;
	DashboardPage objDashboardPage;

	@Severity(SeverityLevel.BLOCKER)
	@Test(priority = 0, description = "Verify Dashboard Page")
	@Description("Test Description : After successful login to application opens Dashboard page")
	@Story("Successful login of application opens Dashboard Page")

	public void DasboardTest() {

		objLogin = new LoginPage(driver);

		// login to application
		objLogin.login("Admin", "admin123");

		// go the dashboard page
		objDashboardPage = new DashboardPage(driver);

		// Verify dashboard page
		objDashboardPage.verifyQuickLaunchOptions();

	}

}

We can group tests with @Epic, @Feature, and @Stories annotations. Click here to know more about other Allure annotations.

TestExecutionListener.class

We can add attachments to our reports by using @Attachment annotation. It can return String, byte [], etc.  I need to add @Listeners({ TestExecutionListener.class }) declaration at the top of the test classes. Click here to know more about other Allure annotations. 

public class TestExecutionListener implements ITestListener {

	@Attachment(value = "Screenshot of {0}", type = "image/png")
	public byte[] saveScreenshot(String name, WebDriver driver) {
		return (byte[]) ((TakesScreenshot) driver).getScreenshotAs(OutputType.BYTES);
	}

	@Override
	public void onTestFailure(ITestResult result) {
		saveScreenshot(result.getName(), BaseTest.driver);
	}

}

Step 5 – Create testng.xml for the project

TestNG.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name = "Allure Reports">
  <test name = "Login Page Tests">
    <classes>
          <class name = "com.example.TestNGAllureReportDemo.tests.LoginTests"/>
         
          </classes>
          </test> 
    <test name =" Dashboard Tests">   
    <classes> 
          <class name = "com.example.TestNGAllureReportDemo.tests.DashboardTests"/>
          </classes>
    </test>

</suite>

Step 6 – Run the Test and Generate Allure Report

To run the tests, use the below command 

mvn clean test

In the below image, we can see that one test is failed and two passed out of three tests.

To create Allure Report, use the below command

allure serve

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

Overview page hosts several default widgets representing basic characteristics of your project and test environment.

  1. Statistics – overall report statistics.
  2. Launches – if this report represents several test launches, statistics per launch will be shown here.
  3. Behaviors – information on results aggregated according to stories and features.
  4. Executors – information on test executors that were used to run the tests.
  5. History Trend – if tests accumulated some historical data, it’s trend will be calculated and shown on the graph.
  6. Environment – information on test environment.

Categories in Allure Report

Categories tab gives you the way to create custom defects classification to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: statuses breakdown or severity and duration diagrams.

Timeline in Allure Report

Timeline tab visualizes retrospective of tests execution, allure adaptors collect precise timings of tests, and here on this tab they are arranged accordingly to their sequential or parallel timing structure.

Behaviors of Allure Report

This tab groups test results according to Epic, Feature and Story tags.

Packages in Allure Report

Packages tab represents a tree-like layout of test results, grouped by different packages.

If you click on the i (hihlighted tab), it will show the test exection report in the below format.

vibssingh

What is Allure Report?

HOME

Allure Framework is a flexible lightweight multi-language test report tool that not only shows a very concise representation of what have been tested in a neat web report form, but allows everyone participating in the development process to extract maximum of useful information from everyday execution of tests.

How Allure Report is generated?

Allure is based on standard xUnit results output but adds some supplementary data. Any report is generated in two steps. During test execution (first step), a small library called adapter attached to the testing framework saves information about executed tests to XML files. We already provide adapters for popular Java, PHP, Ruby, Python, Scala and C# test frameworks. During report generation (second step), the XML files are transformed to a HTML report. This can be done with a command line tool, a plugin for CI or a build too.

Similarly, when we run our tests, every popular test framework generates junit-style xml report or testng style which will be used by Allure to generate HTML report.

In the below example, we use maven surefire plugin which automatically generates xml test reports and stored in target/surefire-reports. And these XML files are transformed to an HTML report by Allure.

Allure reports has provided adapters for Java, PHP, Ruby, Python, Scala and C# test frameworks.

Allure report has below mention annotation.

@Epic
@Features
@Stories/@Story
We can add Epic, Feature and Stories annotations to the test to describe the behaviour of the test.

@Severity(SeverityLevel.BLOCKER) – @Severity annotation is used in order to prioritize test methods by severity.

@Description(“Regression Testing”) – We can add detailed description for each test method. To add such description use @Description annotation.

@Step – In order to define steps in Java code, you need to annotate the respective methods with @Step annotation. When not specified, the step name is equal to the annotated method name.

@Attachment – An attachment in Java code is simply a method annotated with @Attachment that returns either a String or byte[], which should be added to the report.

@Link – We can link the tests to Defect Tracker or JIRA Ticket.

Below is an example which shows how to use various Allure Annotations in the Test.

@Epic("Web Application Regression Testing")
@Feature("Login Page Tests")
@Listeners(TestExecutionListener.class)
public class LoginTests extends BaseTest {

	LoginPage objLogin;
	DashboardPage objDashboardPage;

	@Severity(SeverityLevel.NORMAL)
	@Test(priority = 0, description = "Verify Login Page")
	@Description("Test Description : Verify the title of Login Page")
	@Story("Title of Login Page")
	public void verifyLoginPage() {

		// Create Login Page object
		objLogin = new LoginPage(driver);

		// Verify login page text
		objLogin.verifyPageTitle();
	}
}

Install Allure

For Windows, Allure is available from the Scoop commandline-installer. 

Set-ExecutionPolicy RemoteSigned -scope CurrentUser

Invoke-Expression (New-Object System.Net.WebClient).DownloadString('https://get.scoop.sh')

To install Allure, download and install Scoop and then execute in the Powershell:

scoop install allure

I have allure installed, so I’m getting a message – ‘allure’ (2.14.0) is already installed.

To check if you have Allure installed or not, please use the below command in command line or Powershell.

allure --version

Sample Allure Report

You can find more information on Allure documentation.

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!