Integration Testing of Springboot with Cucumber and TestNG

HOME

In this tutorial, I am going to build an automation framework to test the Springboot application with Cucumber, Rest Assured, and TestNG.

What is Springboot?

Spring Boot is an open-source micro-framework maintained by a company called Pivotal. It provides Java developers with a platform to get started with an auto-configurable production-grade Spring application. With it, developers can get started quickly without losing time on preparing and configuring their Spring application.

What is Cucumber?

Cucumber is a software tool that supports behavior-driven development (BDD). Cucumber can be defined as a testing framework, driven by plain English. It serves as documentation, automated tests, and development aid – all in one.

This framework consists of:

  1. Springboot – 2.5.2
  2. Cucumber – 7.3.4
  3. Java 11
  4. TestNG – 7.3.4
  5. Maven – 3.8.1
  6. RestAssured – 5.1.1

Steps to setup Cucumber Test Automation Framework for API Testing using Rest-Assured

  1. Add SpringbootTest, Rest-AssuredJUnit, and Cucumber dependencies to the project
  2. Create a source folder src/test/resources and create a feature file under src/test/resources
  3. Create the Step Definition class or Glue Code for the Test Scenario under the src/test/java directory
  4. Create a Cucumber Runner class under the src/test/java directory
  5. Run the tests from Cucumber Test Runner
  6. Run the tests from Command Line
  7. Run the tests from TestNG
  8. Generation of TestNG Reports
  9. Cucumber Report Generation

Below is the structure of a SpringBoot application project

We need the below files to create a SpringBoot Application.

SpringBootRestServiceApplication.java

The Spring Boot Application class is generated with Spring Initializer. This class acts as the launching point for the application.

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class SpringBootRestServiceApplication {

    public static void main(String[] args) {

        SpringApplication.run(SpringBootRestServiceApplication.class, args);
    }
}

Student.java

This is JPA Entity for Student class

import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.Id;
import javax.validation.constraints.NotBlank;
import javax.validation.constraints.NotNull;
import javax.validation.constraints.Size;

@Entity
public class Student {
    @Id
    @GeneratedValue
    private Long id;

    @NotNull
    @Size(min = 4, message = "Name should have atleast 4 characters")
    private String name;

    @NotBlank(message = "passportNumber is mandatory")
    private String passportNumber;

    public Student() {
        super();
    }

    public Student(Long id, String name, String passportNumber) {
        super();
        this.id = id;
        this.name = name;
        this.passportNumber = passportNumber;
    }

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getPassportNumber() {
        return passportNumber;
    }

    public void setPassportNumber(String passportNumber) {
        this.passportNumber = passportNumber;
    }
}

StudentRepository.java 

This is JPA Repository for Student. This is created using Spring Data JpaRepository.

import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;

@Repository
public interface StudentRepository extends JpaRepository<Student, Long>{

}

StudentController.java

Spring Rest Controller exposes all services on the student resource.

import static org.springframework.hateoas.server.mvc.WebMvcLinkBuilder.linkTo;
import static org.springframework.hateoas.server.mvc.WebMvcLinkBuilder.methodOn;
import java.net.URI;
import java.util.List;
import java.util.Optional;
import javax.validation.Valid;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.hateoas.EntityModel;
import org.springframework.hateoas.server.mvc.WebMvcLinkBuilder;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.servlet.support.ServletUriComponentsBuilder;

@RestController
public class StudentController {

    @Autowired
    private StudentRepository studentRepository;

    @GetMapping("/students")
    public List<Student> retrieveAllStudents() {
        return studentRepository.findAll();
    }

    @GetMapping("/students/{id}")
    public EntityModel<Student> retrieveStudent(@PathVariable long id) {
        Optional<Student> student = studentRepository.findById(id);

        if (!student.isPresent())
            throw new StudentNotFoundException("id-" + id);

        EntityModel<Student> resource = EntityModel.of(student.get());

        WebMvcLinkBuilder linkTo = linkTo(methodOn(this.getClass()).retrieveAllStudents());

        resource.add(linkTo.withRel("all-students"));

        return resource;
    }

    @PostMapping("/students")
    public ResponseEntity<Object> createStudent(@Valid @RequestBody Student student) {
        Student savedStudent = studentRepository.save(student);

        URI location = ServletUriComponentsBuilder.fromCurrentRequest().path("/{id}")
                .buildAndExpand(savedStudent.getId()).toUri();

        return ResponseEntity.created(location).build();

    }
}

application.properties

Spring Boot automatically loads the application.properties whenever it starts up. You can de-reference values from the property file in the java code through the environment.

spring.jpa.defer-datasource-initialization=true

data.sql 

Data is loaded from data.sql into the Student table. Spring Boot would execute this script after the tables are created from the entities.

insert into student values(10001,'Annie', 'E1234567');
insert into student values(20001,'John', 'A1234568');
insert into student values(30001,'David','C1232268');
insert into student values(40001,'Amy','D213458');

Test Automation Framework Implementation

Step 1 – Add SpringbootTest, Cucumber, Rest-Assured, and TestNG dependencies to the project (Maven project)

 <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <rest-assured.version>5.1.1</rest-assured.version>
        <cucumber.version>7.3.4</cucumber.version>
    </properties>

<dependencies>
        
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>io.rest-assured</groupId>
            <artifactId>rest-assured</artifactId>
            <version>${rest-assured.version}</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>${cucumber.version}</version>
        </dependency>

        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-testng</artifactId>
            <version>${cucumber.version}</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-spring</artifactId>
            <version>${cucumber.version}</version>
            <scope>test</scope>
        </dependency>

</dependencies>

Step 2 – Create a source folder src/test/resources and create a feature file under src/test/resources

By default, the Maven project has an src/test/java directory only. Create a new Source Folder under src/test with the name of resources. Create a folder name as Features within the src/test/resources directory.

Create a feature file to test the Springboot application. Below is a sample feature file.

Feature: Verify springboot application using Cucumber and TestNG

  @ReceiveUserDetails
  Scenario Outline: Send a valid Request to get user details
    Given I send a request to the URL "/students" to get user details
    Then The response will return status 200 
    And The response contains id <studentID> and names "<studentNames>" and passport_no "<studentPassportNo>"

    Examples:
      |studentID    |studentNames  |studentPassportNo|
      |10001        |Annie         |E1234567         |
      |20001        |John          |A1234568         |
      |30001        |David         |C1232268         |
      |40001        |Amy           |D213458          |
      
   
  @CreateUser
  Scenario: Send a valid Request to create a user 
    Given I send a request to the URL "/students" to create a user with name "Annie" and passportNo "E1234567"
    Then The response will return status 201
    And Resend the request to the URL "/students" and the response returned contains name "Annie" and passport_no "E1234567"

Step 3 – Create the Step Definition class or Glue Code for the Test Scenario under src/test/java

The corresponding step definition file of the above feature file is shown below.

import static io.restassured.RestAssured.given;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.hasItem;
import org.json.JSONObject;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.web.server.LocalServerPort;
import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.spring.CucumberContextConfiguration;
import io.restassured.RestAssured;
import io.restassured.http.ContentType;
import io.restassured.response.ValidatableResponse;
import io.restassured.specification.RequestSpecification;

@CucumberContextConfiguration
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class SpringbootDefinitions {

	private final static String BASE_URI = "http://localhost";

	@LocalServerPort
	private int port;

	private ValidatableResponse validatableResponse, validatableResponse1;

	private void configureRestAssured() {
		RestAssured.baseURI = BASE_URI;
		RestAssured.port = port;
	}

	protected RequestSpecification requestSpecification() {
		configureRestAssured();
		return given();
	}

	@Given("I send a request to the URL {string} to get user details")
	public void getStudentDetails(String endpoint) throws Throwable {
		validatableResponse = requestSpecification().contentType(ContentType.JSON).when().get(endpoint).then();
		System.out.println("RESPONSE :" + validatableResponse.extract().asString());
	}

	@Given("I send a request to the URL {string} to create a user with name {string} and passportNo {string}")
	public void createStudent(String endpoint, String studentName, String studentPassportNumber) throws Throwable {

		JSONObject student = new JSONObject();
		student.put("name", studentName);
		student.put("passportNumber", studentPassportNumber);

		validatableResponse = requestSpecification().contentType(ContentType.JSON).body(student.toString()).when()
				.post(endpoint).then();
		System.out.println("RESPONSE :" + validatableResponse.extract().asString());
	}

	@Then("The response will return status {int}")
	public void verifyStatusCodeResponse(int status) {
		validatableResponse.assertThat().statusCode(equalTo(status));

	}

	@Then("The response contains id {int} and names {string} and passport_no {string}")
	public void verifyResponse(int id, String studentName, String passportNo) {
		validatableResponse.assertThat().body("id", hasItem(id)).body(containsString(studentName))
				.body(containsString(passportNo));

	}

	@Then("Resend the request to the URL {string} and the response returned contains name {string} and passport_no {string}")
	public void verifyNewStudent(String endpoint, String studentName, String passportNo) {

		validatableResponse1 = requestSpecification().contentType(ContentType.JSON).when().get(endpoint).then();
		System.out.println("RESPONSE :" + validatableResponse1.extract().asString());
		validatableResponse1.assertThat().body(containsString(studentName)).body(containsString(passportNo));

	}
}

To make Cucumber aware of your test configuration you can annotate a configuration class on your glue path with @CucumberContextConfiguration and with one of the following annotations: @ContextConfiguration, @ContextHierarchy, or @BootstrapWith.It is imported from:

import io.cucumber.spring.CucumberContextConfiguration;

As we are using SpringBoot, we are annotating the configuration class with @SpringBootTest. It is imported from:

import org.springframework.boot.test.context.SpringBootTest;

By default, @SpringBootTest does not start the webEnvironment to refine further how your tests run. It has several options: MOCK(default), RANDOM_PORT, DEFINED_PORT, NONE.

RANDOM_PORT loads a WebServerApplicationContext and provides a real web environment. The embedded server is started and listens on a random port. LocalServerPort is imported from the package:

import org.springframework.boot.web.server.LocalServerPort;

Step 4 – Create a Cucumber TestNG Runner class under src/test/java

A runner will help us to run the feature file and acts as an interlink between the feature file and StepDefinition Class. The TestRunner should be created within the directory src/test/java.

import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;

@CucumberOptions(features = {"src/test/resources/Features"}, glue = {"com.example.demo.definitions"})
public class CucumberRunnerTests extends AbstractTestNGCucumberTests {

}

The @CucumberOptions annotation is responsible for pointing to the right feature package, configuring the plugin for a better reporting of tests in the console output, and specifying the package where extra glue classes may be found. We use it to load configuration and classes that are shared between tests.

Step 5 – Run the tests from Cucumber Test Runner

You can execute the test script by right-clicking on TestRunner class -> Run As TestNG in Eclipse.

In case you are using IntelliJ, select Run CucumberRunnerTests.

SpringBootTest creates an application context containing all the objects we need for the Integration Testing It, starts the embedded server, creates a web environment, and then enables methods to do Integration testing.

Step 6 – Run the tests from Command Line

Use the below command to run the tests through the command line.

mvn clean test

Step 7 – Run the tests from TestNG

Create a testng.xml in the project as shown below:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name = "Suite1">
  <test name = "SpringBoot Cucumber TestNG Demo">
    <classes>
          <class name = "com.example.demo.runner.CucumberRunnerTests"/>
     </classes>  
   </test>
</suite>

Step 8 – Generation of TestNG Reports

TestNG generates various types of reports under the test-output folder like emailable-report.html, index.html, testng-results.xml.

We are interested in the “emailable-report.html” report. Open “emailable-report.html”, as this is an HTML report, and open it with the browser. The below image shows emailable-report.html.

TestNG also produce “index.html” report, and it resides under test-output folder. The below image shows index.html report.

Step 9 – Cucumber Report Generation

Add cucumber.properties under src/test/resources and add the below instruction in the file.

cucumber.publish.enabled=true

The link to the Cucumber Report is present in the execution status.

Below is the image of the Cucumber Report generated using the Cucumber Service.

Complete Source Code:
Refer to GitHub for the source code.

Congratulations!! We are able to build a test framework to test the SpringBoot application using Cucumber, Rest Assured, and TestNG.

Explicit Wait in Serenity

HOME

In the previous tutorial, I explained the Implicit Wait in Serenity. This tutorial will explain the Explicit Wait in Serenity.

What is Explicit Wait?

The explicit wait is used to wait for a specific web element on the web page for a specified amount of time. You can configure wait time element by element basis.

By default, the explicit wait is for 5 sec, with an interval of 10 ms.

Below is the example where I have created two classes – ExplicitWaitDemo and SynchronizationTests.

ExplicitWaitDemo

@DefaultUrl("http://the-internet.herokuapp.com/dynamic_loading/1")
public class ExplicitWaitDemo extends PageObject {

    //Incorrect XPath
	@FindBy(xpath = "//*[@id='start']/buttons")
	WebElementFacade startButton;

	@FindBy(xpath = "//*[@id='finish']/h4")
	WebElementFacade pageText;

	public void explicitWaitDemo1() throws InterruptedException {

		open();

		startButton.waitUntilClickable().click();

	}
}

SynchronizationTests

@RunWith(SerenityRunner.class)
public class SynchronizationTests {

	ExplicitWaitDemo ewaitDemo;

	@Managed
	WebDriver driver;

	@Test
	public void waitTest1() throws InterruptedException {

		ewaitDemo.explicitWaitDemo1();

	}

}

You can see that Serenity waited for 5 sec, with an interval of 100 ms.

When we need to wait for a web element for a specific amount of time, then the below-mentioned command can be added to serenity.conf.

webdriver {
      wait {
         for {
            timeout = 6000
          
        }  
   } 
}

The same can be added to serenity.properties as shown below.

webdriver.wait.for.timeout = 6000

Now, let us run the same above test. I have used the incorrect XPath for the button. So the test should fail after trying to locate the button for 6 secs.

You can print the explicit wait time by using the method – getWaitForTimeout().

In the below example, I have used the explicit wait as 6 sec and which is also returned by the method – getWaitForTimeout().

@DefaultUrl("http://the-internet.herokuapp.com/dynamic_loading/2")
public class ExplicitWaitDemo extends PageObject {

	@FindBy(xpath = "//*[@id='start']/button")
	WebElementFacade startButton;

	@FindBy(xpath = "//*[@id='finish']/h4")
	WebElementFacade pageText;

	public void explicitWaitDemo1() throws InterruptedException {

		open();

		startButton.click();

		System.out.println("Explicit Time defined for the test (in seconds):" + getWaitForTimeout().toSeconds());

	}
}

The output of the above program is

You can override the value of explicit wait mentioned in the serenity.properties or serenity.conf files. This can be done by using the method – withTimeoutOf(Duration duration).

@DefaultUrl("http://the-internet.herokuapp.com/dynamic_loading/1")
public class ExplicitWaitDemo extends PageObject {

    //Incorrect XPath
	@FindBy(xpath = "//*[@id='start']/buttons")
	WebElementFacade startButton;

	@FindBy(xpath = "//*[@id='finish']/h4")
	WebElementFacade pageText;

	public void explicitWaitDemo1() throws InterruptedException {

		open();

       //Override the value mentioned in serenity.conf for timeout from 6 sec to 8 sec
		startButton.withTimeoutOf(Duration.ofSeconds(8)).click();

	}
}

The output of the above program is

You can also wait for more arbitrary conditions, e.g.

@DefaultUrl("http://the-internet.herokuapp.com/dynamic_loading/2")
public class ExplicitWaitDemo extends PageObject {

	@FindBy(xpath = "//*[@id='start']/button")
	WebElementFacade startButton;

	@FindBy(xpath = "//*[@id='finish']/h4")
	WebElementFacade pageText;

	public void explicitWaitDemo1() throws InterruptedException {

		open();
        startButton.click();
		String expected = waitFor(pageText).getText();
		System.out.println("Value of Page :" + expected);
		Assert.assertEquals("Hello World!", expected);

	}
}

The output of the above program is

You can also specify the timeout for a field. For example, if you wanted to wait for up to 8 seconds for a button to become clickable before clicking on it, you could do the following:

startButton.withTimeoutOf(Duration.ofSeconds(8)).waitUntilClickable().click();

Finally, if a specific element of a PageObject needs to have a bit more time to load, you can use the timeoutInSeconds attribute in the Serenity @FindBy annotation, e.g.

import net.serenitybdd.core.annotations.findby.FindBy;
...
@FindBy(xpath = ("//*[@id='start']/button"), timeoutInSeconds="10"))
public WebElementFacade startButton;

To wait for a specific text on the web page, you can use waitForTextToAppear attribute

waitForTextToAppear("Example 1").waitFor(startButton).click();

There are many other methods that can be used with Explicit Wait.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

Data Driven Tests in Serenity with JUnit

HOME

In the previous tutorial, I have explained the Testing of Web Application using Serenity with JUnit4. In this tutorial, I will explain Data Driven Tests in Serenity with JUnit4. Serenity provides features to support Data Driven tests. Refer this tutorial to know how to setup a Serenity project with JUnit4.

There is a parameterized Test Runner to perform data driven tests in JUnit4.

@RunWith(SerenityParameterizedRunner.class)

This runner is very similar to the JUnit Parameterized test runner. Here, @TestData annotation is used to provide test data to the test, and you can use all of the other Serenity annotations like (@Managed, @Steps, @Title and so on). This test runner will also generate proper serenity reports for the executed tests.

Below is an example of data-driven serenity test. In this test, I have created a Test Class (ParameterizationTests) and Step Class (StepLoginPage) and Action Class (NavigateActions). I am passing a set of incorrect credentials to the Login page and will verify the error message.

Here is the code for ParameterizationTests.

@RunWith(SerenityParameterizedRunner.class)
public class ParameterizationTests {

    private final String userName;
    private final String passWord;
    private final String errorMessage;

    @Managed(options = "--headless")
    WebDriver driver;

    @Steps
    NavigateActions navigate;

    @Steps
    StepLoginPage loginPage;

    public ParameterizationTests(String userName, String passWord, String errorMessage) {
        super();
        this.userName = userName;
        this.passWord = passWord;
        this.errorMessage = errorMessage;
    }

    @TestData(columnNames = "Username, Password, ErrorMessage")
    public static Collection<Object[]> testData() {
        return Arrays.asList(new Object[][] { { "Admin12", "", "Password cannot be empty" },
                { "", "abc12", "Username cannot be empty" }, { "_Admin1", "admin123_", "Invalid credentials" },
                { " ", " ", "Username cannot be empty" } });
    }

    @Qualifier
     public String qualifier(){return " - " + " Username = " + userName + " and " + " Password = " + passWord + " should display " + errorMessage;}
    @Test
    @Title("Login to application with invalid credential generates error message")
    public void unsuccessfulLogin() {

        // Given
        navigate.toTheHomePage();

        // When
        loginPage.inputUserName(userName);
        loginPage.inputPassword(passWord);
        loginPage.clickLogin();

        // Then
        Serenity.reportThat("Passing invalid credentials generates error message",
                () -> assertThat(loginPage.loginPageErrorMessage()).isEqualToIgnoringCase(errorMessage));
    }

}

@TestData is the annotation for a method which provides parameters to be injected into the test class constructor by Parameterized. testData() method returns an array list of objects as shown above.

The test data is injected into member variables – userName and passWord. These values are represented as instance variables in the test class, and instantiated via the constructor. These member variables are used in the test.

@Managed is annotated as a WebDriver field that is managed by the Test Runner. The Serenity Test Runner will instantiate this WebDriver before the tests start, and close it once they have all finished.

Here is the code for the StepLoginPage.

public class StepLoginPage extends PageObject {

    @FindBy(name = "txtUsername")
    WebElementFacade username;

    @FindBy(name = "txtPassword")
    WebElementFacade txtPassword;

    @FindBy(name = "Submit")
    WebElementFacade submitButton;

    @FindBy(id = "spanMessage")
    WebElementFacade errorMessage;

    @FindBy(xpath = "//*[@id='forgotPasswordLink']/a")
    WebElementFacade forgotPasswordLinkText;

    @Step("Enter Username")
    public void inputUserName(String userName) {
        $("[name='txtUsername']").sendKeys((userName));
    }

    @Step("Enter Password")
    public void inputPassword(String passWord) {
        txtPassword.sendKeys((passWord));
    }

    @Step("Click Submit Button")
    public void clickLogin() {
        submitButton.click();
    }

    @Step("Error Message on unsuccessful login")
    public String loginPageErrorMessage() {
        return errorMessage.getText();
    }

    @Step("Click Forget Password Link")
    public void clickForgetPasswordLink() {
        forgotPasswordLinkText.click();
    }
}

NavigateActions

public class NavigateActions extends UIInteractionSteps {

    @Step
    public void toTheHomePage() {
        openPageNamed("loginForm");
    }
}

There are two ways to run the tests.

  1. Run the tests as JUnit Tests. Right click on the test and select Run As ->JUnit Test.

2. Run the tests through command line using below command.

mvn clean verify

This will run the tests as well as generate the test execution reports – Index.html and serenity-emailable.html.

So, the tests are run and the reports are generated at the shown path.

Index.html

The heading of parameters present in the Serenity Report (Index.html) like Username, Password and Error Message are generated by @TestData as shown below:

@TestData(columnNames = "Username, Password, ErrorMessage")

The description of Test Step in the Serenity Report is modified by using @Qualifier.

It is used to mark a method as a qualifier in an instantiated data-driven test case.

  @Qualifier
    public String qualifier(){return " - " + " Username = " + userName + " and " + " Password = " + passWord + " should display " + errorMessage;}

Serenity-Summary.html

It is a single-page, self-contained HTML summary report, containing an overview of the test results, and a configurable breakdown of the status of different areas of the application.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

How to generate Serenity Report in customized path

HOME

In the previous tutorial, I explained the generation of Serenity Report. In this tutorial, I will explain how to generate the Serenity Report in Customized path.

Before going through this tutorial, please refer to the tutorial for Serenity Report Generation.

Add outputDirectory to serenity-maven-plugin and mention the path where we want to save our Reports.

 <plugin>
       <groupId>net.serenity-bdd.maven.plugins</groupId>
       <artifactId>serenity-maven-plugin</artifactId>
       <version>${serenity.version}</version>
       <dependencies> 
            <dependency>
                 <groupId>net.serenity-bdd</groupId>
                 <artifactId>serenity-single-page-report</artifactId>
                 <version>${serenity.version}</version>
             </dependency>                
         </dependencies>
         <configuration>
               <tags>${tags}</tags>
               <reports>single-page-html</reports> 
         </configuration>
         <executions>
             <execution>
                  <id>serenity-reports</id>
                  <phase>post-integration-test</phase>
                  <goals>
                       <goal>aggregate</goal>
                  </goals>
                  <configuration>
                            <outputDirectory>C:\\Users\\Vibha\\Projects\\Vibha_Personal\\DetailedReport</outputDirectory>
                   </configuration>
              </execution>
        </executions>
  </plugin>
  .........

Execute the tests using the command line

mvn clean verify

You can see that the reports are generated at the above-specified path.

Another way is to add outputDirectory detail in serenity.properties file.

serenity.project.name = Serenity and Cucumber Report Demo
serenity.outputDirectory=C:\\Users\\Reports\\SerenityReports\\SummaryReport

How to create a Serenity Report for specified tests?

Suppose we want to run a set of tests, not a complete test suite, and we want to get the report containing the details of only executed tests, in the short a very specific report. This can be achieved by using @tags.

 Suppose you mark each test suite with a tag @E2E. So to run only the tests for the @E2E, you could run the following:

mvn clean verify -Dtags="E2E"

You will also need to configure the serenity-maven-plugin to use the tags you provide at the command line:

 <plugin>
     <groupId>net.serenity-bdd.maven.plugins</groupId>
     <artifactId>serenity-maven-plugin</artifactId>
     <version>${serenity.version}</version>
     <dependencies> 
        <dependency>
            <groupId>net.serenity-bdd</groupId>
            <artifactId>serenity-single-page-report</artifactId>
            <version>${serenity.version}</version>
        </dependency>                
      </dependencies>
        <configuration>
             <tags>${tags}</tags>
             <reports>single-page-html</reports> 
        </configuration>
        ......... 

When you run the tests with this configuration, you will get a test report with only the tests related to the @E2E tag. 

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

Manual Tests in Serenity with JUnit5

HOME

In this tutorial, I will explain about Manual Tests in Serenity JUnit5.

You can annotate @Test not @Steps as @Manual.

In contrast to Junit4 a test method annotated with @Manual will actually be executed. This allows to further specify the example using @Step methods and show them the report.

Below is an example where tests are annotated with @Manual with description.

@SerenityTest
public class LoginTests {

	@Managed
	WebDriver driver;

	@Steps
	StepLoginPage loginPage;

	@Steps
	StepDashboardPage dashPage;

	@Steps
	StepForgetPasswordPage forgetpasswordPage;

	@Test
	@Title("Login to application should be successful")
	public void sucessfulLogin() {

		// Given
		loginPage.open();

		// When
		loginPage.inputUserName("Admin");
		loginPage.inputPassword("admin123");
		loginPage.clickLogin();

		// Then
		dashPage.loginVerify();
	}

	@Test
	@Title("Login to application should be unsuccessful with error message")
	public void unsucessfulLogin() throws InterruptedException {

		// Given
		loginPage.open();

		// When
		loginPage.inputUserName("abc");
		loginPage.inputPassword("abc12");
		loginPage.clickLogin();

		// Then
		String actualErrorMessage = loginPage.errorMessage();
		assertEquals("Invalid credentials", actualErrorMessage);
	}

	@Test
	@Manual
	void manualDefault() {
		loginPage.manualStep();
	}

	@Test
	@Manual(result = TestResult.SUCCESS)
	void manualSuccess() {
		loginPage.manualStep();
	}

	@Test
	@Manual(result = TestResult.COMPROMISED)
	void manualCompromised() {
		loginPage.manualStep();
	}

	@Test
	@Manual(result = TestResult.ERROR)
	void manualError() {
		loginPage.manualStep();
	}

	@Test
	@Manual(result = TestResult.ERROR, reason = "A reason for the error")
	void manualErrorWithReason() {
		loginPage.manualStep();
	}

	@Test
	@Manual(result = TestResult.FAILURE)
	void manualFailure() {
		loginPage.manualStep();
	}

	@Test
	@Manual(result = TestResult.IGNORED)
	void manualIgnored() {
		loginPage.manualStep();
	}

	@Test
	@Manual(result = TestResult.PENDING)
	void manualPending() {
		loginPage.manualStep();
	}

	@Test
	@Manual(result = TestResult.SKIPPED)
	void manualSkipped() {
		loginPage.manualStep();
	}

	@Test
	@Manual(result = TestResult.UNDEFINED)
	void manualUndefined() {
		loginPage.manualStep();
	}

	@Test
	@Manual(result = TestResult.UNSUCCESSFUL)
	void manualUnsuccessful() {
		loginPage.manualStep();
	}

}

StepLoginPage.java

public class StepLoginPage extends PageObject {

	@FindBy(name = "txtUsername")
	WebElementFacade username;

	@FindBy(name = "txtPassword")
	WebElementFacade password;

	@FindBy(name = "Submit")
	WebElementFacade submitButton;

	@FindBy(id = "spanMessage")
	WebElementFacade errorMessage;

	@FindBy(id = "forgotPasswordLink")
	WebElementFacade linkText;

	@Step("Enter Username")
	public void inputUserName(String userName) {
		username.sendKeys((userName));
	}

	@Step("Enter Password")
	public void inputPassword(String passWord) {
		password.sendKeys((passWord));
	}

	@Step("Click Submit Button")
	public void clickLogin() {
		submitButton.click();
	}

	@Step("Error Message on unsuccessful login")
	public String errorMessage() {
		String actualErrorMessage = errorMessage.getText();
		System.out.println("Actual Error Message :" + actualErrorMessage);
		return actualErrorMessage;
	}

	@Step("Manual Test Step")
	public void manualStep() {

		System.out.println("Verify various status of manual step");

	}

}

StepDashboardPage.java

public class StepDashboardPage extends PageObject {

	@FindBy(id = "welcome")
	WebElementFacade dashboardText;

	@Step("Successful login")
	public void loginVerify() {
		String dashboardTitle = dashboardText.getText();
		assertThat(dashboardTitle, containsString("Welcome"));
	}
}

Execute these tests by using the below command in commandline.

mvn clean verify

There are two automated tests and rest all are Manual tests. We have Manual Test marked as Default, SUCCESS, COMPROMISED, ERROR, FAILURE, IGNORED, PENDING, SKIPPED, UNDEFINED and UNSUCCESSFUL.

The execution status looks like as shown below.

The reports are generated under /target/site/serenity. There are 2 types of Reports are generated – index.html and serenity-summary.html. To know how to generate Serenity Reports, please refer tutorials for index.html and serenity-summary.html.

By default, @manual scenarios are marked as pending in the Serenity reports.

All scenarios highlighted by blue color are Pending ones whereas pink color are Broken ones.

Serenity-Summary.html

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

How to manage screenshots in Serenity Report

HOME

Serenity provides a wide range of options to manage screenshots in the report. By default, Serenity has set option serenity.take.screenshots=BEFORE_AND_AFTER_EACH_STEP, which means the screenshot is saved before and after each step as shown in the below image. Before this tutorial, refer to the previous tutorial on How to generate Serenity Report.

However, recording many screenshots can slow down test execution. So, maybe we like to record the screenshot of failed steps in the scenario. To achieve this flexibility, configure serenity.take.screenshots property in serenity.properties file.

There are various other types of options for managing screenshots in Serenity Report. This property can take the following values:

  1. FOR_EACH_ACTION: Saves a screenshot at every web element action (like click(), typeAndEnter(), type(), typeAndTab() etc.).
  2. BEFORE_AND_AFTER_EACH_STEP: Saves a screenshot before and after every step.
  3. AFTER_EACH_STEP: Saves a screenshot after every step
  4. FOR_FAILURES: Saves screenshots only for failing steps.
  5. DISABLED: Doesn’t save screenshots for any steps.

In the below option, I have used FOR_FAILURES option in the serenity.properties file.

serenity.project.name = Serenity and Cucumber Report Demo
current.target.version = sprint-1
serenity.take.screenshots = FOR_FAILURES

Below is the screenshot of the passed test case. We can see that there is no screenshot attached to any of the test steps.

Below is the screenshot of the failed test case. We can see that there is a screenshot attached to the failed test step only, not all the test steps. In below example, it is a scenario outline with four different test data. Out of four, only one set of test data has failed. So, the screenshot is generated for the failed step of that particular test data.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

How to report Manual Tests in Serenity Report

HOME

Cucumber is primarily and traditionally used for automating executable specifications. But with Serenity BDD, you can add special tags to indicate that a scenario represents a manual test case.

You can flag any Cucumber scenario as manual simply by using the @manual tag. In the below example, I have tagged a scenario as “@manual”. The last scenario is tagged as “manual”. By default, @manual scenarios are marked as pending in the Serenity reports.

Feature: Login to HRM  

   @ValidCredentials
   Scenario: Login with valid credentials
   
    Given User is on Home page
    When User enters username as "Admin"
    And User enters password as "admin123"
    Then User should be able to login successfully
    
    @InValidCredentials    
    Scenario Outline: Login with invalid credentials
   
    Given User is on Home page
    When User enters username as '<username>'
    And User enters password as '<password>'
    Then User should be able to see error message '<errorMessage>'
      
   Examples:
    |username  |password  |errorMessage                    |
    |admin     |admin     |Invalid credentials             |
    |          |admin123  |Username cannot be empty        | 
    |Admin     |          |Password cannot be empty        |
    |          |          |Username can be empty        |
 
   @ForgetPassword  
   Scenario: Verify Forget Password Functionality
   
    Given User is on Home page
    When User clicks on Forgot your password link
    Then User should be able to see new page which contains Reset Password button
   
   @manual
   Scenario: Verify credentials present in Master Database not older than 30 days
   
    Given User is connected to Master Database
    Then Username "Admin" and password "admin123" are present in Master Database not older than 30 days

Execute the test suite using below command

mvn clean verify

The scenario marked with @manual tag will now appear as a Manual test case in the Serenity report (Index.html). To know how to create Serenity Report, click here.

We can indicate a different result by adding the @manual-result tag as shown here:

A passing test: @manual-result:passed
A failing test: @manual-result:failed
A compromised test: @manual-result:compromised

If we want to record the result of a manual test, we should include both the @manual and the @manual-result tags.

   @manual
   @manual-result:passed
   Scenario: Verify credentials present in Master Database not older than 30 days
   
   Given User is connected to Master Database
   Then Username "Admin" and password "admin123" are present in Master Database not older than 30 days
    
   @manual
   @manual-result:failed
   Scenario: Verify different credentials are provided to Admin, Dev and QA to access Master Database
   
   Given User is connected to Master Database
   Then Different credentials are provided to Admin, Business, Dev and QA to access Master Database

This image shows that there are 2 manual tests. I have marked one manual test as passed and another one as failed which is clearly shown in this image.

How to update Manul Test Results

In the below example, we are considering that the team is working on Sprint-1. We have executed the manual tests and marked the status in the feature file as shown below.

  @manual
  @manual-result:passed
  @manual-last-tested:sprint-1
  Scenario: Verify credentials present in Master Database not older than 30 days
   
  Given User is connected to Master Database
  Then Username "Admin" and password "admin123" are present in Master Database not older than 30 days
    
  @manual
  @manual-result:failed
  @manual-last-tested:sprint-1
  Scenario: Verify different credentials are provided to Admin, Dev and QA to access Master Database
   
  Given User is connected to Master Database
  Then Different credentials are provided to Admin, Business, Dev and QA to access Master Database

In the Serenity properties , the team also records the current version (or sprint number):

serenity.project.name = Serenity and Cucumber Report Demo
current.target.version = sprint-1

Now, execute the feature file. This is how the report look like.

Now, we are in next sprint. Update the value of current.target.version in serenity.properties file.

serenity.project.name = Serenity and Cucumber Report Demo
current.target.version = sprint-2

Now, when the manual scenario is processed, it will be marked as pending, with a note indicating that a new manual test is required:

Both the maual tests which were marked as pass and fail are now pending tests as shown in the image.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

Serenity Emailable HTML Report

HOME

In the previous tutorial, I explained the Generation of Serenity Report (index.html) using Cucumber6 and JUnit. Index.html report that acts both as a test report and living documentation for the product. It has various views like Overall Test Status, Requirement View, Capabilities View, and Features View.

Sometimes it is useful to be able to send a short summary of the test outcomes via email. Serenity allows us to generate a single-page, self-contained HTML summary report, containing an overview of the test results, and a configurable breakdown of the status of different areas of the application. 

Pre-Requisite

  1. Java 11 installed
  2. Maven installed
  3. Eclipse or IntelliJ installed

This framework consists of:

  1. Java 11
  2. Maven – 3.8.1
  3. Serenity – 2.6.0
  4. Serenity Maven – 2.6.0
  5. Serenity Cucumber6 – 2.6.0
  6. JUnit – 4.13.2
  7. Maven Surefire Plugin – 3.0.0-M5
  8. Maven Failsafe Plugin – 3.0.0-M5
  9. Maven Comiler Plugin – 3.8.1

Implementation Steps

  1. Update Properties section in Maven pom.xml
  2. Add repositories and pluginRepository to Maven pom.xml
  3. Add Serenity, Serenity Cucumber and JUnit dependencies to POM.xml
  4. Update Build Section of pom.xml
  5. Create source folder – src/test/resources and features folder within src/test/resources to create test scenarios in Feature file
  6. Create the Step Definition class or Glue Code
  7. Create a Serenity-Cucumber Runner class
  8. Create serenity.conf file under src/test/resources
  9. Create serenity.properties file in the root of the project
  10. Run the tests through commandline which generates Serenity Report

To know about Step 1 to 3, please refer here. These steps are the same for Index.html report and emailable report.

Now, add the below-mentioned plugin. These reports are configured in the Serenity Maven plugin, where you need to do two things. First, you need to add a dependency for the serenity-emailer module in the plugin configuration. Then, you need to tell Serenity to generate the email report when it performs the aggregation task.

<plugin>
    <groupId>net.serenity-bdd.maven.plugins</groupId>
    <artifactId>serenity-maven-plugin</artifactId>
    <version>${serenity.version}</version>
    <dependencies> 
        <dependency>
            <groupId>net.serenity-bdd</groupId>
            <artifactId>serenity-single-page-report</artifactId>
            <version>${serenity.version}</version>
        </dependency>
        <dependency>
            <groupId>net.serenity-bdd</groupId>
            <artifactId>serenity-navigator-report</artifactId>
            <version>${serenity.version}</version>
        </dependency>
    </dependencies>
    <configuration>
        <tags>${tags}</tags>
        <reports>single-page-html,navigator</reports> 
    </configuration>
    <executions>
        <execution>
            <id>serenity-reports</id>
            <phase>post-integration-test</phase>
            <goals>
                <goal>aggregate</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Step 10 – Run the tests through commandline which generates Serenity Report

Open the command line and go to the location where pom.xml of the project is present and type the below command.

mvn verify -Dwebdriver.gecko.driver="C:\\Users\\Vibha\\Software\\geckodriver-v0.26.0-win64\\geckodriver.exe"

I have provided the location of Firefoxdriver through the command line. I believe this is the best way to run the test. We can hard-code the path in the test code or in serenity.conf file. In that case, you don’t need to provide the location of Firefoxdriver through command line. You can use the below command.

mvn verify

The output of the above program is

This image shows that two different types of reports are generated by Serenity – Full Report (index.html) and Single Page HTML Summary ( serenity-summary.html ).

This emailable report is called serenity-summary.html. This is generated under site/serenity/ serenity-summary.html

You can see a sample of such a report here:

As you can see in the above execution status, out of six tests, one test failed. The same information is displayed in the report.

This report provides a summary of the test execution.

The Functional Coverage section lets us highlight key areas of your application. By default, this section will list test results for each Feature. But we can configure the report to group results by other tags as well.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

Allure Report with Selenium and JUnit5

HOME

In the previous tutorial, I explained the Integration of the Allure Report with Selenium and JUnit4. In this tutorial, I will explain how to Integrate Allure Report with Selenium and JUnit5.

Prerequisite

  1. Java 11 is installed
  2. Maven is installed
  3. Eclipse or IntelliJ is installed
  4. Allure is installed

Dependency List:

  1. Selenium – 3.141.59
  2. Java 11
  3. JUnit – 4.13.2
  4. Maven – 3.8.1
  5. Allure Report – 2.14.0
  6. Allure JUnit4 – 2.14.0

Structure of Project

Step 1 – Update Properties section in Maven pom.xml

 <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <java.version>11</java.version>
    <selenium.version>3.141.59</selenium.version> 
    <junit.jupiter.version>5.8.0-M1</junit.jupiter.version>
    <junit.platform.version>1.8.0-M1</junit.platform.version>
    <allure.maven.version>2.10.0</allure.maven.version>
    <allure.junit5.version>2.14.0</allure.junit5.version>
    <maven.surefire.plugin.version>3.0.0-M3</maven.surefire.plugin.version>
    <maven.compiler.plugin.version>3.8.1</maven.compiler.plugin.version>
    <aspectj.version>1.9.6</aspectj.version>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
  </properties>

Step 2 – Add Selenium, JUnit5, and Allure-JUnit5 dependencies in POM.xml

<dependencies>
  
      <!--Selenium Dependency-->
      <dependency>
          <groupId>org.seleniumhq.selenium</groupId>
          <artifactId>selenium-java</artifactId>
          <version>${selenium.version}</version>
       </dependency>
  
     <!--JUNIT 5 Dependencies-->
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>${junit.jupiter.version}</version>
        </dependency>
        
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>${junit.jupiter.version}</version>
        </dependency>
        
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-engine</artifactId>
            <version>${junit.platform.version}</version>
        </dependency>
        
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-launcher</artifactId>
            <version>${junit.platform.version}</version>
        </dependency>
        
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-runner</artifactId>
            <version>${junit.platform.version}</version>
        </dependency>
        
        <!--Allure Reporting Dependencies-->
        <dependency>
            <groupId>io.qameta.allure</groupId>
            <artifactId>allure-junit5</artifactId>
            <version>${allure.junit5.version}</version>
        </dependency>
        
  </dependencies>

Step 3 – Update the Build Section of pom.xml in Allure Report Project

<build>
        <plugins>
        <plugin>
               
                <artifactId>maven-surefire-plugin</artifactId>
                <version>${maven.surefire.plugin.version}</version>
                <configuration>
                <testFailureIgnore>false</testFailureIgnore>
                        <argLine>
                            -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                        </argLine>
                    <systemProperties>
                        <property>
                            <name>junit.jupiter.extensions.autodetection.enabled</name>
                            <value>true</value>
                        </property>
                    </systemProperties>
                </configuration>
                
                <dependencies>
                   
                    <dependency>
                        <groupId>org.aspectj</groupId>
                        <artifactId>aspectjweaver</artifactId>
                        <version>${aspectj.version}</version>
                    </dependency>
                </dependencies>
                
            </plugin>
            <plugin>
                <groupId>io.qameta.allure</groupId>
                <artifactId>allure-maven</artifactId>
                <version>${allure.maven.version}</version>
                <configuration>
                    <reportVersion>2.4.1</reportVersion>
                </configuration>
            </plugin>
      </plugins>
  </build>

Step 4 – Create Pages and Test Code for the pages

Below is the sample project which uses Selenium and JUnit4 which is used to generate an Allure Report.

We have 2 pages. Below is the code for Login Page which contains all the web elements and methods related to that web elements.

Note:- This is a sample code. There could be the probability that XPath would have changed. So, the tests won’t run as expected and please keep this in mind.

public class LoginPage {

	WebDriver driver;

	By userName = By.name("txtUsername");

	By password = By.name("txtPassword");

	By titleText = By.id("logInPanelHeading");

	By login = By.id("btnLogin");

	By errorMessage = By.id("spanMessage");

	public LoginPage(WebDriver driver) {
		this.driver = driver;
	}

	// Set user name in textbox
	public void setUserName(String strUserName) {
		driver.findElement(userName).sendKeys(strUserName);
	}

	// Set password in password textbox
	public void setPassword(String strPassword) {
		driver.findElement(password).sendKeys(strPassword);
	}

	// Click on login button
	public void clickLogin() {
		driver.findElement(login).click();
	}

	@Step("Verify title of Login Page")
	public void verifyPageTitle() {
		String loginPageTitle = driver.findElement(titleText).getText();
		assertTrue(loginPageTitle.contains("LOGIN Panel"));
	}

	/* Failed Test */
	@Step("Verify error message when invalid credentail is provided")
	public void verifyErrorMessage() {
		String invalidCredentialErrorMessage = driver.findElement(errorMessage).getText();
		assertTrue(invalidCredentialErrorMessage.contains("Incorrect Credentials"));
	}

	@Step("Enter username and password")
	public void login(String strUserName, String strPasword) {

		// Fill user name
		this.setUserName(strUserName);

		// Fill password
		this.setPassword(strPasword);

		// Click Login button
		this.clickLogin();

	}
}

assertTrue() is imported from the below JUnit package for assertion.

import static org.junit.jupiter.api.Assertions.assertTrue;

DashboardPage.java

public class DashboardPage {

	WebDriver driver;

	By dashboardPageTitle = By.id("welcome");

	By assignLeaveOption = By.cssSelector(
			"#dashboard-quick-launch-panel-menu_holder > table > tbody > tr > td:nth-child(1) > div > a > span");

	By leaveListOption = By.cssSelector(
			"#dashboard-quick-launch-panel-menu_holder > table > tbody > tr > td:nth-child(2) > div > a > span");

	By timesheetsOption = By.cssSelector(
			"#dashboard-quick-launch-panel-menu_holder > table > tbody > tr > td:nth-child(3) > div > a > span");

	By applyLeaveOption = By.cssSelector(
			"#dashboard-quick-launch-panel-menu_holder > table > tbody > tr > td:nth-child(4) > div > a > span");

	public DashboardPage(WebDriver driver) {
		this.driver = driver;

	}

	@Step("Verify title of Dashboard page")
	public void verifyDashboardPageTitle() {
		String DashboardPageTitle = driver.findElement(dashboardPageTitle).getText();
		assertTrue(DashboardPageTitle.contains("Welcome"));
	}

	@Step("Verify Assign Leave Quick Launch Options on Dashboard page")
	public void verifyAssignLeaveOption() {
		String QuickLaunchOptions = driver.findElement(assignLeaveOption).getText();
		assertTrue(QuickLaunchOptions.contains("Assign Leave"));
	}

	@Step("Verify Leave List Quick Launch Options on Dashboard page")
	public void verifyLeaveListOption() {
		String LeaveListQuickLaunchOption = driver.findElement(leaveListOption).getText();
		assertTrue(LeaveListQuickLaunchOption.contains("Leave List"));
	}

	@Step("Verify Assign Leave Quick Launch Options on Dashboard page")
	public void verifytimesheetsOption() {
		String timesheetsOptionQuickLaunchOption = driver.findElement(timesheetsOption).getText();
		assertTrue(timesheetsOptionQuickLaunchOption.contains("Timesheets"));
	}

	@Step("Verify Leave List Quick Launch Options on Dashboard page")
	public void verifyApplyLeaveOption() {
		String applyLeaveQuickLaunchOptions = driver.findElement(applyLeaveOption).getText();
		assertTrue(applyLeaveQuickLaunchOptions.contains("Apply Leave"));
	}

}


Test Classes related to various Pages

BaseTest.java

public class BaseTest {

	public static WebDriver driver;
	LoginPage objLogin;
	DashboardPage objDashboardPage;

	@Step("Start the application")
	@BeforeEach
	public void setup() {
		System.setProperty("webdriver.gecko.driver",
				"C:\\Users\\Vibha\\Software\\geckodriver-v0.26.0-win64\\geckodriver.exe");
		driver = new FirefoxDriver();
		driver.manage().window().maximize();
		driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
		driver.get("https://opensource-demo.orangehrmlive.com/");
	}

	@Step("Stop the application")
	@AfterEach
	public void close() {
		driver.close();
	}
}

@BeforeEach is used to signal that the annotated method should be executed before each @Test, @RepeatedTest, @ParameterizedTest, @TestFactory, and @TestTemplate method in the current test class. It is imported from:-

import org.junit.jupiter.api.BeforeEach;

@AfterEach is used to signal that the annotated method should be executed after each @Test, @RepeatedTest, @ParameterizedTest, @TestFactory, and @TestTemplate method in the current test class. It is imported from:-

import org.junit.jupiter.api.AfterEach;

LoginTests.java

@Epic("Web Application Regression Testing using JUnit5")
@Feature("Login Page Tests")
public class LoginTests extends BaseTest {

	LoginPage objLogin;
	DashboardPage objDashboardPage;

	@Severity(SeverityLevel.NORMAL)
	@Test
	@Description("Test Description : Verify the title of Login Page")
	@Story("Title of Login Page")
	public void verifyLoginPage() {

		// Create Login Page object
		objLogin = new LoginPage(driver);

		// Verify login page text
		objLogin.verifyPageTitle();
	}

	@Severity(SeverityLevel.BLOCKER)
	@Test
	@Description("Test Description : Login Test with invalid credentials")
	@Story("Unsuccessful Login to Application")
	public void invalidCredentialTest() {

		// Create Login Page object
		objLogin = new LoginPage(driver);
		objLogin.login("test", "test123");

		// Verify login page text
		objLogin.verifyErrorMessage();

	}

}

DashboardTests.java

package com.example.Junit5AllureReportDemo.tests;

import org.junit.jupiter.api.Test;

import com.example.Junit5AllureReportDemo.pages.DashboardPage;
import com.example.Junit5AllureReportDemo.pages.LoginPage;

import io.qameta.allure.Description;
import io.qameta.allure.Epic;
import io.qameta.allure.Feature;
import io.qameta.allure.Severity;
import io.qameta.allure.SeverityLevel;
import io.qameta.allure.Story;

@Epic("Web Application Regression Testing using JUnit5")
@Feature("Dashboard Page Tests")
public class DashboardTests extends BaseTest {

	LoginPage objLogin;
	DashboardPage objDashboardPage;

	@Severity(SeverityLevel.BLOCKER)
	@Test
	@Description("Test Description : Verify title of Dashboard page")
	@Story("Title of Dashboard Page")
	public void dashboardTitleTest() {

		objLogin = new LoginPage(driver);

		// login to application
		objLogin.login("Admin", "admin123");

		// go the dashboard page
		objDashboardPage = new DashboardPage(driver);

		objDashboardPage.verifyDashboardPageTitle();

	}

	@Severity(SeverityLevel.BLOCKER)
	@Test
	@Description("Test Description : Verify Assign Leave Option in Quick Link Menu")
	@Story("Validation of Assign Leave Option")
	public void assignLeaveOptionTest() {

		objLogin = new LoginPage(driver);

		// login to application
		objLogin.login("Admin", "admin123");

		// go the dashboard page
		objDashboardPage = new DashboardPage(driver);

		objDashboardPage.verifyAssignLeaveOption();

	}

	@Severity(SeverityLevel.BLOCKER)
	@Test
	@Description("Test Description : Verify Apply Leave Option in Quick Link Menu")
	@Story("Validation of Apply Leave Option")
	public void applyLeaveOptionTest() {

		objLogin = new LoginPage(driver);

		// login to application
		objLogin.login("Admin", "admin123");

		// go the dashboard page
		objDashboardPage = new DashboardPage(driver);

		objDashboardPage.verifyApplyLeaveOption();

	}

	@Severity(SeverityLevel.BLOCKER)
	@Test
	@Description("Test Description : Verify Leave List Option in Quick Link Menu")
	@Story("Validation of Leave List Option")
	public void leaveListOptionTest() {

		objLogin = new LoginPage(driver);

		// login to application
		objLogin.login("Admin", "admin123");

		// go the dashboard page
		objDashboardPage = new DashboardPage(driver);

		objDashboardPage.verifyLeaveListOption();

	}

	@Severity(SeverityLevel.BLOCKER)
	@Test
	@Description("Test Description : Verify Timesheets Option in Quick Link Menu")
	@Story("Validation of Timesheets Option")
	public void timesheetsOptionTest() {

		objLogin = new LoginPage(driver);

		// login to application
		objLogin.login("Admin", "admin123");

		// go the dashboard page
		objDashboardPage = new DashboardPage(driver);

		objDashboardPage.verifyTimesheetsOption();

	}

}

Step 5 – Run the Test and Generate Allure Report

To run the tests, use the below command

mvn clean test

In the below image, we can see that one test failed and six passed out of seven tests.

This will create the allure-results folder with all the test reports. These files will be used to generate Allure Report.

To create Allure Report, use the below command

allure serve

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

Categories in Allure Report

The categories tab gives you a way to create custom defect classifications to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: status breakdown or severity and duration diagrams.

Timeline in Allure Report

Timeline tab visualizes retrospective of tests execution, allure adaptors collect precise timings of tests, and here on this tab they are arranged accordingly to their sequential or parallel timing structure.

Behaviors of Allure Report

This tab groups test results according to Epic, Feature, and Story tags.

Packages in Allure Report

Packages tab represents a tree-like layout of test results, grouped by different packages.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

Integration of Allure Report with Selenium and JUnit4

HOME

In the previous tutorial, I have explained the Integration of the Allure Report with Selenium and TestNG. In this tutorial, I will explain how to Integrate Allure Report with Selenium and JUnit4.

What is Allure Framework?

Allure is an open-source framework designed to create interactive and comprehensive test report by Yandex QA Team.

Below example covers the implementation of Allure Reports in Selenium using JUnit4, Java and Maven.

Pre-Requisite

  1. Java 11 installed
  2. Maven installed
  3. Eclipse or IntelliJ installed

This framework consists of:

  1. Selenium – 3.141.59
  2. Java 11
  3. JUnit – 4.13.2
  4. Maven – 3.8.1
  5. Allure Report – 2.14.0
  6. Allure JUnit4 – 2.14.0

Implementation Steps

  1. Update Properties section in Maven pom.xml
  2. Add Selenium, JUnit4 and Allure-JUnit4 dependencies in POM.xml
  3. Update Build Section of pom.xml in Allure Report Project.
  4. Create Pages and Test Code for the pages
  5. Run the Test and Generate Allure Report

Structure of Project

Step 1 – Update Properties section in Maven pom.xml

 <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <selenium.version>3.141.59</selenium.version>
    <junit.version>4.13.2</junit.version>
    <allure.junit4.version>2.14.0</allure.junit4.version>
    <maven.compiler.plugin.version>3.5.1</maven.compiler.plugin.version>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <aspectj.version>1.9.6</aspectj.version>
    <maven-surefire-plugin-version>3.0.0-M5</maven-surefire-plugin-version>
  </properties>

Step 2 – Add Selenium, JUnit4 and Allure-JUnit4 dependencies in POM.xml

<dependencies>
   <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>${selenium.version}</version>
    </dependency>
    
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>${junit.version}</version>
      <scope>test</scope>
    </dependency>
    
     <dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-junit4</artifactId>
        <version>${allure.junit4.version}</version>
        <scope>test</scope>
    </dependency>   
  </dependencies>

Step 3 – Update Build Section of pom.xml in Allure Report Project

<build>
       
       <plugins>
   <!-- Compiler plug-in -->
  
           <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${maven.compiler.plugin.version}</version>
                <configuration>
                    <source>${maven.compiler.source}</source> <!--For JAVA 8 use 1.8-->
                    <target>${maven.compiler.target}</target> <!--For JAVA 8 use 1.8-->
                </configuration>
            </plugin>
            
     <!-- Added Surefire Plugin configuration to execute tests -->       
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>${maven-surefire-plugin-version}</version>
            <configuration>
                <testFailureIgnore>false</testFailureIgnore>
                <argLine>
                    -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                </argLine>
                <properties>
                    <property>
                        <name>listener</name>
                        <value>io.qameta.allure.junit4.AllureJunit4</value>
                    </property>
                </properties>
            </configuration>
            <dependencies>
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjweaver</artifactId>
                    <version>${aspectj.version}</version>
                </dependency>
            </dependencies>
        </plugin>
      </plugins>
  </build>

Step 4 – Create Pages and Test Code for the pages

Below is the sample project which uses Selenium and JUnit4 which is used to generate an Allure Report.

We have 2 pages. Below is the code for Login Page which contains all the web elements and methods related to that web elements.

public class LoginPage {

	WebDriver driver;

	By userName = By.name("txtUsername");

	By password = By.name("txtPassword");

	By titleText = By.id("logInPanelHeading");

	By login = By.id("btnLogin");

	By errorMessage = By.id("spanMessage");

	public LoginPage(WebDriver driver) {
		this.driver = driver;
	}

	// Set user name in textbox
	public void setUserName(String strUserName) {
		driver.findElement(userName).sendKeys(strUserName);
	}

	// Set password in password textbox
	public void setPassword(String strPassword) {
		driver.findElement(password).sendKeys(strPassword);
	}

	// Click on login button
	public void clickLogin() {
		driver.findElement(login).click();
	}

	@Step("Verify title of Login Page")
	public void verifyPageTitle() {
		String loginPageTitle = driver.findElement(titleText).getText();
		Assert.assertTrue(loginPageTitle.contains("LOGIN Panel"));
	}

	/* Failed Test */
	@Step("Verify error message when invalid credentail is provided")
	public void verifyErrorMessage() {
		String invalidCredentialErrorMessage = driver.findElement(errorMessage).getText();
		Assert.assertTrue(invalidCredentialErrorMessage.contains("Incorrect Credentials"));
	}

	@Step("Enter username and password")
	public void login(String strUserName, String strPasword) {

		// Fill user name
		this.setUserName(strUserName);

		// Fill password
		this.setPassword(strPasword);

		// Click Login button
		this.clickLogin();

	}
}

DashboardPage.java

public class DashboardPage {

	WebDriver driver;

	By dashboardPageTitle = By.id("welcome");

	By options = By.cssSelector(
			"#dashboard-quick-launch-panel-menu_holder > table > tbody > tr > td:nth-child(1) > div > a > span");

	public DashboardPage(WebDriver driver) {
		this.driver = driver;

	}

	@Step("Verify title of Dashboard page")
	public void verifyDashboardPageTitle() {
		String DashboardPageTitle = driver.findElement(dashboardPageTitle).getText();
		Assert.assertTrue(DashboardPageTitle.contains("Welcome"));
	}

	@Step("Verify Quick Launch Options on Dashboard page")
	public void verifyQuickLaunchOptions() {
		String QuickLaunchOptions = driver.findElement(options).getText();
		Assert.assertTrue(QuickLaunchOptions.contains("Assign Leave"));
	}

}

Test Classes related to various Pages

BaseTest.java

public class BaseTest {

	public static WebDriver driver;
	LoginPage objLogin;
	DashboardPage objDashboardPage;

	@Step("Start the application")
	@Before
	public void setup() {
		System.setProperty("webdriver.gecko.driver",
				"C:\\Users\\Vibha\\Software\\geckodriver-v0.26.0-win64\\geckodriver.exe");
		driver = new FirefoxDriver();
		driver.manage().window().maximize();
		driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
		driver.get("https://opensource-demo.orangehrmlive.com/");
	}

	@Step("Stop the application")
	@After
	public void close() {
		driver.close();
	}
}

LoginTests.java

@Epic("Web Application Regression Testing using JUnit4")
@Feature("Login Page Tests")
@Listeners(TestExecutionListener.class)
public class LoginTests extends BaseTest {

	LoginPage objLogin;
	DashboardPage objDashboardPage;

	@Severity(SeverityLevel.NORMAL)
	@Test(priority = 0, description = "Verify Login Page")
	@Description("Test Description : Verify the title of Login Page")
	@Story("Title of Login Page")
	public void verifyLoginPage() {

		// Create Login Page object
		objLogin = new LoginPage(driver);

		// Verify login page text
		objLogin.verifyPageTitle();
	}

	@Severity(SeverityLevel.BLOCKER)
	@Test(priority = 1, description = "Login with invalid username and password")
	@Description("Test Description : Login Test with invalid credentials")
	@Story("Unsuccessful Login to Application")
	public void invalidCredentialTest() {

		// Create Login Page object
		objLogin = new LoginPage(driver);
		objLogin.login("test", "test123");

		// Verify login page text
		objLogin.verifyErrorMessage();

	}

}

DashboardTests.java

@Epic("Web Application Regression Testing using JUnit4")
@Feature("Dashboard Page Tests")
public class DashboardTests extends BaseTest {

	LoginPage objLogin;
	DashboardPage objDashboardPage;

	@Severity(SeverityLevel.BLOCKER)
	@Test
	@Description("Test Description : After successful login to application opens Dashboard page")
	@Story("Successful login of application opens Dashboard Page")

	public void DasboardTest() {

		objLogin = new LoginPage(driver);

		// login to application
		objLogin.login("Admin", "admin123");

		// go the dashboard page
		objDashboardPage = new DashboardPage(driver);

		// Verify dashboard page
		objDashboardPage.verifyQuickLaunchOptions();

	}

}

Step 5 – Run the Test and Generate Allure Report

To run the tests, use the below command

mvn clean test

In the below image, we can see that one test is failed and two passed out of three tests.

This will create allure-results folder with all the test report. These files will be use to generate Allure Report.

To create Allure Report, use the below command

allure serve

This will generate the beautiful Allure Test Report as shown below.

Allure Report Dashboard

Overview page hosts several default widgets representing basic characteristics of your project and test environment.

  1. Statistics – overall report statistics.
  2. Launches – if this report represents several test launches, statistics per launch will be shown here.
  3. Behaviors – information on results aggregated according to stories and features.
  4. Executors – information on test executors that were used to run the tests.
  5. History Trend – if tests accumulated some historical data, it’s trend will be calculated and shown on the graph.
  6. Environment – information on test environment.

Categories in Allure Report

Categories tab gives you the way to create custom defects classification to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).

Suites in Allure Report

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.

Graphs in Allure Report

Graphs allow you to see different statistics collected from the test data: statuses breakdown or severity and duration diagrams.

Timeline in Allure Report

Timeline tab visualizes retrospective of tests execution, allure adaptors collect precise timings of tests, and here on this tab they are arranged accordingly to their sequential or parallel timing structure.

Behaviors of Allure Report

This tab groups test results according to Epic, Feature and Story tags.

Packages in Allure Report

Packages tab represents a tree-like layout of test results, grouped by different packages.