TestNG was developed by a famous programmer named “Cedric Beust”. It is distributed under Apache Software License and is easily available to download. TestNG requires JDK 7 or higher. TestNG is a testing framework inspired by JUnit and NUnit, but introduces some new functionalities that make it more powerful and easier to use.
Serenity BDD is an open-source library that aims to make the idea of living documentation a reality. Serenity BDD helps you write cleaner and more maintainable automated acceptance and regression tests faster. Serenity also uses the test results to produce illustrated, narrative reports that document and describe what your application does and how it works. Serenity tells you not only what tests have been executed, but more importantly, what requirements have been tested
BDD is a set of practices that helps to reduce the rework caused by misunderstanding or vague requirements, narrow the communication gaps between the development team, testing team, and customers, and promote continuous communication among them. Cucumber is one such open-source tool, which supports Behaviour Driven Development(BDD). In simple words, Cucumber can be defined as a testing framework, driven by plain English. It serves as documentation, automated tests, and a development aid – all in one.
Cucumber Introduction, Installation, and Configuration
Provide username and password and click on Sign in.
Step 5: Download and Install Maven Plugin
Click on the Manage Jenkins.
Choose Manage Plugins.
Step 6: Add the Maven Integration plugin
On the Plugins Page, go to the Available option
Select the Maven Integration Plugin
Click on Install without restart. The plugin will take a few moments to finish downloading depending on your internet connection, and will be installed automatically.
You can also select the option Download now and Install after the restartbutton. In which plugin is installed after the restart
You will be shown a “No updates available” message if you already have the Maven plugin installed.
The plugin “Maven Integration” has been installed successfully.
Step 7: Restart Jenkins
Click on the checkbox “Restart Jenkins when installation is complete when no jobs are running“.
The Jenkins is being restarted, It is about to restart.
Again, log in to Jenkins UI.
Step 8: Create a new project using the Maven project plugin
Give the Name of the project – SeleniumTestNG_MavenDemo.
Click on the Maven project.
Click on the OK button.
In the General section, enter the project description in the Description box.
Step 9: Build Management
Go to the Buildsection of the new job.
In the Root POMtextbox, enter the full path to pom.xml
In the Goals and options section, enter “clean test”
Click on the Apply and Savebuttons.
We have created a new Maven project “SeleniumTestNG_MavenDemo” with the configuration to run the Selenium with TestNG Tests
Step 10: Execute the tests
Click on the Build Now link. Maven will build the project. It will then have TestNG execute the test cases.
To see the current status of the execution, click on the “console output“.
Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!
Selenium is an open-source tool. It is not a single automation tool like QTP. Instead, it is a suite of software or a set of JAR files. These tools automate web browsers across various browsers. Selenium is used by many companies, but a few to mention are Netflix, Google, HubSpot, Fitbit, Accenture, IBM, and more.
Selenium – Introduction, Installation, Test Script
In the previous tutorial, I discussed the Jenkins pipeline. This tutorial will discuss the steps to create the Jenkins pipeline for Selenium tests. This is an important step in CI/CD.
3. To generate a HTML Report in Jenkins, we need to download the HTML Publisher Plugin. Please refer to this tutorial to install the plugin – How to generate HTML Reports in Jenkins
Implementation Steps
Step 1: Create a new pipeline project
Give the Name of the project – Selenium_PipelineDemo.
Click on the pipeline project.
Click on the OK button.
In the General section, enter the project description in the Description box.
Step 2: Scroll down to Pipeline
From the Definition field, choose the “Pipeline script from SCM” option. This option instructs Jenkins to obtain your Pipeline from Source Control Management (SCM), which will be your locally cloned Git repository.
From the SCM field, choose Git.
The “Repositories” section contains the “Repository URL” and “Credentials“.
In the Repository URL field, specify the directory path of the GitLab/GitHub project.
In the Credentials field, specify the username and password needed to log in to GitLab/GitHub.
In this case, I have the project present in GitLab and using it.
Step 3: Create Jenkinsfile
Create and save a new text file with the name Jenkinsfile at the root of the project in the GitLab repository. Here, we are using the Selenium project with TestNG. To learn more about the Integration of Selenium with TestNG, please refer to this tutorial – Integration of Selenium and TestNG.
For this tutorial, we are using Declarative syntax. The sample example is given below:
Here, I have used emailable-report.html, you can also use index.html and that report will be published.
pipeline {
agent any
stages {
stage('Test') {
steps {
bat "mvn -D clean test"
}
post {
// If Maven was able to run the tests, even if some of the test
// failed, record the test results and archive the jar file.
success {
publishHTML([
allowMissing: false,
alwaysLinkToLastBuild: false,
keepAll: false,
reportDir: 'target/surefire-reports/',
reportFiles: 'emailable-report.html',
reportName: 'HTML Report',
reportTitles: '',
useWrapperFileDirectly: true])
}
}
}
}
}
Step 4: Specify branches to build a section under Repositories
Branch Specifier – */master (This is my main branch)
ScriptPath – Jenkinsfile
Click on the Apply and Save buttons.
We have created a new Maven project “Selenium_PipelineDemo” with the configuration to run the Selenium Test with TestNG.
Step 5: Execute the tests
Let’s execute it now by clicking on the “Build Now” button.
Right-click on Build Number (here in my case it is #11) and click on Console Output to see the result.
Below is the test execution summary.
Step 6: Pipeline Steps
Once the execution is completed, and we want to see the Pipeline Steps, click on the Pipeline Steps mentioned on the left side of the page.
Step 7: View the Report
Once the execution is completed, go back to “Selenium_PipelineDemo”. We can see below that the HTML Report is generated.
We could see a link to view ‘HTML Reports’. Click on the HTML Reports. It displays the emailable-report.html Report.
Tip: If you don’t see the Report UI intact, then you need to configure a simple groovy script. For that, go to Dashboard–>Manage Jenkins–>Script Console and add the script as:
Selenium needs Java to be installed on the system to run the tests. Click here to know How to install Java.
Step 2 – Download and setup Eclipse IDE on the system
The Eclipse IDE (integrated development environment) provides strong support for Java developers. Click here to know How to install Eclipse.
Step 3 – Setup Gradle
To build a test framework, we need to add several dependencies to the project. This can be achieved by any build tool. I have used Gradle Build Tool. Click here to know How to install Gradle.
Step 4 – Create a new Gradle Project
Below are the steps to create the Gradle project from command line.
Step 5 – Add Selenium and TestNG dependencies to the Gradle project
dependencies {
// Use TestNG framework, also requires calling test.useTestNG() below
testImplementation 'org.testng:testng:7.10.0'
// This dependency is used by the application.
implementation libs.guava
implementation 'org.seleniumhq.selenium:selenium-java:4.24.0'
}
Step 6 – Add Gradle Test Task to build.gradle
tasks.named('test') {
// Use TestNG for unit tests.
useTestNG() {
useDefaultListeners = true
outputDirectory = file("$projectDir/TestNG_Reports")
}
reports.html.setDestination(file("$projectDir/GradleReports"))
}
The complete gradle.build looks like something shown below.
/*
* This file was generated by the Gradle 'init' task.
*
* This generated file contains a sample Java application project to get you started.
* For more details on building Java & JVM projects, please refer to https://docs.gradle.org/8.10/userguide/building_java_projects.html in the Gradle documentation.
*/
plugins {
// Apply the application plugin to add support for building a CLI application in Java.
id 'java'
}
repositories {
// Use Maven Central for resolving dependencies.
mavenCentral()
}
dependencies {
// Use TestNG framework, also requires calling test.useTestNG() below
testImplementation 'org.testng:testng:7.10.0'
// This dependency is used by the application.
implementation libs.guava
implementation 'org.seleniumhq.selenium:selenium-java:4.24.0'
}
// Apply a specific Java toolchain to ease working on different environments.
java {
toolchain {
languageVersion = JavaLanguageVersion.of(17)
}
}
tasks.named('test') {
// Use TestNG for unit tests.
useTestNG() {
useDefaultListeners = true
outputDirectory = file("$projectDir/TestNG_Reports")
}
reports.html.setDestination(file("$projectDir/GradleReports"))
}
Step 7 – Create Test Code under src/test/java
Let us write the code to test a web application. I have created 3 tests and out of 3, 1 test will fail intentionally.
package org.example;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.testng.Assert;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;
import java.time.Duration;
public class LoginTests {
WebDriver driver;
@BeforeMethod
public void setUp() {
ChromeOptions options = new ChromeOptions();
driver = new ChromeDriver(options);
driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(10));
driver.manage().window().maximize();
driver.get("https://opensource-demo.orangehrmlive.com/");
}
@Test(description = "This test validates error message when credentials are incorrect", priority = 0)
public void verifyIncorrectCredentials() {
driver.findElement(By.name("username")).sendKeys("Admin");
driver.findElement(By.name("password")).sendKeys("admin123$$");
driver.findElement(By.xpath("//*[@class='oxd-form']/div[3]/button")).submit();
String actualErrorMessage = driver.findElement(By.xpath("//*[@class='orangehrm-login-error']/div/div/p")).getText();
// Verify Error Message
Assert.assertEquals(actualErrorMessage,"Invalid credentials");
}
@Test(description = "This test will fail", priority = 1)
public void verifyBlankCredentials() {
driver.findElement(By.name("username")).sendKeys("");
driver.findElement(By.name("password")).sendKeys("admin123$$");
driver.findElement(By.xpath("//*[@class='oxd-form']/div[3]/button")).submit();
String actualErrorMessage = driver.findElement(By.xpath("//*[@class='oxd-form-row']/div/span")).getText();
// Verify Error Message
Assert.assertEquals(actualErrorMessage,"Invalid credentials");
}
@Test(description = "This test validates successful login to Home page", priority = 2)
public void verifyLoginPage() {
driver.findElement(By.name("username")).sendKeys("Admin");
driver.findElement(By.name("password")).sendKeys("admin123");
driver.findElement(By.xpath("//*[@class='oxd-form']/div[3]/button")).submit();
String homePageHeading = driver.findElement(By.xpath("//*[@class='oxd-topbar-header-breadcrumb']/h6")).getText();
//Verify new page - HomePage
Assert.assertEquals(homePageHeading,"Dashboard");
}
@AfterMethod
public void tearDown() {
driver.quit();
}
}
Step 8 – Create testng.xml
Right-click on the project and select TestNG and select Convert to TestNG.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
<test name="Selenium Tests with TestNG">
<classes>
<class name="org.example.LoginTests"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->
Step 9 – Run the tests from TestNG
Right-Click on the testng.xml and select Run As TestNG Suite.
The output of the above tests in Eclipse Console is as shown below.
This also generates a folder with the name test-output that contains the TestNG reports like index.html,emailable-report.html.
Step 10 – Run the tests from Command Line
To run the tests from the command line, use the below-mentioned command.
gradle clean test
The output of the above program is
Step 11 – TestNG and Gradle Report generation
Once the test execution is finished, refresh the project. We will see 2 folders – GradleReports and TestNG_ Reports.
Gradle Reports
This folder contains index.html.
Right-click on index.html and select open with Web Browser. This report shows the summary of all the tests executed. As you can see that Failed tests are selected (highlighted in blue), so the name of the test failed along with the class name is displayed here.
TestNG Reports
Go to TestNG_Reports folder and right-click and open emailable-report.html.
Index.html
Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!
JUnit is an open source Unit Testing Framework for JAVA. JUnit is a simple framework to write repeatable tests. It is an instance of the xUnit architecture for unit testing frameworks.
To generate Serenity Report in Jenkins, we need to download HTML Publisher. Please refer to this tutorial to install the plugin – How to install Plugins in Jenkins
Implementation Steps
Step 1: Create a new project using the Maven project plugin
Give the Name of the project.
Click on the Maven project.
Click on the OK button.
In the General section, enter the project description in the Description box.
Step 2: Build Management
Go to the Build section of the new job.
In the Root POM textbox, enter the full path to pom.xml
In the Goals and options section, enter “clean test“
Click on the Advanced button.
Step 3: Select custom workspace
Mention the full path of the project in the directory.
Step 4: Select “Publish HTML Reports” from “Post Build Actions”
Select “Publish HTML reports“.
HTML directory to archive – target/site/serenity
Index page[s] – Index.html
Report title – HTML Report
Click on the Apply and Save buttons.
We have created a new Maven project “SerenityDemo” with the configuration to run the Serenity Tests and also to generate a Serenity Report after execution using Jenkins.
Step 5: Execute the tests
Let’s execute it now by clicking on the “Build Now” button.
Right-click on Build Number (here in my case it is #5) and click on Console Output to see the result.
Below is the execution summary of the tests.
We can see below that the Serenity Reports are generated.
Once the execution is completed, click on “Back to Project“.
Step 6: Serenity Report
We could see a link to view ‘HTML Reports’. Click on the HTML Reports. It displays the Index.html Report of Serenity.
Serenity Report is in HTML format. So, we need to install HTML Publisher Plugin to view the Serenity Reports. Please refer to this tutorial to download and install the HTML Plugin in Jenkins – How to generate HTML Reports in Jenkins.
The Serenity Report looks like as shown below:
How to find the path of reports in Jenkins?
After running successfully, go to the job and click on the “Workspace” link. As soon as we click on the above-mentioned link, we will be inside this directory. Now click on the “target” link.
As soon as we click on the above-mentioned link, we will be inside this directory. Now click on the “site/serenity” link.
As soon as we click on the above-mentioned link, we will be inside this directory. Now we can see all the reports of Serenity.
This way, we could run Serenity tests and generate Serenity Reports using Jenkins.
Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!
Masterthought library provides pretty HTML reports for Cucumber. The cucumber JSON file is used to generate the HTML for the website. This Java report publisher was made particularly with publishing cucumber reports to the Jenkins build server in mind. It releases aesthetically pleasing HTML reports with charts displaying the outcomes of cucumber runs.
Prerequisite
Java 11 installed
Maven installed
Eclipse or IntelliJ installed
This framework consists of:
Selenium – 4.3.0
Java 11
Cucumber – 7.6.0
Maven – 3.8.1
JUnit– 4.13.2
Cucumber Reporting Plugin – 5.7.4
Project Structure
Implementation Steps
Download and Install Javaon the system
Download and setup Eclipse IDE on the system
Setup Maven
Install Cucumber Eclipse Plugin (For Eclipse IDE)
Create a new Maven Project
Add Selenium, JUnit4, Cucumber, and Masterthought dependencies to the project
Create a featurefile under src/test/resources
Create the test code locating the web elements in src/main/java
Create the Step Definition class or Glue Code in src/test/java
Create a JUnit4 Cucumber Runner class in src/test/java
Run the tests from Command Line
Cucumber Report Generation
Step 1- Download and Install Java
Cucumber and Selenium need Java to be installed on the system to run the tests. Click here to know How to install Java.
Step 2 – Download and setup Eclipse IDE on the system
The Eclipse IDE (integrated development environment) provides strong support for Java developers, which is needed to write Java code. Click here to know How to install Eclipse.
Step 3 – Setup Maven
To build a test framework, we need to add a number of dependencies to the project. It is a very tedious and cumbersome process to add each dependency manually. So, to overcome this problem, we use a build management tool. Maven is a build management tool that is used to define project structure, dependencies, build, and test management. Click here to know How to install Maven.
The Cucumber Eclipse plugin is a plugin that allows eclipse to understand the Gherkin syntax. The Cucumber Eclipse Plugin highlights the keywords present in Feature File. Click here to know more – Install Cucumber Eclipse Plugin.
Step 7 – Create a feature file (LoginPage.feature) containing all the test scenarios under src/test/resources/features
It is recommended to create a features folder in src/test/resources directory. Create all the feature files in this features folder. Featurefile should be saved as an extension of .feature.
Feature: Login to HRM Application
Background:
Given User is on HRMLogin page "https://opensource-demo.orangehrmlive.com/"
@ValidCredentials
Scenario: Login with valid credentials
When User enters username as "Admin" and password as "admin123"
Then User should be able to login successfully and new page open
@InvalidCredentials
Scenario Outline: Login with invalid credentials
When User enters username as "<username>" and password as "<password>"
Then User should be able to see error message "<errorMessage>"
Examples:
| username | password | errorMessage |
| $$$$$ | ££££££££ | Invalid credentials |
| admin | Admin123 | Invalid credentials |
| Admin123 | admin | Invalid credentials |
Step 8 – Create the test code locating the web elements in src/main/java
LoginPageLocators
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;
public class LoginPageLocators {
@FindBy(name = "username")
public WebElement userName;
@FindBy(name = "password")
public WebElement password;
@FindBy(id = "logInPanelHeading")
public WebElement titleText;
@FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/form/div[3]/button")
public WebElement login;
@FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/div/div[1]/div[1]/p")
public WebElement errorMessage;
}
HomePageLocators
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;
public class HomePageLocators {
@FindBy(xpath = "//*[@id='app']/div[1]/div[1]/header/div[1]/div[1]/span/h6")
public WebElement homePageUserName;
}
LoginPageActions
import org.openqa.selenium.support.PageFactory;
import com.example.locators.LoginPageLocators;
import com.example.utils.HelperClass;
public class LoginPageActions {
LoginPageLocators loginPageLocators = null;
public LoginPageActions() {
this.loginPageLocators = new LoginPageLocators();
PageFactory.initElements(HelperClass.getDriver(),loginPageLocators);
}
public void login(String strUserName, String strPassword) {
// Fill user name
loginPageLocators.userName.sendKeys(strUserName);
// Fill password
loginPageLocators.password.sendKeys(strPassword);
// Click Login button
loginPageLocators.login.click();
}
//Get the title of Login Page")
public String getLoginTitle() {
return loginPageLocators.titleText.getText();
}
// Get the error message of Login Page
public String getErrorMessage() {
return loginPageLocators.errorMessage.getText();
}
}
HomePageActions
import org.openqa.selenium.support.PageFactory;
import com.example.locators.HomePageLocators;
import com.example.utils.HelperClass;
public class HomePageActions {
HomePageLocators homePageLocators = null;
public HomePageActions() {
this.homePageLocators = new HomePageLocators();
PageFactory.initElements(HelperClass.getDriver(),homePageLocators);
}
// Get the User name from Home Page
public String getHomePageText() {
return homePageLocators.homePageUserName.getText();
}
}
HelperClass
import java.time.Duration;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import io.github.bonigarcia.wdm.WebDriverManager;
public class HelperClass {
private static HelperClass helperClass;
private static WebDriver driver;
public final static int TIMEOUT = 10;
private HelperClass() {
WebDriverManager.chromedriver().setup();
driver = new ChromeDriver();
driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(TIMEOUT));
driver.manage().window().maximize();
}
public static void openPage(String url) {
driver.get(url);
}
public static WebDriver getDriver() {
return driver;
}
public static void setUpDriver() {
if (helperClass==null) {
helperClass = new HelperClass();
}
}
public static void tearDown() {
if(driver!=null) {
driver.close();
driver.quit();
}
helperClass = null;
}
}
Step 9 – Create the Step Definition class or Glue Code in src/test/java
It is recommended to create a definitions folder in src/test/java directory. The StepDefinition files should be created in this definitionsdirectory. within the folder called definitions.
LoginPageDefinitions
import org.junit.Assert;
import com.example.actions.HomePageActions;
import com.example.actions.LoginPageActions;
import com.example.utils.HelperClass;
import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;
public class LoginPageDefinitions{
LoginPageActions objLogin = new LoginPageActions();
HomePageActions objHomePage = new HomePageActions();
@Given("User is on HRMLogin page {string}")
public void loginTest(String url) {
HelperClass.openPage(url);
}
@When("User enters username as {string} and password as {string}")
public void goToHomePage(String userName, String passWord) {
// login to application
objLogin.login(userName, passWord);
// go the next page
}
@Then("User should be able to login successfully and new page open")
public void verifyLogin() {
// Verify home page
Assert.assertTrue(objHomePage.getHomePageText().contains("Dashboard"));
}
@Then("User should be able to see error message {string}")
public void verifyErrorMessage(String expectedErrorMessage) {
// Verify error message
Assert.assertEquals(expectedErrorMessage,objLogin.getErrorMessage());
}
}
Hooks
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import com.example.utils.HelperClass;
import io.cucumber.java.After;
import io.cucumber.java.Before;
import io.cucumber.java.Scenario;
public class Hooks {
@Before
public static void setUp() {
HelperClass.setUpDriver();
}
@After
public static void tearDown(Scenario scenario) {
//validate if scenario has failed
if(scenario.isFailed()) {
final byte[] screenshot = ((TakesScreenshot) HelperClass.getDriver()).getScreenshotAs(OutputType.BYTES);
scenario.attach(screenshot, "image/png", scenario.getName());
}
HelperClass.tearDown();
}
}
Step 10 – Create a JUnit 4 Cucumber Runner class in src/test/java
import org.junit.runner.RunWith;
import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
@RunWith(Cucumber.class)
@CucumberOptions(tags = "", features = "src/test/resources/features/LoginPage.feature", glue = "com.example.definitions",
plugin= {"pretty", "json:target/cucumber-reports/reports.json",
"json:target/cucumber-reports/cucumber.runtime.formatter.JSONFormatter"})
public class CucumberRunnerTests {
}
Step 11 – Run the tests from Command Line
Use the below command to execute the tests
mvn clean test
The output of the above program is
Step 12 – Cucumber Report Generation
Refresh your project and check inside \target\cucumber-html-reports that the report generated with name feature-overview.
There are different types of HTML reports gets generated as a part of the test execution cycle.
1. feature-overview – This HTML report gives an overall overview of test execution. Main HTML report which covers all different sections like Features, Tags, Steps, and Failures.
2. failures-overview – This HTML report gives an overview of all failed tests.
3. step-overview – This HTML report shows step statistics for the current cycle.
4. tag-overview – This HTML report shows passing and failing statistics for different tags used in test execution.
Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!
QA Automation Expert 