The flexible aspects of Selenium WebDrivers and GitHub Actions enable users to create powerful, fast, and efficient automated testing workflows in CI/CD environments.
CI/CD pipelines have contributed to the success of the DevOps cycle in all software development projects. This is a holistic process that bridges development and operations. Continuous integration helps development teams deploy code efficiently, and continuous delivery automates code deployment.
Important points
1. The Serenity Web tests need to run in the headless mode. As we are using Chrome browser, use the below code in the serenity.config:
headless.mode = true
2. Install Chrome browser in ubuntu. Use the below code:
I have a repository available in GitHub – Serenity_Cucumber-JUnit5 as shown in the below image. Go to the “Actions” tab. Click on the “Actions” tab.
Step 2 – Select the type of Actions
You will see that GitHub recommends Actions depending on the project. In our case, it is recommending actions suitable for a Java project. I have selected the “Java with Maven” option as my project is built in Maven.
Step 3 – Generation of Sample pipeline
If you choose an existing option, it will automatically generate a .yaml for the project as shown below.
We will replace the current workflow with the following yml file as shown below:
name: Serenity Tests in GitHub
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up JDK 17
uses: actions/setup-java@v4
with:
java-version: '17'
distribution: 'temurin'
cache: maven
- uses: browser-actions/setup-chrome@latest
- run: chrome --version
- name: Build with Maven
run: mvn clean verify
- name: Test Report Generation
uses: actions/upload-artifact@v4
if: success() || failure()
with:
name: Serenity Report # Name of the folder
path: target/site/serenity/ # Path to test results
Step 4 – Commit the changes
After the changes, hit the “Start Commit” button.
This will give the option to add a description for the commit. It will also enable the user to commit either to the main branch or commit to any other branch that exists in the project. Click on the “Commit new file” button to set up the workflow file.
It will create a maven.yml file as shown below in the project.
Step 5 – Verify that the workflow is running
Next, head over to the “Actions” tab, and you will see your YAML workflow file present under the tab. The yellow sign represents that the job is in the queue.
In Progress – When the job starts building and running, you will see the status change from “Queued” to “in progress”.
Passed – If the build is successful, you will see a green tick mark.
Click on the workflow and the below screen is displayed. It shows the status of the run of the workflow, the total time taken to run the workflow, and the name of the .yml file.
Below shows all the steps of the workflow.
Step 6 – Published Serenity Report on GitHub
From the logs of the Workflow, you can see that the Test Report step was executed successfully.
Once the pipeline run, a Serenity Report folder will be generated as shown in the below image:
When we click on the folder Serenity Report, a zipped file will be downloaded, and we can extract it to see all the files contained within it.
Open the folder and we will see all the files including index.html and others.
In REST API development, blacklisting headers refers to explicitly disallowing or filtering out certain headers from being used or processed.
Starting from REST Assured version 4.2.0, a new feature has been introduced that allows you to blacklist specific headers in order to prevent them from appearing in request or response logs. Instead of displaying the actual header value, it will be replaced with “[ BLACKLISTED ]”. This feature can be enabled on a per-header basis using LogConfig.
Here is an example of how you can enable this feature for a specific header using LogConfig:
In the below code snippet, the “Accept” header will be blacklisted. Once this configuration is set, any requests or responses containing this header will have its value replaced with “[ BLACKLISTED ]” in the logs.
Blacklist single header
blacklistHeader – It blacklists one or more headers. If these headers show up during logging, they will be replaced with ‘BLACKLISTED ‘. The purpose of a blacklist is to prevent sensitive information to be included in the log.
import io.restassured.RestAssured;
import io.restassured.config.LogConfig;
import org.junit.Test;
import java.util.ArrayList;
import java.util.List;
import static io.restassured.RestAssured.given;
import static org.hamcrest.CoreMatchers.equalTo;
public class BlackListDemo {
@Test
public void verifyUser() {
// Given
given()
.config(RestAssured.config().logConfig(LogConfig.logConfig().blacklistHeader("Accept")))
.log().headers()
// When
.when()
.get("https://reqres.in/api/users/2")
// Then
.then()
.statusCode(200).statusLine("HTTP/1.1 200 OK")
.body("data.email", equalTo("janet.weaver@reqres.in"))
.body("data.first_name", equalTo("Janet"))
.body("data.last_name", equalTo("Weaver")).log().all();
}
The output of the above program is
Blacklist multiple headers
In the below example, we want to blacklist “Accept” as well as “Content-Type”. This can be achieved by the below-mentioned example:
import io.restassured.RestAssured;
import io.restassured.config.LogConfig;
import org.junit.Test;
import java.util.ArrayList;
import java.util.List;
import static io.restassured.RestAssured.given;
import static org.hamcrest.CoreMatchers.equalTo;
public class BlackListDemo {
@Test
public void verifyUser() {
// Given
given()
.config(RestAssured.config().logConfig(LogConfig.logConfig().blacklistHeader("Accept","Content-Type")))
.log().headers()
// When
.when()
.get("https://reqres.in/api/users/2")
// Then
.then()
.statusCode(200).statusLine("HTTP/1.1 200 OK")
.body("data.email", equalTo("janet.weaver@reqres.in"))
.body("data.first_name", equalTo("Janet"))
.body("data.last_name", equalTo("Weaver")).log().all();
}
The output of the above program is
BlackList Request and Response using collection
import io.restassured.RestAssured;
import io.restassured.config.LogConfig;
import org.junit.Test;
import java.util.ArrayList;
import java.util.List;
import static io.restassured.RestAssured.given;
import static org.hamcrest.CoreMatchers.equalTo;
public class BlackListDemo {
@Test
public void verifyUser1() {
List headers = new ArrayList<String>();
headers.add("Accept");
headers.add("Content-Type");
// Given
given()
.config(RestAssured.config().logConfig(LogConfig.logConfig().blacklistHeader(headers.toArray(new String[0]))))
.log().headers()
// When
.when()
.get("https://reqres.in/api/users/2")
// Then
.then()
.statusCode(200).statusLine("HTTP/1.1 200 OK")
// To verify booking id at index 3
.body("data.email", equalTo("janet.weaver@reqres.in"))
.body("data.first_name", equalTo("Janet"))
.body("data.last_name", equalTo("Weaver")).log().all();
}
}
The output of the above program is
Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
There are two modes, strict and non-strict in JSONAssert. In most cases, you will probably want to set strict to false, since that will make the tests less brittle. Strict tests require all the elements requested to be returned, and only those elements (ie, the tests are non-extensible). Arrays of elements must be returned to the same order as expected.
In Lenient mode, extensibility will be allowed, and no strict ordering will be checked. Let’s see example programs for comparing JSON Arrays.
Comparing two Exact same JSON Arrays – LENIENT Mode
import org.json.JSONArray;
import org.junit.Test;
import org.skyscreamer.jsonassert.JSONAssert;
import org.skyscreamer.jsonassert.JSONCompareMode;
public class JsonArrayAssertDemo {
@Test
public void sameArray() {
// same no of elements, values and in same order
String jsonArray1 = "[\"Vibha\",\"Abha\",\"Nysha\"]";
String jsonArray2 = "[\"Vibha\",\"Abha\",\"Nysha\"]";
JSONAssert.assertEquals(jsonArray1, jsonArray2, JSONCompareMode.LENIENT);
}
}
The output of the above program is
Comparing two JSON Arrays with the same fields but different in order – Lenient
@Test
public void sameArrayDifferentOrder() {
// Same no of elements but different order
String jsonArray1 = "[\"Vibha\",\"Abha\",\"Nysha\"]";
String jsonArray2 = "[\"Nysha\",\"Vibha\",\"Abha\"]";
JSONAssert.assertEquals(jsonArray1, jsonArray2, JSONCompareMode.LENIENT);
}
The output of the above program is
LENIENT mode does not care about the order of the fields in the JSON.
Comparing two JSON Arrays with the same fields but different in order – Strict
@Test
public void sameArrayDifferentOrder_Strict() {
// same no of elements, values and in same order
String jsonArray1 = "[\"Vibha\",\"Abha\",\"Nysha\"]";
String jsonArray2 = "[\"Nysha\",\"Vibha\",\"Abha\"]";
JSONAssert.assertEquals(jsonArray1, jsonArray2, JSONCompareMode.STRICT);
}
The output of the above program is
STRICT mode cares about the order of the fields in the JSON Array.
Comparing two JSON Arrays with the same fields but different values
@Test
public void sameArrayDifferentValue() {
// Same no of elements but different values
String jsonArray1 = "[\"Vibha Singh\",\"Abha\",\"Nysha\"]";
String jsonArray2 = "[\"Vibha\",\"Abha\",\"Nysha\"]";
JSONAssert.assertEquals(jsonArray1, jsonArray2, JSONCompareMode.LENIENT);
}
The output of the above program is
In this scenario assertEquals() will fail as a value is not matching.
Comparing two JSON Arrays with the case sensitivity
@Test
public void sameArrayCaseSensitive() {
// case sensitive
String jsonArray1 = "[\"VIbha\",\"Abha\",\"Nysha\"]";
String jsonArray2 = "[\"Vibha\",\"Abha\",\"Nysha\"]";
JSONAssert.assertEquals(jsonArray1, jsonArray2, JSONCompareMode.LENIENT);
}
The output of the above program is
Comparing two JSON Arrays with the same fields but different datatype of values
Here, both the JSON Arrays have the same structure, but we have added one more attribute to the second one. In this case, the assertion fails as it counts the number of items in the arrays.
Comparing two JSON Arrays with the addition – STRICT mode
The compare mode can also be defined by using an overloaded method that takes boolean instead of JSONCompareMode where LENIENT = false and STRICT = true.
There are two modes, strict and non-strict in JSONAssert. In most cases, you will probably want to set strict to false, since that will make the tests less brittle. Strict tests require all the elements requested to be returned, and only those elements (ie, the tests are non-extensible). Arrays of elements must be returned to the same order as expected.
In Lenient mode extensibility will be allowed and no strict ordering will be checked. Let’s see example programs for comparing JSONs.
Comparing two Exact same JSON Objects – LENIENT Mode
Here, both the JSONs have the same structure, but we have added one more attribute to the second one. In this case, the assertion matches the first string with the second one and it looks the same. So, the test passes.
Let us reverse the assertion now. We are comparing the jsonObject2 which has 3 attributes with jsonObject1 which has 2 attributes. So, salary is not present in jsonObject1. In this case, the test fails.
The compare mode can also be defined by using an overloaded method that takes boolean instead of JSONCompareMode where LENIENT = false and STRICT = true.
Maven is a powerful and widely used build automation and project management tool primarily used for Java projects. It simplifies the process of managing a software project by providing a standard way to build, manage dependencies, and generate documentation.
In the previous tutorial, we have discussed about How to install Maven on Windows. In this tutorial, we will see How To Create Selenium Maven Project in Eclipse IDE.
Prerequisite:
Maven Installed on machine
Java installed on machine
Integrated Development Environment (IDE) like IntelliJ IDEA or Eclipse is installed
Steps to follow:
Step 1 – To create a new project – Click on the New and then select – Project option.
Step 2 – Select the Maven Projectand click on the “Next” button.
Step 3 – You can select the default workplace or mention the location where you want to save the project.
Click the “Next“ button.
Step 4 – Select maven-archetype-quickstart option as shown below and click on the “Next” button.
Step 5 – Provide the Group Id, Artifact Id details and click the “Finish” button.
Group Id : com.example
Artifact Id : SeleniumMaven_Demo
Step 6 – The structure of the project looks as shown in the below image.
Step 7 – POM file will look like below image
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>SeleniumMaven_Demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>SeleniumMaven_Demo</name>
<!-- FIXME change it to the project's website -->
<url>http://www.example.com</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.7</maven.compiler.source>
<maven.compiler.target>1.7</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<pluginManagement><!-- lock down plugins versions to avoid using Maven defaults (may be moved to parent pom) -->
<plugins>
<!-- clean lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#clean_Lifecycle -->
<plugin>
<artifactId>maven-clean-plugin</artifactId>
<version>3.1.0</version>
</plugin>
<!-- default lifecycle, jar packaging: see https://maven.apache.org/ref/current/maven-core/default-bindings.html#Plugin_bindings_for_jar_packaging -->
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>3.0.2</version>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.1</version>
</plugin>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
<version>3.0.2</version>
</plugin>
<plugin>
<artifactId>maven-install-plugin</artifactId>
<version>2.5.2</version>
</plugin>
<plugin>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.2</version>
</plugin>
<!-- site lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#site_Lifecycle -->
<plugin>
<artifactId>maven-site-plugin</artifactId>
<version>3.7.1</version>
</plugin>
<plugin>
<artifactId>maven-project-info-reports-plugin</artifactId>
<version>3.0.0</version>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
To validate a JSON schema in Java, you can use libraries that provide support for JSON schema validation. JsonSchemaValidator is a Hamcrest matcher that can be used to validate that a JSON document matches a given JSON schema.
Add the following dependency in pom.xml. You can check the latest Maven dependency from here.
Query parameters are a way to pass information to an API flexibly and simply. They are added to the end of the API endpoint URL as a series of key-value pairs. To append query params to the end of a URL, a ‘?’ Is added followed immediately by a query parameter.
Query Param Example
https://reqres.in/api/users?page=2
Below is an example of Query Param.
import org.junit.Test;
import static io.restassured.RestAssured.given;
import static org.hamcrest.CoreMatchers.equalTo;
public class ParamDemo {
@Test
public void verifyQueryParam() {
String endpoint = "https://reqres.in/api/";
// Given
given()
.queryParam("page", "2")
// When
.when()
.get(endpoint + "users/")
// Then
.then()
// To verify the response body
.body("page", equalTo(2))
.body("per_page", equalTo(6))
.body("total_pages", equalTo(2));
}
}
The output of the above program is
.param() or queryParam(): If you don’t specify which, REST Assured will assume GET params are query params and POST params are form params.
Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
Assertions are used to perform various kinds of validations in the tests and help us to decide whether the test has passed or failed.
In the below example, we have 3 assertions, but if the first assertion fails, then the test fails without checking for the rest of the two assertions.
@Test
public void verifyHardAssertion() {
// Given
given()
// When
.when()
.get("https://reqres.in/api/users/2")
// Then
.then()
// To verify the response body
.body("data.email", equalTo("janet.weaver@reqres12.in"))
.body("data.first_name", equalTo("Janet1"))
.body("data.last_name", equalTo("Weaver"));
}
The output of the above program is
In the above example, email as well as first_name has incorrect data. But the assertion has checked email and as it has the incorrect data, it has failed the test without even checking for the rest two assertions.
In the below example, the assertion will check all 3 and will print all the error messages. In .body() you need to specify all assertions separated by comma. The below code will validate all 3 assertions and then fail the test.
@Test
public void verifySoftAssertion() {
// Given
given()
// When
.when()
.get("https://reqres.in/api/users/2")
// Then
.then()
// To verify the response body
.body("data.email", equalTo("janet.weaver@reqres12.in"),
"data.first_name", equalTo("Janet1"),
"data.last_name", equalTo("Weaver"));
}
}
The output of the above program is
Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
In the previous tutorial, I explained theSerenity BDD with Cucumber for Web Application using Junit4. In this tutorial, I will explain the same Test Framework using Serenity, Cucumber, and JUnit5. This tutorial gives a clear picture of the initial setup of a BDD Framework.
The JUnit Platform serves as a foundation for launching testing frameworks on the JVM. It also defines the TestEngine API for developing a testing framework that runs on the platform.
JUnit Jupiter is the combination of the new programming model and extension model for writing tests and extensions in JUnit 5. The Jupiter sub-project provides a TestEngine for running Jupiter based tests on the platform.
JUnit Vintage provides a TestEngine for running JUnit 3 and JUnit 4 based tests on the platform. It requires JUnit 4.12 or later to be present on the class/module path.
JUnit5 is not completely integrated with Serenity with Cucumber. So, it is advisable to usejupiter-vintage-engine for the Cucumber TestRunner classes.
Step 2 – Download and setup Eclipse IDE on the system
The Eclipse IDE (integrated development environment) provides strong support for Java developers which is needed to write Java code. Click here to know How to install Eclipse.
Step 3 – Setup Maven and create a new Maven Project
Step 8 – Create a feature file under src/test/resources
The purpose of the Feature keyword is to provide a high-level description of a software feature, and to group related scenarios. To know more about the Feature file, please refer this tutorial.
Feature: Login to HRM
@ValidCredentials
Scenario: Login with valid credentials
Given User is on Home page
When User enters username as "Admin"
And User enters password as "admin123"
Then User should be able to login successfully
@InValidCredentials
Scenario Outline: Login with invalid credentials
Given User is on Home page
When User enters username as '<username>'
And User enters password as '<password>'
Then User should be able to see error message '<errorMessage>'
Examples:
|username |password |errorMessage |
|admin |admin |Invalid credentials |
|abc |admin123 |Invalid credentials |
|abc |abc123 |Invalid credentials |
|1$£" | 45£"% |Invalid credentials |
@ForgetPassword
Scenario: Verify Forget Password Functionality
Given User is on Home page
When User clicks on Forgot your password link
Then User should be able to see new page which contains Reset Password button
Step 9 – Create junit-platform.properties file under src/test/resources (optional)
This is an optional step. Cucumber of version 6.7 and above provides the functionality to generate a beautiful cucumber report. For this, it is needed to add a file junit-platform.properties under src/test/resources.
cucumber.publish.enabled = true
Step 10 – Create the Step Definition class or Glue Code
A Step Definition is a Java method with an expression that links it to one or more Gherkin steps. When Cucumber executes a Gherkin step in a scenario, it will look for a matching step definition to execute. You can have all of your step definitions in one file, or in multiple files.
LoginPageDefinitions
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
import com.example.SerenityCucumberJunit5Demo.steps.StepDashboardPage;
import com.example.SerenityCucumberJunit5Demo.steps.StepForgetPasswordPage;
import com.example.SerenityCucumberJunit5Demo.steps.StepLoginPage;
import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;
import net.serenitybdd.annotations.Steps;
public class LoginPageDefinitions {
@Steps
StepLoginPage loginPage;
@Steps
StepDashboardPage dashPage;
@Steps
StepForgetPasswordPage forgetpasswordPage;
@Given("User is on Home page")
public void openApplication() {
loginPage.open();
}
@When("User enters username as {string}")
public void enterUsername(String userName) {
loginPage.inputUserName(userName);
}
@When("User enters password as {string}")
public void enterPassword(String passWord) {
loginPage.inputPassword(passWord);
loginPage.clickLogin();
}
@Then("User should be able to login successfully")
public void clickOnLoginButton() {
dashPage.loginVerify();
}
@Then("User should be able to see error message {string}")
public void unsucessfulLogin(String expectedErrorMessage) {
String actualErrorMessage = loginPage.errorMessage();
System.out.println("Actual Error Message :" + actualErrorMessage);
assertEquals(expectedErrorMessage, actualErrorMessage);
}
@When("User clicks on Forgot your password link")
public void clickForgetPasswordLink() {
loginPage.clickForgetPasswordLink();
}
@Then("User should be able to see new page which contains Reset Password button")
public void verifyForgetPasswordPage() {
assertTrue(forgetpasswordPage.ForgetPasswordPage());
}
}
Assertions in JUnit-Vintage Engine are imported from the below package:-
import static org.junit.jupiter.api.Assertions.*;
DashboardPageDefinitions
import com.example.SerenityCucumberJunit5Demo.steps.StepDashboardPage;
import net.serenitybdd.annotations.Step;
import net.serenitybdd.annotations.Steps;
public class DashboardPageDefinitions {
@Steps
StepDashboardPage dashPage;
@Step
public void verifyAdminLogin() {
dashPage.loginVerify();
}
}
The corresponding Test Step classes are – StepLoginPage and StepDashboardPage.
There are multiple ways to identify a web element on the web page – one of the ways is to use @FindBy or $(By.).
I prefer to use @FindBy as I need not find the same element multiple times. Using @FindBy, I have identified a web element and defined a WebElementFacacde for the same which is reusable.
Cucumber runs the feature files via JUnit and needs a dedicated test runner class to actually run the feature files. When you run the tests with Serenity, you use theCucumberWithSerenity test runner. You also need to use the @CucumberOptionsclass to provide the root directory where the feature files can be found.
import static io.cucumber.junit.platform.engine.Constants.GLUE_PROPERTY_NAME;
import static io.cucumber.junit.platform.engine.Constants.PLUGIN_PROPERTY_NAME;
import org.junit.platform.suite.api.ConfigurationParameter;
import org.junit.platform.suite.api.IncludeEngines;
import org.junit.platform.suite.api.SelectClasspathResource;
import org.junit.platform.suite.api.Suite;
@Suite
@IncludeEngines("cucumber")
@SelectClasspathResource("/features")
@ConfigurationParameter(key = GLUE_PROPERTY_NAME, value = "com.example.SerenityCucumberJunit5Demo.definitions")
@ConfigurationParameter(key = PLUGIN_PROPERTY_NAME, value = "io.cucumber.core.plugin.SerenityReporterParallel,pretty,timeline:build/test-results/timeline")
public class SerenityRunnerTest {
}
Step 12 – Create serenity.conf file under src/test/resources
The serenity configuration file is used to configure the drivers so the test cases can run successfully. This file contains an operating system-specific binary. The binary file sits between your test and the browser. It acts as an intermediary, an interface between your tests and the browser you are using.
You can also configure the webdriver.base.url property for different environments in the serenity.conf configuration file.
Step 13 – Create serenity.properties file at the root of the project
serenity.project.name = Serenity and Cucumber and JUnit5 Demo
Step 14 – Run the tests from Command Line
Open the command line and go to the location where pom.xml of the project is present and type the below command.
mvn clean verify
Step 15 – Test Execution Status
The image displayed above shows the execution status.
The feature file contains 3 test cases. Test Case 2 is a Test Scenario that has 4 examples. So, in total we have 6 tests. This information is clearly mentioned in the new version of Serenity.
Step 16 – Serenity Report Generation
The best part about Serenity is the report generation by it. The Reports contain all possible types of information, you can think of with minimal extra effort. There are multiple types of reports are generated. We are interested in index.html and serenity-summary.html. To know more about Serenity Reports, please refer to tutorials for Index.html and Serenity-Summary.html. Below is the new Serenity Report.
index.html
2. serenity-summary.html
Step 17 – Cucumber Report Generation (Optional)
Every Test Execution generates a Cucumber Report (Version 6.7.0) and above as shown in the image.
Copy the URL and paste it to a browser and it shows the report as shown below:
To know more about Cucumber Reports, refertothis tutorial.
We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
QA Automation Expert 