Query parameters are a way to pass information to an API flexibly and simply. They are added to the end of the API endpoint URL as a series of key-value pairs. To append query params to the end of a URL, a ‘?’ Is added followed immediately by a query parameter.
Handling HTTP query parameters in Playwright typically involves setting up your request with the desired parameters before navigating to a URL. Playwright provides methods to easily construct and use URLs with query parameters.
The following prerequisites are required to be installed on the machine to begin with a smooth setup and installation.
Java 11 or higher
IntelliJ IDE or any other IDE to create a project
Maven
Dependency List
Playwright – 1.57.0
Java 17
Maven – 3.9.6
TestNG – 7.11.0
Maven Compiler Plugin – 3.15.0
Maven Surefire Plugin – 3.5.4
Implementation Steps
1. Create a new Maven Project
The first step in setup is to create a new Maven project. I will be using IntelliJ in this tutorial. The following steps need to be followed to create a new Maven project :
Open IntelliJ, Navigate to File >> New >> Project
2. In the New Project window, enter the following details:
Name of the Project – Playwright_API_Demo
Location/path where the project needs to be saved – Documents/Playwright (my location)
Select JDK version — I am using JDK 17
Archetype — Search for quickstart and select maven-archetype-quickstartfrom the results
Click on the Create button to create the project.
This will create a project as shown below in the IntelliJ.
2. Setup Playwright with Java and TestNG
Add the Playwright and TestNG dependencies to the pom.xml. The latest dependency can be found from here.
After adding the dependency, refresh the project. We can see that the Playwright jar files are downloaded in External Libraries.
3. Create API tests
Create Setup Methods
@BeforeMethod: This annotation indicates that `setUp()` will run before each test method, ensuring that the Playwright environment and API request context are properly initialized.
createAPIRequestContext(): Sets up the API request context with a base URL (`https://jsonplaceholder.typicode.com`) and common headers (like “Accept: application/json”) that will be used for all requests.
`@AfterMethod`: Ensures that resources are properly disposed of after every test method execution by calling `request.dispose()` and `playwright.close()`. This avoids resource leaks and ensures clean execution for subsequent tests.
@AfterMethod
void tearDown() {
if (request != null) {
request.dispose();
}
if (playwright != null) {
playwright.close();
}
}
Query Parameter
The corresponding API Test in Playwright Framework.
@Test
public void getCommentsByPostId() {
// GET /comments?postId=1
APIResponse response = request.get("/comments",
RequestOptions.create().setQueryParam("postId", "1"));
Assert.assertEquals(response.status(), 200, "Expected 200 for GET /comments?postId=1");
String body = response.text();
System.out.println("GET Response: " + body);
Gson gson = new Gson();
JsonObject[] posts = gson.fromJson(body, JsonObject[].class);
Assert.assertTrue(posts.length > 0, "Expected at least one post for postId=1");
Assert.assertEquals(posts[0].get("postId").getAsInt(), 1, "Expected postId=1 in first result");
Assert.assertEquals(posts[0].get("id").getAsInt(), 1, "Expected id=1 in first result");
Assert.assertEquals(posts[0].get("email").getAsString(), "Eliseo@gardner.biz", "Expected email is incorrect for first result" + posts[0].get("email").getAsString());
}
Explanation
1.Sending an API GET Request
Sends an HTTP GET request to the `/comments` endpoint with a query parameter `postId` set to `“1”`.
Verifies that the response status code is 200 (OK).
Assert.assertEquals(response.status(), 200, "Expected 200 for GET /comments?postId=1");
3. Read the response body
Prints the entire response content to the console for debugging or logging.
String body = response.text();
System.out.println("GET Response: " + body);
4. Parsing the JSON Response
Parses the response body into an array of `JsonObject`. This assumes the API returns a JSON array of objects, each representing a comment.
Gson gson = new Gson();
JsonObject[] posts = gson.fromJson(body, JsonObject[].class);
5. Validate the Response data
Checks that there is at least one comment in the response, indicating the API return is not empty.
Assert.assertTrue(posts.length > 0, "Expected at least one post for postId=1");
Verifies that the `postId` in the first comment object is `1`.
Verifies that the `id` in the first comment object is `1`.
Verifies that the `email` in the first comment object is `Eliseo@gardner.biz`.
Assert.assertEquals(posts[0].get("postId").getAsInt(), 1, "Expected postId=1 in first result");
4. Test Execution through IntelliJ
Go to the Test class and right-click and select Run ‘API_Tests’. The tests will run as IntelliJ tests.
Below is the test execution result.
5. Run the tests using the command line
Use the below command to run the tests using the command line.
mvn clean test
The output of the above program is
6. TestNG Report Generation
TestNG creates a target directory that contains index.html report.
Right-click on the index.html and open it in the browser.
Summary
Playwright: Utilized for its ability to handle HTTP requests and execute browser interactions. This code leverages Playwright for making API calls rather than web UI interactions.
Gson: Simplifies the parsing of JSON responses into Java objects, allowing easy assertion of specific fields.
TestNG: Provides the testing framework features such as test annotations and assertions used to validate the API responses.
An HTTP request is a message sent by a client (usually a web browser or a tool like cURL or Postman) to a server, requesting a resource or action to be performed. This is a fundamental part of the Web’s client-server communication model. The HTTP protocol defines how requests and responses are formatted and transmitted over the Internet.
System requirements
The following prerequisites are required to be installed on the machine to begin with a smooth setup and installation.
Java 11 or higher
IntelliJ IDE or any other IDE to create a project
Maven
Dependency List
Playwright – 1.57.0
Java 17
Maven – 3.9.6
TestNG – 7.11.0
Maven Compiler Plugin – 3.15.0
Maven Surefire Plugin – 3.5.4
Implementation Steps
1. Create a new Maven Project
The first step in setup is to create a new Maven project. I will be using IntelliJ in this tutorial. The following steps need to be followed to create a new Maven project :
Open IntelliJ, Navigate to File >> New >> Project
2. In the New Project window, enter the following details:
Name of the Project – Playwright_API_Demo
Location/path where the project needs to be saved – Documents/Playwright (my location)
Select JDK version — I am using JDK 17
Archetype — Search for quickstart and select maven-archetype-quickstartfrom the results
Click on the Create button to create the project.
This will create a project as shown below in the IntelliJ.
2. Setup Playwright with Java and TestNG
Add the Playwright and TestNG dependencies to the pom.xml. The latest dependency can be found from here.
After adding the dependency, refresh the project. We can see that the Playwright jar files are downloaded in External Libraries.
3. Create API tests in
Create Setup Methods
@BeforeMethod: This annotation indicates that setUp() will run before each test method, ensuring that the Playwright environment and API request context are properly initialized.
createAPIRequestContext(): Sets up the API request context with a base URL (https://jsonplaceholder.typicode.com) and common headers (like “Accept: application/json”) that will be used for all requests.
@AfterMethod: Ensures that resources are properly disposed of after every test method execution by calling request.dispose()and playwright.close(). This avoids resource leaks and ensures clean execution for subsequent tests.
@AfterMethod
void tearDown() {
if (request != null) {
request.dispose();
}
if (playwright != null) {
playwright.close();
}
}
GET Request
What is GET Request?
The GET HTTP method requests a representation of the specified resource. Requests using GET should only be used to request data and shouldn’t contain a body.
The corresponding API Test in Playwright Framework.
@Test
public void getUserById() {
APIResponse response = request.get("/posts/1");
System.out.println("GET Response :" + response.text());
Assert.assertEquals(response.status(), 200, "Expected 200 for GET /posts/1");
Gson gson = new Gson();
JsonObject jsonResponse = gson.fromJson(response.text(), JsonObject.class);
Assert.assertEquals(jsonResponse.get("userId").getAsInt(), 1, "Expected userId=1 in response body");
Assert.assertEquals(jsonResponse.get("id").getAsInt(), 1, "Expected id=1 in response body");
}
Explanation
We sends a GET request to retrieve a specific post.
Verifies that the response status code is 200 (OK).
Parses the response JSON to check for expected values (`userId` and `id` both equal to 1).
POST Request
What is POST Request?
The POST HTTP method sends data to the server. The type of the body of the request is indicated by the Content-Type header
@Test
public void createUser() {
Map<String, Object> payload = new HashMap<>();
payload.put("title", "architect");
payload.put("body", "post test");
payload.put("userId", 100);
APIResponse response = request.post("/posts", RequestOptions.create().setData(payload));
System.out.println("POST Response :" + response.text());
Assert.assertEquals(response.status(), 201, "Expected 201 for POST /users");
Gson gson = new Gson();
JsonObject body = gson.fromJson(response.text(), JsonObject.class);
Assert.assertEquals(body.get("title").getAsString(), "architect");
Assert.assertEquals(body.get("body").getAsString(), "post test");
Assert.assertEquals(body.get("userId").getAsInt(), 100);
Assert.assertNotNull(body.get("id"), "Expected id in create response");
}
Explanation
Constructs a payload for creating a new post and sends a POST request.
Verifies that the response status code for successful creation (201).
Parses the response and asserts that the payload values match the expected results, ensuring the ID field is not null, indicating successful resource creation.
PUT Request
What is PUT Request?
The PUT HTTP method creates a new resource or replaces a representation of the target resource with the request content.
@Test
public void updateUser() {
APIResponse response = request.get("/posts/1");
System.out.println("GET Response before PUT Request:" + response.text());
Assert.assertEquals(response.status(), 200, "Expected 200 for GET /posts/1");
Map<String, Object> payload = new HashMap<>();
payload.put("id", 1);
payload.put("title", "business manager");
payload.put("body", "Implementing DataWareHouse Migration Project");
payload.put("userId", 50);
APIResponse response1 = request.put("/posts/1", RequestOptions.create().setData(payload));
System.out.println("PUT Response :" + response1.text());
Assert.assertEquals(response1.status(), 200, "Expected 200 for PUT /posts/1");
Gson gson = new Gson();
JsonObject body = gson.fromJson(response1.text(), JsonObject.class);
Assert.assertEquals(body.get("title").getAsString(), "business manager");
Assert.assertEquals(body.get("body").getAsString(), "Implementing DataWareHouse Migration Project");
Assert.assertEquals(body.get("userId").getAsInt(), 50);
Assert.assertNotNull(body.get("id"), "Expected id in create response");
}
Explanation
Retrieves a post using a GET request to ensure it exists.
Constructs a payload with updated information and sends a PUT request to modify the post.
Verifies that the response status code is 200 to verify successful update.
Verifies that the response JSON contains updated values as specified in the payload.
DELETE Request
What is Delete Request?
The DELETE HTTP method asks the server to delete a specified resource. Requests using DELETE should only be used to delete data and shouldn’t contain a body.
Confirms the successful deletion by checking for a 200 response status code.
7. Test Execution through IntelliJ
Go to the Test class and right-click and select Run ‘API_Tests’. The tests will run as IntelliJ tests.
Below is the test execution result.
8. Run the tests using the command line
Use the below command to run the tests using the command line.
mvn clean test
The output of the above program is
9. TestNG Report Generation
TestNG creates a target directory that contains index.html report.
Right-click on the index.html and open it in the browser.
Summary:
Playwright: Utilized for its ability to handle HTTP requests and execute browser interactions. This code leverages Playwright for making API calls rather than web UI interactions.
Gson: Simplifies the parsing of JSON responses into Java objects, allowing easy assertion of specific fields.
TestNG: Provides the testing framework features such as test annotations and assertions used to validate the API responses.
Create a folder – features within src/test/resources to create test scenarios in the Feature file.
Feature file should be saved as an extension of .feature. Add the test scenarios in this feature file. I have added sample test scenarios. In this feature file. The test scenarios are written in Gherkinslanguage.
@allure.label.parent_suite:WebInterface
@allure.label.sub_suite:Login Page
@allure.label.owner:Vibha
Feature: Login to HRM Application
Background:
Given User is on HRMLogin page "https://opensource-demo.orangehrmlive.com/"
@ValidCredentials @Critical
Scenario: Login with valid credentials
When User enters username as "Admin" and password as "admin123"
Then User should be able to login successfully and new page opens with heading "Dashboard"
@InvalidCredentials @High
Scenario Outline: Login with invalid credentials
When User enters username as "<username>" and password as "<password>"
Then User should be able to see error message "<errorMessage>"
Examples:
| username | password | errorMessage |
| Admin | admin12$$ | Invalid credentials |
| admin$$ | admin123 | Invalid credentials |
| abc123 | xyz$$ | Invalid credentials |
@MissingUsername @Medium
Scenario: Login with blank username
When User enters username as " " and password as "admin123"
Then User should be able to see a message "Required1" below Username
7. Create the step definition class in src/test/java
8. Create a TestNG Cucumber Runner class
We need to create a class called Runner class to run the tests. This class will use the TestNG annotation @Test, which tells TestNG what is the test runner class.
Add Allure Report plugin in the Test Runner to generate the Allure Report.
Note:- @Test annotation marks this class as part of the test. So, if we will remove this annotation, the Allure Report executesCucumberRunnerTests as a separate test suite, so there will be duplicate results.
9. Create testng.xml for the project
<?xml version = "1.0"encoding = "UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name = "Suite1">
<test name = "Test Demo">
<classes>
<class name = "com.example.runner.CucumberRunnerTests"/>
</classes>
</test>
</suite>
10. Specifying Allure Results location
Allure, by default, saves test results in the project’s root directory. Still, it is recommended to store your test results in the build output directory.
To configure this, create an allure.properties file and place it in the test resources directory of your project, which is typically located at src/test/resources:
In the image below, we can see that one test failed and four passed out of five tests.
This will create the allure-results folder with all the test reports within build folder. These files will be used to generate Allure Report.
12. How to Generate a Report
allure generate processes the test results and saves an HTML report into the allure-report directory. To view the report, use the allure open command.
allure serve creates the same report as allure generate, then automatically opens the main page of the report in a web browser.
Use the command below to generate the Allure Report
allure serve build/allure-results
This will generate the beautiful Allure Test Report as shown below.
13. How to View a Report
Test reports generated with Allure Report are basically small HTML websites intended to be viewed in a web browser.
Title
A human-readable title of the test. If not provided, the function name is used instead.
Tags
Any number of short terms the test is related to. Usually it’s a good idea to list relevant features that are being tested. Tags can then be used for filtering.
Owner
The team member who is responsible for the test’s stability. For example, this can be the test’s author, the leading developer of the feature being tested, etc.
Severity
A value indicating how important the test is. This give the future reader an idea of how to prioritize the investigations of different test failures.
Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”.
Support for Scenario Outlines
Allure Cucumber-JVM provides complete support for Scenario Outlines, a feature of Cucumber-JVM that allows for parametrized tests. No special setup is needed to take advantage of this capability within Allure.
Attachment
In Allure reports, you have the ability to attach various types of files, which can greatly enhance the comprehensibility of the report. A common practice is to attach screenshots that capture the state of the user interface at specific moments during test execution.
Allure Cucumber-JVM offers multiple methods for creating attachments, whether from pre-existing files or from content generated on-the-fly.
Graphs in Allure Report
Graphs allow you to see different statistics collected from the test data: status breakdown or severity and duration diagrams.
Timeline in Allure Report
The timeline tab visualizes retrospective test execution, allure adaptors collect precise timings of tests, and here on this tab, they are arranged accordingly to their sequential or parallel timing structure.
Generate Reports with the Allure Awesome Plugin
Allure 3 Report also features an advanced report generator plugin – Allure Awesome. It supports additional configuration options, such as generating the report as a single HTML file, setting the theme, custom branding and language of the generated report, and taking known issues into account.
To manually generate a customized Allure Awesome report, use the awesome command:
allure awesome build/allure-results
This will create the allure-report folder with all the test files and index.html report.
Open “index.html“, as this is an HTML report, and open it with the browser. The below image shows index.html.
Right click on index.html->Open In ->Browser ->Edge( any browser).
Summary:
1.Add required dependencies – Include Playwright, TestNG, and Allure TestNG in pom.xml, and configure the Allure Maven plugin.
2. Write Cucumber Scenarios in Feature File – Define the feature files using Gherkin syntax (.feature) and place them in the appropriate directory (src/test/resources/features).
3. Implement Step Definitions – Create step definitions in Java to connect the Gherkin scenarios to Playwright actions. Ensure these are correctly annotated with Cucumber’s annotations.
4. Create Test Runner – Setup a TestNG runner that combines Cucumber options and ensures Allure configurations are included.
5. Configure Allure Reporting – Utilize Allure annotations like @AllureFeature and @AllureStory within your tests to categorize and logically group test cases. Create an allure.properties file in src/test/resources with necessary configurations like resultsDirectory and linkPattern.
6. Execute Tests and Generate Reports – Execute mvn clean test, then run allure serve allure-results to build and open the HTML report.
Congratulations!! We have integrated an allure report with Playwright, Java Cucumber, and TestNG. I hope this tutorial is useful to you.
In this tutorial, I will explain how to IntegrateAllure Report 3 with Playwright, Javaand TestNG.
What is an Allure Report?
Allure Framework is a flexible, lightweight, multi-language test report tool. It shows a very concise representation of what has been tested in a neat web report form. It also allows everyone participating in the development process to extract maximum useful information from the everyday execution of tests.
AllureAttachment class is created to attach binary data (the screenshot) to the currently running test in the Allure lifecycle. This gives a central attachment library for the framework.
package com.example.utils;
import io.qameta.allure.Attachment;
public abstract class AllureAttachments {
@Attachment(value = "Failure Screenshot", type = "image/png")
public static byte[] attachScreenshot(byte[] screenshot) {
return screenshot;
}
}
BaseClass
This class contains the common variables and methods used throughout the project, like setup and teardown methods.
package com.example.utils;
import com.microsoft.playwright.*;
import org.testng.annotations.*;
public class BaseClass {
// Shared between all tests in this class.
Playwright playwright;
Browser browser = null;
// New instance for each test method.
BrowserContext context;
public Page page;
public Page getPage() {
return page;
}
@BeforeClass
public void launchBrowser() {
playwright = Playwright.create();
browser = playwright.chromium().launch(new BrowserType.LaunchOptions().setHeadless(false));
}
@BeforeMethod
public void createContextAndPage() {
context = browser.newContext();
page = context.newPage();
page.navigate("https://opensource-demo.orangehrmlive.com/web/index.php/auth/login");
}
@AfterMethod
public void closeContext() {
context.close();
}
@AfterClass
public void closeBrowser() {
playwright.close();
}
}
TestListener
This class is a TestNG failure hook. Its job is to automatically capture a Playwright screenshot when a test fails. The screenshot is then attached to the Allure report.
By implementing ITestListener, we are plugging into the TestNG test lifecycle. That means this class gets called automatically whenever a test fails.
package com.example.utils;
import com.microsoft.playwright.Page;
import org.testng.ITestListener;
import org.testng.ITestResult;
public class TestListener implements ITestListener {
@Override
public void onTestFailure(ITestResult result) {
System.out.println("❌ onTestFailure triggered");
Page page = BaseClass.getPage();
if (page != null) {
byte[] screenshot = page.screenshot();
AllureAttachments.attachScreenshot(screenshot);
}
}
}
5. Write the Test Scripts
Create a Test files under src/test/java. Use these page classes in your test scripts to perform end-to-end scenarios. This will keep your tests clean and focused on logic rather than details about the UI elements.
We will be automating the following test scenario using Playwright Java and run them in Chromium.
Verify the invalid username generates error message “Invalid credentials”
Verify that the user is able to login to the application successfully.
Verify the missing username generates error message “Required1”. This test will fail
Creating a Login Page Test Class:
package com.example.tests;
import com.example.pages.DashboardPage;
import com.example.pages.LoginPage;
import com.example.utils.BaseClass;
import com.example.utils.TestListener;
import io.qameta.allure.Description;
import io.qameta.allure.Owner;
import io.qameta.allure.Severity;
import org.testng.Assert;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;
import static io.qameta.allure.SeverityLevel.*;
public class LoginTests extends BaseClass {
@Test
@Description("This test attempts to log into the website using a invalid login and a password that result in error")
@Severity(NORMAL)
@Owner("Vibha Singh")
public void unsuccessfulLogin() {
LoginPage loginPage = new LoginPage(page);
loginPage.login("abc","abc");
String actualErrorMessage = loginPage.getErrorMessage();
Assert.assertEquals(actualErrorMessage, "Invalid credentials");
}
@Test
@Description("This test attempts to log into the website using a valid login and a password")
@Severity(CRITICAL)
@Owner("Vibha Singh")
public void successfulLogin() {
LoginPage loginPage = new LoginPage(page);
loginPage.login("Admin","admin123");
DashboardPage dashboardPage = new DashboardPage(page);
String actualHeading = dashboardPage.getDashboardPageHeading();
Assert.assertEquals(actualHeading, "Dashboard");
}
@Test
@Description("This test attempts to log into the website using a blank login and a password that result in error")
@Severity(MINOR)
@Owner("Vibha Singh")
public void missingUsername() {
LoginPage loginPage = new LoginPage(page);
loginPage.login("","admin123");
String actualErrorMessage = loginPage.getMissingUsernameErrorMessage();
Assert.assertEquals(actualErrorMessage, "Required1");
}
}
Use the @Description() annotation to set a description statically or use the description() method to set it dynamically in runtime.
@Description("This test attempts to log into the website using a invalid login and a password that result in error")
6. Create testng.xml for the project
Mention the listener in the testng.xml. This is used for all the test classes.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Playwright test suite">
<listeners>
<listener class-name="com.example.utils.TestListener"/>
</listeners>
<test name="Integration of Playwright Java with TestNG">
<classes>
<class name="com.example.tests.LoginTests">
</class>
</classes>
</test>
</suite>
7. Specifying Allure Results location
Allure, by default, saves test results in the project’s root directory. However, it is recommended to store your test results in the build output directory.
To configure this, create an allure.properties file and place it in the test resources directory of your project, which is typically located at src/test/resources:
In the image below, we can see that one test failed and four passed out of three tests.
This will create the allure-results folder with all the test reports within build folder. These files will be used to generate Allure Report.
9. How to Generate an Allure Report
allure generate processes the test results and saves an HTML report into the allure-report directory. To view the report, use the allure open command.
allure serve creates the same report as allure generate, then automatically opens the main page of the report in a web browser.
Use the command below to generate the Allure Report
allure serve build/allure-results
This will generate the beautiful Allure Test Report as shown below.
10. How to View a Report
Test reports generated with Allure Report are basically small HTML websites intended to be viewed in a web browser.
Title
A human-readable title of the test. If not provided, the function name is used instead.
Owner
The team member who is responsible for the test’s stability. For example, this can be the test’s author, the leading developer of the feature being tested, etc.
Severity
A value indicating how important the test is. This may give the future reader an idea of how to prioritize the investigations of different test failures.
Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”.
Steps
Allure TestNG provides three ways of creating steps and sub-steps: “annotated steps”, “lambda steps” and “no-op steps”. Define a test step with the given name.
Attachment
We can attach any sorts of files to your Allure report. Allure TestNG provides various ways to create an attachment, both from existing files or generated dynamically. To create an attachment using the Annotations API, define a method that returns some data and annotate it with @Attachment. Call the method at any point during your test.
Click on the highlighted portion and we can see the screenshot of the failed test.
Graphs in Allure Report
Graphs allow you to see different statistics collected from the test data: status breakdown or severity and duration diagrams.
Timeline in Allure Report
The timeline tab visualizes retrospective test execution. Allure adaptors collect precise timings of tests. Here on this tab, they are arranged according to their sequential or parallel timing structure.
Generate Reports with the Allure Awesome Plugin
Allure 3 Report also features an advanced report generator plugin – Allure Awesome. It supports additional configuration options, such as generating the report as a single HTML file, setting the theme, custom branding and language of the generated report, and taking known issues into account.
To manually generate a customized Allure Awesome report, use the awesome command:
allure awesome build/allure-results
Congratulations!! We have integrated an allure report with Cucumber, Selenium, and TestNG. I hope this tutorial is useful to you.
Playwright is integrated with Java, Cucumber, and TestNG. This integration creates a robust and scalable test automation framework for modern web applications. It is a BDD-driven framework.
Page Object Model (POM) is a design pattern used in test automation. It enhances test maintenance and reduces code duplication. In this pattern, web pages are represented as classes. The various elements on the page are defined as variables in the class. All possible user interactions can then be implemented as methods in the class.
What is Cucumber?
Cucumber is Behavior Driven Development (BDD). It is a methodology that integrates non-technical stakeholders, testers, and developers into the same conversation. This is achieved by using clearly defined specifications.
In behavior-driven development with Cucumber, test cases are written in Gherkin syntax. They use a Given-When-Then structure. This structure closely mirrors user stories and business rules.
Benefits of Using Cucumber with Playwright
1. Readable, business-friendly tests: Cucumber’s Gherkin syntax (Given–When–Then) makes test scenarios easy to understand for non-technical stakeholders, bridging the gap between QA, developers, and business teams.
2. Better Test Maintainability: With standardized step definitions and modularized test architecture, scaling up your test suite is effortless. Altering the suite won’t harm the entire framework.
3. Clear reporting and traceability: Cucumber’s reports map test results directly to feature files. They map directly to scenarios. This makes it easier to track coverage and failures against requirements.
4. Cross-browser and cross-platform coverage: When combined with Playwright, Cucumber scenarios can be executed across Chromium, Firefox, and WebKit without changing feature files
5. Scales well in CI/CD pipelines: The combination supports parallel execution. It offers tagging and selective test runs. This makes it suitable for continuous testing in CI environments.
What is TestNG?
TestNG is a testing framework widely used with Selenium for building structured, maintainable, and efficient automated tests. Combining TestNG listeners with Playwright gives testers greater control over test monitoring, logging, and custom actions based on test outcomes. In TestNG you can initialize Playwright and Browser in @BeforeClass method and destroy them in @AfterClass.
System requirements
The following prerequisites are required to be installed on the machine to begin with a smooth setup and installation.
Java 11 or higher
IntelliJ IDE or any other IDE to create a project
Maven
Browsers on which tests need to be run, like Chrome, Firefox, etc.
High-Level Architecture
Setting Up Cucumber BDD Framework with Playwright and TestNG
1. Create a new Maven Project
The first step in setup is to create a new Maven project. I will be using IntelliJ in this tutorial. The following steps need to be followed to create a new Maven project :
Open IntelliJ, Navigate to File >> New >> Project
In the New Project window, enter the following details:
Name of the Project – Cucumber_Playwright_TestNG
Location/path where the project needs to be saved – Documents/Playwright (my location)
Select JDK version — I am using JDK 17
Archetype — Search for “quickstart” and select maven-archetype-quickstartfrom the results
Click on the Create button to create the project.
This will create a project as shown in IntelliJ.
2. Set Up the Environment
2.1 Add the Cucumber, Playwright Java, and TestNG dependencies
Add the Cucumber, Playwright Java, and TestNG dependencies to the pom.xml. The latest dependency can be found here.
The compiler plugin is used to compile the source code of a Maven project. This plugin has two goals, which are already bound to specific phases of the default lifecycle:
Creating a well-organized project structure is crucial for maintaining a scalable and efficient automation framework.
1. Maven Configuration (pom.xml): This file manages project dependencies, plugins, and other configurations.
2. Features (src/test/resources/…/features): Contains Gherkin feature files written in plain English.
3. StepDefinition (src/test/java/.../tests/): Here, you write your actual test cases, utilizing the page classes. Cucumber stepsare implemented.
4. Runner (src/main/java/…runner/): It connects Cucumber feature files with TestNGand triggers execution.
5. Hooks (src/test/java/…/hooks): Hooks manage test lifecycle events like Launch browser before scenario, Close browser after scenario, Capture screenshots on failure
6. Page Classes (src/main/java/.../pages/): Each web page in your application should have a corresponding page class. This class encapsulates all the functionalities of the web page, following the Page Object Model.
7. Steps (src/test/java/…/steps): It is an extra abstraction layer between step definitions and pages. It contains code which is common across scenarios.
8. Utility Classes (src/main/java/.../utils/): These classes can include various utilities like configuration readers, browser factory, helper methods for common tasks, etc.
9. Test Data Files (src/test/resources/testdata/): Externalizes test data to avoid hardcoding. This includes items like usernames and passwords. These are stored in external files for easy management.
10. cucumber.properties: It contains global configuration for Cucumber.
11. testng.xml: This file is the central configuration file that defines how the test suite should be executed.
4. Create a feature file under src/test/resources
It is recommended to create a features folder in the src/test/resources directory. Create all the feature files in this features folder. Feature file should be saved as an extension of .feature. The test scenarios in the Feature file are written in Gherkins language. Add the test scenarios in this feature file. I have added sample test scenarios. Out of 5 tests, 4 will pass and 1 will fail.
Feature: Login to HRM Application
Background:
Given User is on HRMLogin page "https://opensource-demo.orangehrmlive.com/"
@ValidCredentials
Scenario: Login with valid credentials
When User enters username as "Admin" and password as "admin123"
Then User should be able to login successfully and new page opens with heading "Dashboard"
@InvalidCredentials
Scenario Outline: Login with invalid credentials
When User enters username as "<username>" and password as "<password>"
Then User should be able to see error message "<errorMessage>"
Examples:
| username | password | errorMessage |
| Admin | admin12$$ | Invalid credentials |
| admin$$ | admin123 | Invalid credentials |
| abc123 | xyz$$ | Invalid credentials |
@MissingUsername
Scenario: Login with blank username
When User enters username as " " and password as "admin123"
Then User should be able to see a message "Required1" below Username
5. Create the utility package in src/test/java
Create a Playwright Factory that contains the methods to start the browser and close the browser.
package com.example.utils;
import com.microsoft.playwright.*;
public class PlaywrightFactory {
private static Playwright playwright;
private static Browser browser;
private static BrowserContext context;
private static Page page;
public static void initBrowser() {
playwright = Playwright.create();
browser = playwright.chromium().launch(new BrowserType.LaunchOptions().setHeadless(false));
context = browser.newContext();
page = context.newPage();
}
public static Page getPage() {
return page;
}
public static void closeBrowser() {
if (page != null) page.close();
if (context != null) context.close();
if (browser != null) browser.close();
if (playwright != null) playwright.close();
}
}
6. Create the hooks package in src/test/java
Create a Hook class that contain the code that runs before or after each scenario. Here, it is responsible for
Each web page in your application should have a corresponding page class. This class encapsulates all the functionalities of the web page, following the Page Object Model.
LoginPage
package com.example.pages;
import com.microsoft.playwright.*;
public class LoginPage {
private Page page;
// Locators
private final Locator usernameLocator;
private final Locator passwordLocator;
private final Locator submitLocator;
private final Locator invalidCredentialsLocator;
private final Locator missingUsernameLocator;
public LoginPage(Page page) {
this.page = page;
this.usernameLocator = page.locator("input[name='username']");
this.passwordLocator = page.locator("input[name='password']");
this.submitLocator = page.locator("button[type='submit']");
this.invalidCredentialsLocator = page.locator("//p[contains(@class, \"oxd-text oxd-text--p oxd-alert-content-text\")]");
this.missingUsernameLocator = page.locator("//span[contains(@class, 'oxd-text oxd-text--span oxd-input-field-error-message oxd-input-group__message')]");
}
public void login(String user, String pass){
usernameLocator.fill(user);
passwordLocator.fill(pass);
submitLocator.click();
}
public void getUrl(String url){
page.navigate(url);
}
public String getMissingUsernameErrorMessage() {
return missingUsernameLocator.textContent();
}
public String getErrorMessage () {
return invalidCredentialsLocator.textContent();
}
}
DashboardPage
package com.example.pages;
import com.microsoft.playwright.Locator;
import com.microsoft.playwright.Page;
public class DashboardPage {
private Page page;
private final Locator headingLocator;
public DashboardPage(Page page){
this.page = page;
this.headingLocator = page.locator("//h6[contains(@class, \"oxd-topbar-header-breadcrumb-module\")]");
}
public String getDashboardPageHeading() {
return headingLocator.textContent();
}
}
8. Create the step package in src/test/java
package com.example.steps;
import com.example.utils.PlaywrightFactory;
import com.microsoft.playwright.Page;
public class BaseStep {
protected Page getPage() {
Page page = PlaywrightFactory.getPage();
if (page == null) {
throw new RuntimeException(
"Page is NULL. Ensure @Before hook ran and glue includes hooks package."
);
}
return page;
}
}
9. Create the step definition class in src/test/java
Create the step definition class corresponding to the feature file to test the scenarios in the src/test/java directory. The StepDefinition files should be created in this definitionsdirectory within the folder called definitions.
Below is the step definition of the LoginPage feature file.
package com.example.definitions;
import com.example.steps.BaseStep;
import com.example.pages.DashboardPage;
import com.example.pages.LoginPage;
import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;
import org.testng.Assert;
public class LoginPageDefinitions extends BaseStep {
private LoginPage loginPage;
private DashboardPage dashboardPage;
@Given("User is on HRMLogin page {string}")
public void userIsOnHRMLoginPage(String baseUrl) {
loginPage = new LoginPage(getPage());
loginPage.getUrl(baseUrl);
}
@When("User enters username as {string} and password as {string}")
public void userEntersUsernameAsAndPasswordAs(String username, String password) {
loginPage.login(username, password);
}
@Then("User should be able to see error message {string}")
public void userShouldBeAbleToSeeErrorMessage(String expectedErrorMessage) {
Assert.assertEquals(loginPage.getErrorMessage(), expectedErrorMessage);
}
@Then("User should be able to see a message {string} below Username")
public void userShouldBeAbleToSeeAMessageBelowUsername(String expectedErrorMessage) {
Assert.assertEquals(loginPage.getMissingUsernameErrorMessage(), expectedErrorMessage);
}
@Then("User should be able to login successfully and new page opens with heading {string}")
public void userShouldBeAbleToLoginSuccessfullyAndNewPageOpen(String expectedHeading) {
dashboardPage = new DashboardPage(getPage());
Assert.assertEquals(dashboardPage.getDashboardPageHeading(), expectedHeading);
}
}
10. Create a TestNG Cucumber Runner class in src/test/java
We need to create a class called Runner class to run the tests. This class will use the TestNG annotation @RunWith(), which tells TestNG what is the test runner class. TestRunner should be created under src/test/java within the folder called runner.
AbstractTestNGCucumberTestsis a class provided by the Cucumber framework when we are integrating Cucumber with the TestNG test framework. This abstract class helps facilitate the execution of Cucumber tests using TestNG.
We need to specify the values of browser in the TestNG XML file that will pass to the test case file.
Let’s create a testng.xml file to run the test using TestNG.
Right click on the project, New >> File
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Playwright test suite" thread-count="3">
<test name="Chromium Tests">
<classes>
<class name="com.example.runner.RunnerTests"/>
</classes>
</test>
</suite>
12. Run the test through Runner Class
Go to the Runner class and right-click and select Run ‘RunnerTests’. The tests will run as TestNG tests (in IntelliJ).
Below is the test execution result.
13. Run the tests from TestNG.xml
Execute the tests by right-clicking on testng.xml and select Run ‘…testng.xml’
This will run the tests in chromium as mentioned in the .xml file.
The output of the above program is
14. Run the tests from the Command Line
Run the below command in the command prompt to run the tests and to get the test execution report.
mvn clean test
The execution screen looks like something as shown below.
15. Cucumber Report Generation
Add cucumber.properties under src/test/resources and add the below instructions in the file.
cucumber.publish.enabled=true
Below is the image of the Cucumber Report generated using the Cucumber Service.
The screenshot of the failed test is also attached in the Cucumber Report.
Click on the highlighted part. It contains the screenshot.
16. TestNG Report Generation
TestNG generates various types of reports under test-output folder like emailable-report.html, index.html, testng-results.xml.
We are interested in the emailable-report.html report. Open emailable-report.html, as this is an HTML report, and open it with the browser. The below image shows emailable-report.html.
TestNG also produces an “index.html” report, and it resides under the test-output folder. The below image shows the index.html report. This is the latest theme of the report.
Recap of Key Points:
Playwright and Java: We explored the powerful combination of Playwright with Java. We highlighted the benefits of this duo in automated testing.
Build: Maven manage the dependencies and execution.
Cucumber (BDD Framework):It enables Behavior-Driven Development (BDD).
Feature: It uses Gherkin feature file to create the scenarios.
StepDefinition: StepDefinition file contains the test code related to the feature file. When Cucumber executes a Gherkin step in a scenario, it will look for a matching step definition to execute.
Test Runner: It acts as a bridge between the feature and StepDefinition file.
TestNG: It controls the test execution lifecycle. It integrates with Cucumber via TestNG runner.
Hooks: Hooks manage browser setup, teardown, and failure handling
Large test suites can be structured to optimize ease of authoring and maintenance. Page object models are one such approach to structure the test suite. In this tutorial blog, we will learn to create and set up the project using Page Object Model in Playwright Java and TestNG.
Page Object Model (POM) is a design pattern used in test automation that enhances test maintenance and reduces code duplication where web pages are represented as classes, and the various elements on the page are defined as variables in the class and all possible user interactions can then be implemented as methods in the class.
Why is Page Object Model Important?
1. Improved Test Maintenance: POM allows for easy updates to test scripts when the UI changes. By separating the test logic from the page definitions, only the page objects need to be updated rather than all test scripts.
2. Maintainability: In this model, separate classes are created for different pages of a web application like login page, the home page, employee detail page, change password page, etc. So, if there is any change in any element of a website then we only need to make changes in one class, and not in all classes.
3. Reusability of Code: Shared page objects can be reused across multiple test cases, reducing redundancy. This ensures consistent interactions with web elements and actions, making the test code more efficient and easier to manage.
4. Facilitates Collaboration: A well-structured POM increases collaboration among teams, as each contributor can work on different parts of the application without interfering with one another. This approach helps in managing multiple test scenarios simultaneously.
5. Ease of Debugging: With POM, finding and fixing bugs becomes more straightforward since each page action is located in a specific place. Developers can easily pinpoint the location of a problem without sifting through complex test logic.
System requirements
The following prerequisites are required to be installed on the machine to begin with a smooth setup and installation.
Java 11 or higher
IntelliJ IDE or any other IDE to create a project
Maven
Browsers on which tests need to be run, like Chrome, Firefox, etc.
Implementation Steps
1. Create a new Maven Project
The first step in setup is to create a new Maven project. I will be using IntelliJ in this tutorial. The following steps need to be followed to create a new Maven project :
Open IntelliJ, Navigate to File >> New >> Project
2. In the New Project window, enter the following details:
Name of the Project – Playwright_TestNG_Demo
Location/path where the project needs to be saved – Documents/Playwright (my location)
Select JDK version — I am using JDK 17
Archetype — Search for “quickstart” and select maven-archetype-quickstart from the results
Click on the Create button to create the project.
This will create a project as shown below in the IntelliJ.
2. Set Up the Environment
Add the Playwright and TestNG dependencies to the pom.xml. The latest dependency can be found here.
After adding the dependency, refresh the project. We will see that the Playwright jar files are downloaded in External Libraries.
3. Project Structure for Maintainability
Creating a well-organized project structure is crucial for maintaining a scalable and efficient automation framework.
1. Page Classes (src/main/java/.../pages/): Each web page in your application should have a corresponding page class. This class encapsulates all the functionalities of the web page, following the Page Object Model.
2. Utility Classes (src/main/java/.../utils/): These classes can include various utilities like configuration readers, helper methods for common tasks, etc.
3. Test Classes (src/test/java/.../tests/): Here, you write your actual test cases, utilizing the page classes.
4. Test Data Files (src/test/resources/testdata/): Keep your test data in external files. This ensures easy management. It also helps to avoid hardcoding usernames and passwords.
5. Maven Configuration (pom.xml): This file manages project dependencies, plugins, and other configurations.
6. testng.xml: This file is the central configuration file that defines how the test suite should be executed.
4. Creating Page Object Classes
Page classes serve as an interface to a web page’s UI elements. Each page class corresponds to a page in your application, encapsulating the actions that can be performed on that page.
Identify the UI Elements: Determine all the elements on the web page that your test will interact with, such as text boxes, buttons, and links.
Define Selectors: Store the selectors for these elements in your page class. It’s a best practice to externalize these selectors, but for simplicity, we’ll define them directly in the class.
Implement Actions:Create methods for each action that can be performed on the page, like entering text, clicking buttons, etc.
Creating LoginPage Class
import com.microsoft.playwright.Page;
public class LoginPage {
private Page page;
//Locators
private final String usernameLocator = "input[name='username']";
private final String passwordLocator = "input[name='password']";
private final String submitButton = "button[type='submit']";
private final String errorMessage = "//p[contains(@class, 'oxd-text oxd-text--p oxd-alert-content-text')]";
//Constructor
public LoginPage(Page page){
this.page = page;
}
public void login(String username, String password) {
page.fill(usernameLocator,username);
page.fill(passwordLocator,password);
page.click(submitButton);
}
public String getErrorMessage(){
return page.textContent(errorMessage);
}
}
Creating DashboardPage Class
package org.example.pages;
import com.microsoft.playwright.Page;
public class DashboardPage {
private Page page;
//Locators
private final String dashboardHeading = "//h6[contains(@class, 'oxd-topbar-header-breadcrumb-module')]";
//Constructor
public DashboardPage(Page page) {
this.page = page;
}
// Methods
public String getHeading() {
return page.textContent(dashboardHeading);
}
}
5. Write the Test Scripts
Create a Test files under src/test/java. Use these page classes in your test scripts to perform end-to-end scenarios. This will keep your tests clean and focused on logic rather than details about the UI elements.
A typical test class includes:
Setup Method: Initializes the Playwright browser and other prerequisites before each test.
Test Methods: Individual test cases, each representing a different scenario.
Teardown Method:Closes the browser and performs any cleanup after each test.
We will be automating the following test scenario using Playwright Java and run them in Chromium.
Verify the invalid username generates error message “Invalid credentials”
Creating BaseClass
This class contains the common variables and methods used throughout the project, like setup and teardown methods.
package org.example.utils;
import com.microsoft.playwright.*;
import org.testng.annotations.AfterClass;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.BeforeMethod;
public class BaseTests {
// Shared between all tests in this class.
protected Playwright playwright;
protected Browser browser;
// New instance for each test method.
protected BrowserContext context;
protected Page page;
@BeforeClass
public void launchBrowser() {
playwright = Playwright.create();
browser = playwright.chromium().launch(new BrowserType.LaunchOptions().setHeadless(false));
}
@BeforeMethod
public void createContextAndPage() {
context = browser.newContext();
page = context.newPage();
page.navigate("https://opensource-demo.orangehrmlive.com/web/index.php/auth/login");
}
@AfterMethod
public void closeContext() {
context.close();
}
@AfterClass
public void closeBrowser() {
playwright.close();
}
}
Explanation
1. Playwright playwright:
Initialized to manage the browser automation session, allowing interactions with different browsers.
2. Browser browser:
Declared to store the browser instance that will be shared across test methods within the class.
Playwright playwright;
Browser browser = null;
3. BrowserContext context:
Created anew for each test method to simulate independent browser sessions. This ensures that the browser state doesn’t carry over between tests.
4. Page page:
Represents a single tab or window in a browser used for performing actions and assertions.
BrowserContext context;
Page page;
5. @BeforeClass:
This method is annotated with @BeforeClass, indicating it runs once before any test methods in the current class. The browser is launched in non-headless mode (setHeadless(false)), meaning an actual browser window is opened.
This method runs before each test method in the class, as indicated by the @BeforeMethod annotation. It creates a new BrowserContext and a Page, ensuring each test runs in a clean, isolated state. The method also navigates to a login page (“https://opensource-demo.orangehrmlive.com/web/index.php/auth/login”), setting up the initial state for each test.
Runs once after all test methods in the current class have executed, as indicated by the @AfterClass annotation. Closes the Playwright instance, freeing up all resources used for automation.
package org.example.tests;
import org.example.pages.DashboardPage;
import org.example.pages.LoginPage;
import org.example.utils.BaseTests;
import org.testng.Assert;
import org.testng.annotations.Test;
public class LoginTests extends BaseTests {
@Test
public void loginToApplication(){
LoginPage loginPage = new LoginPage(page);
loginPage.login("Admin","admin123");
DashboardPage dashboardPage = new DashboardPage(page);
String actualHeading = dashboardPage.getHeading();
Assert.assertEquals(actualHeading,"Dashboard", "Unable to login to the application");
}
@Test
public void invalidCredentials() {
LoginPage loginPage = new LoginPage(page);
loginPage.login("Playwright","test");
Assert.assertEquals(loginPage.getErrorMessage(), "Invalid credentials" ,"Incorrect Error Message is displayed");
}
}
Explanation
1. loginToApplication
This method enters correct credentials and login to the application.
2. verifyInvalidCredentials
This method enter the invalid username and password and click on the Submit button. Then, it assert that the error message generated by the application is equal to “Invalid credentials”.
6. Create testng.xml
We need to specify the values of browser in the TestNG XML file that will pass to the test case file. Let’s create a testng.xml file to run the test using TestNG.
Right click on the project, New >> File
Below is the sample testng.xml file.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Page Object Model Playwright test suite">
<test name="Chrome Test">
<parameter name="browser" value="chromium" />
<classes>
<class name="org.example.tests.LoginTests"/>
</classes>
</test>
</suite>
7. Test Execution through IntelliJ
Go to the Test class and right-click and select Run ‘LoginTests’. The tests will run as TestNG tests (in IntelliJ).
Below is the test execution result.
8. Run the tests from TestNG.xml
Execute the tests by right-clicking on testng.xml and select Run ‘…testng.xml’
This will run the tests in chromium as mentioned in the .xml file.
The output of the above program is
9. TestNG Report Generation
TestNG generates various types of reports under test-output folder like emailable-report.html, index.html, testng-results.xml.
We are interested in the “emailable-report.html” report. Open “emailable-report.html“, as this is an HTML report, and open it with the browser. The below image shows emailable-report.html.
TestNG also produces an “index.html” report, and it resides under the test-output folder. The below image shows the index.html report. This is the latest theme of the report.
Recap of Key Points:
Playwright and Java:We explored the powerful combination of Playwright with Java. We highlighted the benefits of this duo in automated testing.
Project Structure:An optimal project layout was discussed, emphasizing maintainability and organization.
Page Object Model (POM): The implementation of POM was outlined, showcasing its importance in creating scalable and maintainable test scripts.
Page Classes:We explored how to create page classes, which are a core component of POM. We also provided examples to illustrate their structure. These examples demonstrate their functionality.
Test Classes:We covered the structure and best practices for writing effective test classes. This ensures clarity and efficiency in your tests.
In this tutorial, we will learn to create and set up the project using Cross Browser Testing in Playwright Java.
Cross-browser testing with Playwright and Java allows us to ensure consistent functionality in a web application. This can be achieved across major browser engines using a single, unified API. Playwright supports Chromium (Chrome/Edge), Firefox, and WebKit (Safari), enabling true engine-level testing rather than browser emulation.
Cross-browser testing is the process of checking whether a website or web application functions correctly and looks consistent across multiple web browsers, operating systems, and devices. This includes checking compatibility with various versions of browsers like Chrome, Firefox, Safari, Edge, and Internet Explorer, among others.
Why is Cross-Browser Testing Important?
1. User Experience: Different users use different browsers. Ensuring your website works smoothly across all of them enhances the overall user experience.
2. Functionality Testing: Different browsers can interpret HTML, CSS, and JavaScript differently. Cross-browser testing ensures all the functionality of your website works correctly, without errors or display issues.
Real Example:
Imagine a large e-commerce website like Amazon. Their site needs to function seamlessly across all browsers. This ensures users can view products. They can add items to the cart. They can also make purchases smoothly. Amazon employs cross-browser testing to ensure that images load correctly. Checkout processes function without glitches. Every element displays properly across various browsers and devices.
System requirements
The following prerequisites are required to be installed on the machine to begin with a smooth setup and installation.
Java 8 or higher
IntelliJ IDE or any other IDE to create a project
Maven
Browsers on which tests need to be run, like Chrome, Firefox, etc.
Implementation Steps
1. Create a new Maven Project
The first step in setup is to create a new Maven project. I will be using IntelliJ in this tutorial. The next steps need to be followed to create a new Maven project :
Open IntelliJ, Navigate to File >> New >> Project
2. In the New Project window, enter the following details:
Name of the Project – Playwright_TestNG_Demo
Location/path where the project needs to be saved – Documents/Playwright (my location)
Select JDK version — I am using JDK 17
Archetype — Search for “quickstart”and select maven-archetype-quickstartfrom the results
Click on the Create button to create the project.
This will create a project as shown in IntelliJ.
2. Setup Playwright with Java
Add the Playwright Java and TestNG dependencies to the pom.xml. The latest dependency can be found here.
Verify that the invalid username generates an error message “Invalid credentials”
Let us create a helper class that includes the methods to initialize and close the browser.
package com.example;
import com.microsoft.playwright.*;
import org.testng.annotations.*;
public class BaseClass {
// Shared between all tests in this class.
Playwright playwright;
Browser browser = null;
// New instance for each test method.
BrowserContext context;
Page page;
@BeforeClass
@Parameters("browser")
void launchBrowser(String browserName) {
playwright = Playwright.create();
BrowserType browserType = null;
if (browserName.equals("chromium")) {
browserType = playwright.chromium();
} else if (browserName.equals("firefox")) {
browserType = playwright.firefox();
} else if (browserName.equals("webkit")) {
browserType = playwright.webkit();
}
browser = browserType.launch(new BrowserType.LaunchOptions().setHeadless(false));
}
@BeforeMethod
void createContextAndPage() {
context = browser.newContext();
page = context.newPage();
page.navigate("https://opensource-demo.orangehrmlive.com/web/index.php/auth/login");
}
@AfterMethod
void closeContext() {
context.close();
}
@AfterClass
void closeBrowser() {
playwright.close();
}
}
Explanation
1. Playwright playwright:
Initialized to manage the browser automation session, allowing interactions with different browsers.
2. Browser browser:
Declared to store the browser instance that will be shared across test methods within the class.
Playwright playwright;
Browser browser = null;
3. BrowserContext context:
Created anew for each test method to simulate independent browser sessions. This ensures that the browser state doesn’t carry over between tests.
4. Page page:
Represents a single tab or window in a browser used for performing actions and assertions.
BrowserContext context;
Page page;
5. @BeforeClass:
This method is annotated with @BeforeClass, indicating it runs once before any test methods in the current class. It is parameterized (@Parameters(“browser”)) to accept a browser name as input, allowing the test to be run with different browsers. Depending on the browserName (chromium, firefox, or webkit), the corresponding Playwright BrowserType is used. The browser is launched in non-headless mode (setHeadless(false)), meaning an actual browser window is opened.
This method runs before each test method in the class, as indicated by the @BeforeMethod annotation. It creates a new BrowserContext and Page, ensuring each test runs in a clean, isolated state. The method also navigates to a login page (“https://opensource-demo.orangehrmlive.com/web/index.php/auth/login”), setting up the initial state for each test.
Runs once after all test methods in the current class have executed, as indicated by the @AfterClass annotation. Closes the Playwright instance, freeing up all resources used for automation.
This method verifies that the title of the web page is “OrangeHRM”.
2. verifyInvalidCredentials
This method enters an invalid username and password and clicks on the Submit button. Then, it asserts that the error message generated by the application is equal to “Invalid credentials”.
4. Create testng.xml
We need to specify the values of the browser in the TestNG XML file that will be passed to the test case file. Let’s create a testng.xml file to run the test using TestNG.
Go to the Test class and right-click and select Run ‘TestNG_Demo_Tests’. The tests will run as TestNG tests (in IntelliJ).
Below is the test execution result.
All the tests have failed with the error message – Parameter ‘browser’ is required by BeforeClass on method launchBrowser but has not been marked @Optional or defined. Parameter is expecting to get the browser name either through the command line or testng.xml which we have not provided.
We can mark the launchBrowser() method with Optional, so that if we don’t provide the browser name through testng.xml, it will run the tests on Chromium.
Execute the tests by right-clicking on crossbrowser_testng.xml and select Run ‘…\crossbrowser_testng.xml’
This will run the tests in all 3 browsers as mentioned in the .xml file.
Chromium
Firefox
Webkit
The output of the above program is
7. TestNG Report Generation
TestNG generates various types of reports under test-output folder like emailable-report.html, index.html, testng-results.xml.
We are interested in the “emailable-report.html” report. Open “emailable-report.html”, as this is an HTML report, and open it with the browser. The image shows emailable-report.html.
TestNG also produces an “index.html” report, and it resides under the test-output folder. The image shows the index.html report.
Conclusion:
Playwright and Java: We explored the powerful combination of Playwright with Java, highlighting the benefits of this duo in automated testing.
Setup & Configuration: Playwright integrates easily with Java projects via Maven or Gradle and manages browser installation automatically. Minimal configuration is required to get started.
TestNG Parameters: Dynamically inject browsers into tests.
Browser Support: Playwright supports Chromium (Chrome, Edge), Firefox, and WebKit (Safari). Tests run against real browser engines, offering high confidence in cross-browser compatibility.
Test Execution Approach: The same test logic can be executed across different browsers. This is done by switching the browser type programmatically or via system properties. This avoids code duplication and improves maintainability.
The following prerequisites are required to be installed on the machine to begin with a smooth setup and installation.
Java 8 or higher
IntelliJ IDE or any other IDE to create a project
Maven
Browsers on which tests need to be run, like Chrome, Firefox, etc.
Implementation Steps
1. Create a new Maven Project
The first step in setup is to create a new Maven project. I will be using IntelliJ in this tutorial. The following steps need to be followed to create a new Maven project :
Open IntelliJ, Navigate to File >> New >> Project
2. In the New Project window, enter the following details:
Name of the Project – Playwright_TestNG_Demo
Location/path where the project needs to be saved – Documents/Playwright (my location)
Select JDK version — I am using JDK 17
Archetype — Search for “quickstart” and select maven-archetype-quickstart from the results
Click on the Create button to create the project.
This will create a project as shown below in the IntelliJ.
2. Update the Dependencies
Add the Playwright and TestNG dependencies to the pom.xml. The latest dependency can be found from here.
After adding the dependency, refresh the project. We will see that the Playwright and testng jar files are downloaded in External Libraries.
3. Create testng.xml
Let’s create a testng.xml file to run the test using TestNG.
Right click on the project, New >> File
It will help us execute the other tests by adding the test method name to the methods block in the file.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Playwright test suite">
<test name="Integration of Playwright Java with TestNG">
<classes>
<class name="com.example.TestNG_Demo_Tests">
</class>
</classes>
</test>
</suite>
4. Write the tests
Create a Test file under src/test/java called – TestNG_Demo_Tests.
We will be automating the following test scenario using Playwright Java.
Initialized to manage the browser automation session, allowing interactions with different browsers.
2. Browser browser
Declared to store the browser instance that will be shared across test methods within the class.
Playwright playwright;
Browser browser = null;
3. BrowserContext context
Created anew for each test method to simulate independent browser sessions. This ensures that the browser state doesn’t carry over between tests.
4. Page page
Represents a single tab or window in a browser used for performing actions and assertions.
BrowserContext context;
Page page;
5. @BeforeClass
This method is annotated with @BeforeClass, indicating it runs once before any test methods in the current class.
The browser is launched in Chromium and non-headless mode (setHeadless(false)), meaning an actual browser window is opened.
6. @BeforeMethod createContextAndPage()
This method runs before each test method in the class, as indicated by the@BeforeMethod annotation. It creates a new BrowserContextand a Page, ensuring each test runs in a clean, isolated state. The method also navigates to a login page (“https://opensource-demo.orangehrmlive.com/web/index.php/auth/login”), setting up the initial state for each test.
Runs once after all test methods in the current class have executed, as indicated by the @AfterClass annotation. Closes the Playwright instance, freeing up all resources used for automation.
Go to the Test class and right-click and select Run ‘TestNG_Demo_Tests’. The tests will run as TestNG tests (in IntelliJ).
By default, the tests are run in headless mode, so the browser UI is not loaded.
Below is the test execution result.
6. Run the tests from TestNG.xml
Execute the tests by right-clicking on testng.xml and select Run ‘…\testng.xml’
Below is the test execution result.
7. TestNG Report Generation
TestNG generates various types of reports under test-output folder like emailable-report.html, index.html, testng-results.xml.
We are interested in the “emailable-report.html” report. Open “emailable-report.html“, as this is an HTML report, and open it with the browser. The below image shows emailable-report.html.
TestNG also produces an “index.html” report, and it resides under the test-output folder. The below image shows the index.html report. This is the latest theme of the report.
Are you familiar with Rest API and want to test your understanding? This post contains 25 useful TestNG multiple-choice questions (quiz) to self-test your knowledge of TestNG .
1) 1) A testing framework for java inspired by JUnit
2) 2) Cédric Beust
3) 2) After
4) 2) Next Generation
5) 4) All of the above
TestNG offers extensive control over the execution of test cases, including setting dependencies, priorities, and parallel execution.
Prior to running test case X, run multiple test cases as a pre-request – TestNG allows us to set dependencies between test methods using dependsOnMethods or dependsOnGroups, so you can ensure that certain test cases run before others.
TestNG provides powerful features such as annotations, test grouping, sequencing, and parameterization, which resolve the limitations found in earlier frameworks like JUnit.
6) 4) None of the above
7) 4) All of the above
8) 2) Independently of
This means that each test case can be executed on its own without depending on the execution or results of other test cases. However, TestNG also allows us to create dependencies between test cases using the dependsOnMethods or dependsOnGroups attributes if needed.
9) 1) Java
10) 4) All of the above
11) 3) XML
In TestNG, an XML file (commonly named as testng.xml) is used to define and configure test suites, test groups, and the sequence of test execution.
12) 2) Testng.xml
Sample testng.xml
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >
<suite name="SampleSuite">
<test name="SampleTest">
<classes>
<class name="com.example.MyTestClass" />
</classes>
</test>
</suite>
13) 4) All of the above
14) 3) Both A and B
15) 4) All of the above
16) 1) BeforeSuite
17) 3) Before each and every test method in the class i.e., the method with @BeforeMethod annotation will execute for every test method in the class
18) 2) Once before the execution of test methods in the class begins
19) 1) AfterSuite
@AfterSuite: This method is executed after all the test methods in an entire suite have been completed.
20) 2) Annotation
21) 3) BeforeTest
@BeforeTest: This method is executed before any test methods in the <test> tag in the testng.xml file, which can include multiple classes and folders.
22) 2) AfterTest
@AfterTest: This method is executed after all the test methods in the <test> tag in the testng.xml file have been run, which can include multiple classes.
23) 2) BeforeClass
@BeforeClass: This method is invoked before the first test method in the current class is executed.
24) 3) AfterClass
The annotations have different scopes:
@AfterClass:This method is invoked after all the test methods in the current class have been executed.
@AfterMethod: Runs after each individual test method.
@AfterSuite: Runs once after all the tests in the entire suite have finished.
@AfterTest: Runs after all test methods in the <test> tag of the testng.xml file.
25) 1) BeforeMethod
Each of these annotations is executed before a test method, but at different levels:
@BeforeMethod: Runs before each individual test method in the current class.
@BeforeTest: Runs before any test methods in the <test> tag in the testng.xml file.
@BeforeClass: Runs before the first test method in the current class.
@BeforeSuite: Runs once before all tests in the entire suite defined in the testng.xml file.
Are you familiar with TestNG and want to test your understanding? This post contains 25 useful TestNG multiple-choice questions (quiz) to self-test your knowledge.
QA Automation Expert 