InnvocationCount is one of the feature available in TestNG. InvocationCount is used when we want to run the same test multiple times. If we want to run single @Test 10 times at a single thread, then invocationCount can be used. To invoke a method multiple times, below syntax is used.
@Test(invocationCount = 3)
In this example, the @Test method will execute for 3 times each on a single thread.
In this tutorial, we will illustrate how to get the current invocation count.
Step 1 − Create a TestNG class, TestInvocationCount.
Step 2 − Write two @Test methods in the class TestInvocationCountas shown in the programming code section below. Add invocationCount=3 to method verifyLinkedIn and 2 to validLoginTest.
Step 3 − Create the testNG.xml as given below to run the TestNG classes.
Step 4 − Now, run the testNG.xml or directly testNG class in IDE or compile and run it using command line.
Step 5 − In the output, the user can see a total of 1 thread running sequentially for all invocations of @Test.
import static org.testng.Assert.assertTrue;
import java.util.concurrent.TimeUnit;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.testng.Assert;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;
import io.github.bonigarcia.wdm.WebDriverManager;
public class TestInvocationCount {
WebDriver driver;
@BeforeMethod
public void setup() throws Exception {
driver = WebDriverManager.firefoxdriver().create();
driver.get("https://opensource-demo.orangehrmlive.com/");
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
driver.manage().window().maximize();
}
@Test(invocationCount = 3)
public void verifyLinkedIn() {
System.out.println("Test Case 1 with Thread Id - "+Thread.currentThread().getId());
driver.manage().window().maximize();
Boolean linkedInIcon = driver.findElement(By.xpath("//*[@id='social-icons']/a[1]/img")).isEnabled();
assertTrue(linkedInIcon);
}
@Test(invocationCount = 2)
public void validLoginTest() throws InterruptedException {
System.out.println("Test Case 2 with Thread Id - "+Thread.currentThread().getId());
driver.findElement(By.name("txtUsername")).sendKeys("Admin");
driver.findElement(By.name("txtPassword")).sendKeys("admin123");
driver.findElement(By.id("btnLogin")).click();
String expectedTitle = driver.findElement(By.xpath("//*[@id='content']/div/div[1]/h1")).getText();
Assert.assertTrue(expectedTitle.contains("Dashboard"));
}
@AfterMethod
public void closeBrowser() {
driver.quit();
}
}
testng.xml
This is a configuration file that is used to organize and run the TestNG test cases. It is very handy when limited tests are needed to execute rather than the full suite.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
<test name="Invocation Test">
<classes>
<class name="com.example.listeners.TestInvocationCount"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->
The output of the above program is
We can add threadPoolSize to the @Test.
threadPoolSize – It defines the size of the thread pool for any method. The method will be invoked from multiple threads as specified by invocationCount.
@Test(invocationCount = 3, threadPoolSize)
Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!
TestNG is a well thought Test Framework. It provides a lot of different features which makes the life of a tester a little easy. It happens sometimes that a test execution fails, but the failure is not a product bug, but there can be different reasons for the failure such as the environment is down, third party web service is down, or the browser becomes unresponsive. Imagine a scenario where we need to run a test suite consisting of 100 tests and a few tests failed as a result of a known intermittent environment issue. We know that these tests can pass if rerun a couple of times. So, in this case, the retry functionality of TestNG comes to the rescue. This is one of the best and most frequently used functionality.
In this tutorial let us study how we can implement retry on failed tests in TestNG. In order to achieve this, we have to first understand the org.testng.IRetryAnalyzer interface.
To start with, please add the below dependencies to the Maven Project.
IRetryAnalyzer – It is an interface to implement to be able to have a chance to retry a failed test. The definition of this interface is
public interface IRetryAnalyzer {
/**
* Returns true if the test method has to be retried, false otherwise.
*
* @param result The result of the test method that just ran.
* @return true if the test method has to be retried, false otherwise.
*/
boolean retry(ITestResult result);
}
This method implementation returns true if you want to re-execute your failed test and false if you don’t want to re-execute your test.
When you bind a retry analyzer to a test, TestNG automatically invokes the retry analyzer to determine if TestNG can retry a test case again in an attempt to see if the test that just fails now passes. Here is how you use a retry analyzer:
Bind this implementation to the @Test annotation for e.g., @Test(retryAnalyzer = Retry.class)
Build an implementation of the interface org.testng.IRetryAnalyzer
1. Add IRetryAnalyzer to the @TestAnnotation
First of all, you need to create a class that implements the IRetryAnalyzer like the below example:
import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;
public class Retry implements IRetryAnalyzer {
int retryCount = 0;
int maxRetryCount = 2;
public boolean retry(ITestResult result) {
if(!result.isSuccess()) { //Check if test is failed
if(retryCount<maxRetryCount) { //Check if the maximum number of test execution is reached
System.out.println("Retrying Test : Re-running " + result.getName() +
" for " + (retryCount+1) + " time(s)."); //Print the number of Retry attempts
retryCount++; //Increase the maxRetryCount by 1
result.setStatus(ITestResult.FAILURE); //Mark test as failed
return true; //Rerun the failed test
} else {
result.setStatus(ITestResult.FAILURE); //TestNG marks last run as failed, if last run is max retry
}
}else {
result.setStatus(ITestResult.SUCCESS); //TestNG parks test as passed when the test test passes
}
return false;
}
}
This example shows that failed test case will run 3 times till it passes. In case it fails the third time, test execution will stop and TestNG will mark this case as failed. We can change the number of tries by changing the value of maxRetryCount.
Using retryAnalyzer attribute in the @Test annotation
The next step is to associate your test cases with IRetryAnalyzer. In order to do this, you need to use the method below.
@Test(retryAnalyzer = Retry.class)
public void verifyLoginPage() {
}
Let us see the complete implementation with the help of the below example.
In the above example, test – verifyLoginPage() will be retried a maximum of 3 times, if the test fails. To run the tests, Right-click on the class and select Run As ->TestNG Suite.
The output of the above program is
2. Implement Interface ITestAnnotationTransformer to retry failed tests
In this case, you would need to implement ITestAnnotationTransformer interface. The implementation of this interface is
public interface IAnnotationTransformer extends ITestNGListener {
/**
* This method will be invoked by TestNG to give you a chance to modify a TestNG annotation read
* from your test classes. You can change the values you need by calling any of the setters on the
* ITest interface.
*
* <p>Note that only one of the three parameters testClass, testConstructor and testMethod will be
* non-null.
*
* @param annotation The annotation that was read from your test class.
* @param testClass If the annotation was found on a class, this parameter represents this class
* (null otherwise).
* @param testConstructor If the annotation was found on a constructor, this parameter represents
* this constructor (null otherwise).
* @param testMethod If the annotation was found on a method, this parameter represents this
* method (null otherwise).
*/
default void transform(
ITestAnnotation annotation, Class testClass, Constructor testConstructor, Method testMethod) {
// not implemented
}
The transform method is called for every test during the test run. We can use this listener for our retry analyzer as shown below:
import java.lang.reflect.Constructor;
import java.lang.reflect.Method;
import org.testng.IAnnotationTransformer;
import org.testng.annotations.ITestAnnotation;
public class RetryListener implements IAnnotationTransformer{
public void transform(ITestAnnotation arg0, Class arg1, Constructor arg2,Method arg3) {
arg0.setRetryAnalyzer(Retry.class);
}
}
Now let us create a class that contains all the tests.
import java.util.concurrent.TimeUnit;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.testng.Assert;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;
import io.github.bonigarcia.wdm.WebDriverManager;
public class RetryTests {
WebDriver driver;
@BeforeTest
public void setUp() {
WebDriverManager.chromedriver().setup();
ChromeOptions chromeOptions = new ChromeOptions();
driver = new ChromeDriver(chromeOptions);
driver.get("https://opensource-demo.orangehrmlive.com/");
driver.manage().window().maximize();
driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
}
@Test(description = "This test validates title of login functionality")
public void verifyLoginPage() {
String expectedTitle = driver.findElement(By.xpath("//*[@id='logInPanelHeading']")).getText();
System.out.println("Title :" + expectedTitle);
Assert.assertTrue(expectedTitle.equalsIgnoreCase("LOGIN Panel !!"));
}
@Test(description = "This test validates successful login to Home page")
public void verifyHomePage() {
System.out.println("Username Entered");
driver.findElement(By.name("txtUsername")).sendKeys("Admin");
System.out.println("Password Entered");
driver.findElement(By.name("txtPassword")).sendKeys("admin123");
driver.findElement(By.id("btnLogin")).submit();
String newPageText = driver.findElement(By.id("welcome")).getText();
System.out.println("newPageText :" + newPageText);
Assert.assertTrue(newPageText.contains("Welcome"));
}
@AfterTest
public void teardown() {
driver.quit();
}
}
Once we have the implementation of IAnnotationTransformer, we just need to add it as a listener in the testng.xml. Like this:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
<listeners>
<listener class-name="com.example.retrydemo.RetryListener"></listener>
</listeners>
<test name="Test">
<classes>
<class name="com.example.retrydemo.RetryTests"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->
Now let us run the tests. Right-click on testng.xml and select Run As -> TestNG Suite.
The output of the above program is
This is pretty much it on this topic. Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!
A batch file (.bat) is used in DOS and Windows, which is an unformatted text file that consists of a series of commands to be executed by the command line interpreter.
Pre-Requisite:
Selenium
TestNG
Maven
Java 11
Maven Complier Plugin
Maven Surefire Plugin
Notepad
Let us first create some tests in a class.
import static org.testng.Assert.assertTrue;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.MatcherAssert.assertThat;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;
public class TestNGRunFromCommandLine {
WebDriver driver;
@BeforeTest
public void setUp() {
System.setProperty("webdriver.gecko.driver",
"C:\\Users\\Vibha\\Software\\geckodriver\\geckodriver.exe");
driver = new FirefoxDriver();
driver.get("https://opensource-demo.orangehrmlive.com/");
driver.manage().window().maximize();
}
@Test(description = "This test validates title of login functionality", priority = 0)
public void verifyLoginPage() {
String expectedTitle = driver.findElement(By.xpath("//*[@id='logInPanelHeading']")).getText();
System.out.println("Title :" + expectedTitle);
assertTrue(expectedTitle.equalsIgnoreCase("LOGIN Panel12"));
}
@Test(description = "This test validates successful login to Home page", priority = 1)
public void verifyHomePage() throws InterruptedException {
System.out.println("Username Entered");
driver.findElement(By.name("txtUsername")).sendKeys("Admin");
System.out.println("Password Entered");
driver.findElement(By.name("txtPassword")).sendKeys("admin123");
driver.findElement(By.id("btnLogin")).submit();
Thread.sleep(2000);
String newPageText = driver.findElement(By.xpath("//*[@id='content']/div/div[1]/h1")).getText();
System.out.println("newPageText :" + newPageText);
assertThat(newPageText, containsString("Dashboard"));
}
@AfterTest
public void teardown() {
driver.quit();
}
}
The below is the testng.xml file which has a class that we have created above and we will be invoking this xml file using batch file (.bat).
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
<test name="Test">
<classes>
<class name="TestNG_Demo.TestNGRunFromCommandLine"/>
</classes>
</test>
</suite>
How to create a batch file?
Step 1: Open notepad Step 2: Paste the below lines of code – You may need to add your project location. In the example, project location is set as C:\Users\Vibha\Projects\Vibha_Personal\ParallelTestsTestNG Step 3: Save the file as ‘TestNGProject.bat’ in location that you want to save.
cd C:\Users\Vibha\Projects\Vibha_Personal\ParallelTestsTestNG
mvn compile test
Now, to run the tests, double click on the TestNGProject.bat file and all the commands mentioned in the file will be executed one by one.
As we know, TestNG generates lot of Reports automatically. We are going to look into 2 reports – emailable-report.html and index.html. The reports are generated under surefire-reports folder within target directory.
Emailable-Report.html
Index.html
Hope this article helps you to invoke your tests using .bat file.
The previous tutorial has explained the Integration of Selenium with TestNG and the tests are executed through either TestNG Suite or testng.xml. This tutorial explains the steps to run the TestNG Tests through the command line.
Pre-Requisite
Selenium
TestNG
Maven
Java 11
Maven Compiler Plugin
Maven Surefire Plugin
Imagine we need to run the TestNG Tests in CI/CD pipelines like Jenkins or GitLab, then we can’t right click and select TestNG Suite or tesng.xml to run the tests. In such situations, the tests can be executed through command line.
We need to add plugins to pom.xml to compile the test code and then run the tests. To know more about Maven Surefire Plugin for TestNG, refer this blog.
Create sample class which has @Test methods. In the example below we have created a class as below:
import static org.testng.Assert.assertTrue;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.MatcherAssert.assertThat;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;
public class TestNGRunFromCommandLine {
WebDriver driver;
@BeforeTest
public void setUp() {
System.setProperty("webdriver.gecko.driver",
"C:\\Users\\Vibha\\Software\\geckodriver\\geckodriver.exe");
driver = new FirefoxDriver();
driver.get("https://opensource-demo.orangehrmlive.com/");
driver.manage().window().maximize();
}
@Test(description = "This test validates title of login functionality", priority = 0)
public void verifyLoginPage() {
String expectedTitle = driver.findElement(By.xpath("//*[@id='logInPanelHeading']")).getText();
System.out.println("Title :" + expectedTitle);
assertTrue(expectedTitle.equalsIgnoreCase("LOGIN Panel"));
}
@Test(description = "This test validates successful login to Home page", priority = 1)
public void verifyHomePage() {
System.out.println("Username Entered");
driver.findElement(By.name("txtUsername")).sendKeys("Admin");
System.out.println("Password Entered");
driver.findElement(By.name("txtPassword")).sendKeys("admin123");
driver.findElement(By.id("btnLogin")).submit();
String newPageText = driver.findElement(By.xpath("//*[@id='content']/div/div[1]/h1")).getText();
System.out.println("newPageText :" + newPageText);
assertThat(newPageText, containsString("Dashboard"));
}
@AfterTest
public void teardown() {
driver.quit();
}
}
The below is the testng.xml file, which will execute all the tests that are available under ‘TestNGRunFromCommandLine‘ class.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
<test name="Test">
<classes>
<class name="TestNG_Demo.TestNGRunFromCommandLine"/>
</classes>
</test>
</suite>
Below commands are used to execute ‘testng.xml’ file from command line. First, we need to go the place where pom.xml of the project is placed. Then use the mvn compile test to compile the code and execute the TestNG tests.
cd C:\Users\Vibha\Projects\Vibha_Personal\ParallelTestsTestNG
mvn compile test
After executing above command, it should execute the tests that we have specified in testng.xml file. Below is the screen shot after the execution of the tests.
This execution generate various TestNG Reports. We are concerned about emailable-report.html and index.html.
Emailable-Report.html
Emailable report is a type of summary report that one can transfer to other people in the team through any medium. Click on option “emailable-report.html”. Click on option web browser. The output reports in TestNG reporting will looks like below:
Index.html
Index report contains the index-like structure of different parts of the report, such as failed tests, test file, passed test, etc.
Right click on the index.html from the project directory. Select option open with web browser option.
The result will look like this:
Congratulations. This tutorial has explain running the tests of TestNG using Command Line. Happy Learning!!
The previous tutorial has explained the DataProviders in TestNG. The DataProvider in TestNG is a way to pass the parameters in the test functions. Using DataProvider in TestNG, we can easily inject multiple values into the same test case. It comes inbuilt into TestNG and is popularly used in data-driven frameworks.
It is an option for the parallel execution of tests in TestNG.
It is advisable to create 2 classes – one class contains the Test cases and another class defines TestNG parameters – DataProviders.
Let us create a class for the DataProvider method with all the Test Data as shown below:
import org.testng.annotations.DataProvider;
public class DataProviderDemo {
@DataProvider(name = "testData", parallel=true)
public Object[][] dataProvFunc() {
return new Object[][] {
{"","","Username cannot be empty"},
{"","Test","Username cannot be empty"},
{"$%1234","2345%$","Invalid credentials"}
};
}
}
An extra parameter “parallel” is required to initiate parallel execution in TestNG using the data provider.
Below is the test which uses the parameter from the data provider and runs the tests parallelly.
A new ThreadLocal is instantiated for each test class, since it’s in the BeforeClass annotation.
private static final ThreadLocal<WebDriver> WEB_DRIVER_THREAD_LOCAL = new ThreadLocal<WebDriver>();
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite" thread-count="2" data-provider-thread-count="2">
<test name="Test">
<classes>
<class name="DataProvider.DataProviderParallelTests"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->
In this file, the data-provider-thread-count is set to 2, then two browsers will be opened, and the first two tests will run from the list.
Run the test script from testng.xml, Right-Click on the XML, and select Run As ->TestNG Suite.
The execution status shown below shows that 2 threads are active at a time, which execute 2 sets of data provider parameters – Thread 14 and Thread 15. Once the tests are finished for Thread 14 and Thread 15, they are closed and a new Thread 15 is again initiated to start the test execution of the 3rd parameter.
TestNG generates multiple test reports under the folder test-output. We are mainly concerned about 2 reports – emailable-report.html and index.html.
Emailable-report.html
Emailable reports are a type of summary report that one can transfer to other people in the team through any medium.
Index.html
Index report contains the index-like structure of different parts of the report, such as failed tests, test files, passed tests, etc. We can divide this report into two parts. The left part contains the index, and this is the reason it is called an index report, while the right part contains the explored content of that index.
We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
I have created an Excel – SearchInBing.xlsx and placed it on the Desktop. You can create a Test Data folder in your project and place the Excel file within it. So, my datasheet looks like the below:
Next, we will create a DataProvider method that will use another method to read the Excel file & create a 2D object from the row & column values of the Excel and return the same value, so that our test script can use it. The code for it would look like the below:
import org.apache.poi.ss.usermodel.Cell;
import org.testng.annotations.DataProvider;
import java.io.FileInputStream;
import java.io.IOException;
import org.apache.poi.xssf.usermodel.XSSFRow;
import org.apache.poi.xssf.usermodel.XSSFSheet;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
public class ExcelDataProvider {
@DataProvider(name = "excelData")
public Object[][] excelDataProvider() throws IOException {
// We are creating an object from the excel sheet data by calling a method that
// reads data from the excel stored locally in our system
Object[][] arrObj = getExcelData(
C:\\Users\\Vibha\\Desktop\\SearchInBing.xlsx","Details");
return arrObj;
}
// This method handles the excel - opens it and reads the data from the
// respective cells using a for-loop & returns it in the form of a string array
public String[][] getExcelData(String fileName, String sheetName) throws IOException {
String[][] data = null;
try {
FileInputStream fis = new FileInputStream(fileName);
XSSFWorkbook workbook = new XSSFWorkbook(fis);
XSSFSheet sheet = workbook.getSheet(sheetName);
XSSFRow row = sheet.getRow(0);
int noOfRows = sheet.getPhysicalNumberOfRows();
int noOfCols = row.getLastCellNum();
Cell cell;
data = new String[noOfRows - 1][noOfCols];
for (int i = 1; i < noOfRows; i++) {
for (int j = 0; j < noOfCols; j++) {
row = sheet.getRow(i);
cell = row.getCell(j);
data[i - 1][j] = cell.getStringCellValue();
}
}
} catch (Exception e) {
System.out.println("The exception is: " + e.getMessage());
}
return data;
}
}
Now, create a class that contains the test code. By default, the data provider will be looked for in the current test class or one of its base classes. If you want to put your data provider in a different class, it needs to be a static method or a class with a no-arg constructor, and you specify the class where it can be found in the data provider class attribute.
In the last tutorial, I have explain the Parameters in TestNG which passes different test data to the test case as arguments. Similar to TestNG Parameters, DataProviders are a means to pass data to test scripts in TestNG. In this tutorial, I will explain about the DataProviders in TestNG.
What is DataProvider in TestNG?
The DataProvider in TestNG is another way to pass the parameters in the test function, the other one being TestNG parameters. Using DataProvider in TestNG, we can easily inject multiple values into the same test case. It comes inbuilt in TestNG and is popularly used in data-driven frameworks.
Syntax of DataProvider
@DataProvider (name = "name_of_dataprovider")
public Object[][] dpMethod() {
return new Object [][] { values}
}
A Data Provider is a method on the class that returns an array of array of objects. This method is annotated with @DataProvider
A @Test method specifies its Data Provider with the dataProvider attribute. This name must correspond to a method on the same class annotated with @DataProvider(name=”…”) with a matching name.
TestNG dataprovider returns a 2d list of objects..An array of array of objects (Object[][]) where the first dimension’s size is the number of times the test method will be invoked and the second dimension size contains an array of objects that must be compatible with the parameter types of the test method.
DataProviders are not declared on top of the functions like TestNG parameters but have a method of their own, which in regular speaking terms called a dataprovider method. For example, dpMethod here.
The dataprovider name calls the dataprovider method, and if there is no name specified by the tester, then the dataprovider method is the default name used in the receiving @Test case.
Data providers can run in parallel with the attribute parallel.
Below is the basic example of using DataProvider in TestNG.
import org.openqa.selenium.By;
import org.openqa.selenium.Keys;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
public class DataProviderDemo {
WebDriver driver;
@DataProvider(name = "testData")
public Object[][] dataProvFunc() {
return new Object[][] { { "Selenium" }, { "TestNG" } };
}
@BeforeMethod
public void setUp() {
System.out.println("Start the test");
System.setProperty("webdriver.chrome.driver",
"C:\\Users\\Vibha\\Software\\chromedriver\\chromedriver.exe");
driver = new ChromeDriver();
driver.get("https://www.bing.com/");
driver.manage().window().maximize();
}
// Passing the dataProvider to the test method through @Test annotation
@Test(dataProvider = "testData")
public void search(String keyWord) {
WebElement txtBox = driver.findElement(By.id("sb_form_q"));
txtBox.sendKeys(keyWord);
System.out.println("Keyword entered is : " + keyWord);
txtBox.sendKeys(Keys.ENTER);
System.out.println("Search result is displayed.");
}
@AfterMethod
public void burnDown() {
driver.quit();
System.out.println("End the test");
}
}
In the above example, I am passing two search keywords, viz “Selenium” and “TestNG” to the test method using the DataProvider method. You can run the code and check the output. It will be as shown below-
Here, Test is executed with two values, but we have run the test only once.
Inheriting DataProvider in TestNG
It is messy to have supporting methods like DataProvider and test code in one class. It is always preferred to declare the test case in one class and define TestNG parameters like DataProviders in another class. By default, the data provider will be looked for in the current test class or one of its base classes. If you want to put your data provider in a different class, it needs to be a static method or a class with a non-arg constructor, and you specify the class where it can be found in the dataProviderClass attribute.
Let us create separate classes for the DataProvider method and the test method, as shown below:
DataProvider Class
public class DPDemo {
@DataProvider(name = "testData")
public Object[][] dataProvFunc() {
return new Object[][] {
{ "Selenium" }, { "TestNG" }, { "Automation" } };
}
}
We can see that all we did was create a DataProvider method in a Class and create a new class for Test Code.
public class DataProviderInheritanceDemo {
WebDriver driver;
@BeforeMethod
public void setUp() {
System.out.println("Start the test");
System.setProperty("webdriver.chrome.driver",
"C:\\Users\\Vibha\\Software\\chromedriver\\chromedriver.exe");
driver = new ChromeDriver();
driver.get("https://www.bing.com/");
driver.manage().window().maximize();
}
// Passing the dataProvider to the test method through @Test annotation
@Test(dataProvider = "testData", dataProviderClass = DPDemo.class)
public void search(String keyWord) {
WebElement txtBox = driver.findElement(By.id("sb_form_q"));
txtBox.sendKeys(keyWord);
System.out.println("Keyword entered is : " + keyWord);
txtBox.sendKeys(Keys.ENTER);
System.out.println("Search result is displayed.");
}
@AfterMethod
public void burnDown() {
driver.quit();
System.out.println("End the test");
}
}
As you can see, to handle the inheritance, all we did was add an attribute to the test method (highlighted above), which specifies the class that has the DataProvider method.
Passing Multiple Parameter Values in TestNG DataProviders
Passing multiple values is pretty similar to passing numerous parameters. The only difference is that we will pass various values to a single parameter so that a string of input(s) is sent in one go.
Let us quickly understand this concept with the help of the code as shown below.
DataProvider Class
public class DPDemo {
@DataProvider(name = "testData")
public Object[][] dataProvFunc() {
return new Object[][] { { "Automation Tester", "2-5 years" }, { "Performance Tester", "3+ years" },
{ "DevOps", "5+ years" } };
}
}
Test Code – DataProviderInheritanceDemo
public class DataProviderInheritanceDemo {
WebDriver driver;
@BeforeMethod
public void setUp() {
System.out.println("Start the test");
System.setProperty("webdriver.chrome.driver",
"C:\\Users\\Vibha\\Software\\chromedriver\\chromedriver.exe");
driver = new ChromeDriver();
driver.get("https://www.bing.com/");
driver.manage().window().maximize();
}
// Passing the dataProvider to the test method through @Test annotation
@Test(dataProvider = "testData", dataProviderClass = DPDemo.class)
public void search(String keyWord1, String keyWord2) {
WebElement txtBox = driver.findElement(By.id("sb_form_q"));
txtBox.sendKeys(keyWord1, keyWord2);
System.out.println("Keyword entered is : " + keyWord1 + " " + keyWord2);
txtBox.sendKeys(Keys.ENTER);
System.out.println("Search result is displayed.");
}
@AfterMethod
public void burnDown() {
driver.quit();
System.out.println("End the test");
}
}
Run the test script, and you will see both the values for the TestNG parameters being passed in one go, the output for it would be as follows-
We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
There are 2 ways to run TestNG tests – using Run As TestNG tests and from testng.xml.
In this tutorial, I’m explaining how to create and run tests using TestNG.xml.
Step 1 – In the below project, there is no testng.xml present.
Step 2 – Right click on class name “API_Test.java” and navigate “TestNG–>Convert to TestNG“.
Step 3 – Now a new window for testng.xml will be generated as shown below. Click the Finish Button.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
<test thread-count="5" name="Test">
<classes>
<class name="com.example.RestAssured_TestNG_Demo.API_Test"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->
Step 4 – A xml file named “testng.xml” will be generated in project hierarchy.
Step 5 – Right click on this testng.xml and select Run As–> Testng Suite.
Step 6 – You will view the result in two different tabs: Console and “Results of running suite”. Below is the image of Rest of running suite tab.
Multiple Classes
Let us imagine a sceanrio where there are 3 classes and we want to run them all together, you can done that by creating a testng.xm and mention all 3 classes in that testng.xml.
Select all 3 classes and right click and navigate “TestNG–>Convert to TestNG”
This is how the testng.xml will look like for multiple classes.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
<test thread-count="5" name="Test">
<classes>
<class name="com.example.RestAssured_TestNG_Demo.API_Test1"/>
<class name="com.example.RestAssured_TestNG_Demo.Test3"/>
<class name="com.example.RestAssured_TestNG_Demo.Test2"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->
Right click on this testng.xml and select Run As–> Testng Suite. You will get the result in two tabs: Console and “Results of running suite”. Rhis shows that all the tests present withing the three classes are executed.
Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!
Hard Assertion throws AssertionError immediately when an Assert Condition fails and moves to the next @Test method
Suppose, there are 2 assertions in a Test and the first assertion fails, then HardAssertion does not execute the second Assertion Condition and declares the test as failed
As you can see in the below example, there are 2 assert conditions under Test – AssertionFailure(). As the first Assert Condition fails, it moved directly to the second test without executing another Assert Condition.
mport org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.Assert;
import org.testng.annotations.Test;
public class HardAssertionDemo {
@Test
public void AssertionFailure() {
System.setProperty("webdriver.gecko.driver", "C:\\Users\\Vibha\\Software\\geckodriver-v0.31.0-win64\\geckodriver.exe");
WebDriver driver = new FirefoxDriver();
driver.get("https://duckduckgo.com/");
String actualTitle = "DuckDuckGo — Privacy, simplified";
String expectedTitle = driver.getTitle();
String expectedText = driver.findElement(By.xpath("/html/body/div/div[2]/div/div[3]/div[1]")).getText();
/* Hard Assert */
System.out.println("Verify Title");
Assert.assertEquals(expectedTitle, actualTitle, "Incorrect page title");
System.out.println("Verify Text");
Assert.assertEquals("Privacy Protection For Any Device", expectedText);
driver.quit();
}
@Test
public void print() {
System.out.println("Hard Assertion is displayed");
}
}
The output of the above program is
What is Soft Assert?
To overcome the above-mentioned problem, there is another type of assertion called Soft Assert.
Soft Assert does not throw an exception when an Assert Condition fails, and continues with the next step after the Assert Condition.
Soft assert does not include by default in TestNG. For this, you need to include the below package :
org.testng.asserts.SoftAssert;
The first step is to create an instance of SoftAssert class.
SoftAssert softAssertion = new SoftAssert();
After this, we can use this softAssert variable instead of hard assert.
Create an object of SoftAssertion to run Assert Conditions
Below is an example of a Soft Assert
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.Test;
import org.testng.asserts.SoftAssert;
public class SoftAssertionDemo {
@Test
public void assertionFailure() {
SoftAssert softAssertion = new SoftAssert();
System.setProperty("webdriver.gecko.driver", "C:\\Users\\Vibha\\Software\\geckodriver-v0.31.0-win64\\geckodriver.exe");
WebDriver driver = new FirefoxDriver();
driver.manage().window().maximize();
driver.get("https://duckduckgo.com/");
String actualTitle = "DuckDuckGo — Privacy, simplified";
String expectedTitle = driver.getTitle();
String expectedText1 = driver.findElement(By.xpath("/html/body/div/div[2]/div/div[3]/div[1]")).getText();
/* Soft Assert */
System.out.println("Verify Title");
softAssertion.assertEquals(expectedTitle, actualTitle, "Incorrect page title");
System.out.println("Verify Text");
softAssertion.assertEquals("Privacy Protection For Any Device", expectedText1);
driver.quit();
}
@Test
public void print() {
System.out.println("Soft Assertion is displayed");
}
}
The output of the above program is
AssertAll
If there is any exception, and you want to throw it, then you need to use assertAll() method as a last statement in the @Test and test suite again to continue with the next @Test as it is.
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.Test;
import org.testng.asserts.SoftAssert;
public class AssertAllDemo {
@Test
public void assertionFailure() {
SoftAssert softAssertion = new SoftAssert();
System.setProperty("webdriver.gecko.driver", "C:\\Users\\SingVi04\\Vibha\\Software\\geckodriver-v0.31.0-win64\\geckodriver.exe");
WebDriver driver = new FirefoxDriver();
driver.manage().window().maximize();
driver.get("https://duckduckgo.com/");
String actualTitle = "DuckDuckGo — Privacy, simplified";
String expectedTitle = driver.getTitle();
String expectedText1 = driver.findElement(By.xpath("/html/body/div/div[2]/div/div[3]/div[1]")).getText();
/* Soft Assert */
System.out.println("Verify Title");
softAssertion.assertEquals(expectedTitle, actualTitle, "Incorrect page title");
System.out.println("Verify Text");
softAssertion.assertEquals("Privacy Protection For Any Device", expectedText1);
softAssertion.assertAll();
driver.quit();
}
@Test
public void print() {
System.out.println("Soft Assertion is displayed");
}
}
The output of the above program is
In the above program, we can see that both assertions of Test – assertionFailure are executed, but as the first assertion has failed, the test – assertionFailure is marked as failed.
Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!
The listener is defined as an interface that modifies the default TestNG’s behaviour. There are several interfaces that allow you to modify TestNG’s behaviour that are called “TestNG Listeners”. It allows customizing TestNG reports or logs. There are many types of TestNG listeners available. Here are a few listeners:
IAnnotationTransformer
IAnnotationTransformer2
IHookable
IInvokedMethodListener
IMethodInterceptor
IReporter
ISuiteListener
ITestListener
When you implement one of these interfaces, you can let TestNG know about it in either of the following ways:
Using in yourtestng.xml file.
Using the@Listeners annotation on any of your test classes.
ITestListenerhas the following methods
OnStart – Invoked before running all the test methods belonging to the classes inside the tag and calling all their Configuration methods.
onTestSuccess – onTestSuccess method is called on the success of any Test.
onTestFailure – onTestFailure method is called on the failure of any Test.
onTestSkipped – onTestSkippedmethod is called on skipped of any Test.
onTestFailedButWithinSuccessPercentage– method is called each time Test fails but is within success percentage. Invoked each time a method fails but has been annotated with successPercentage and this failure still keeps it within the success percentage requested.
onFinish – Invoked after all the test methods belonging to the classes inside the tag have run and all their Configuration methods have been called.
onTestStart – Invoked each time before a test will be invoked. The ITestResult is only partially filled with the references to class, method, start millis, and status.
Here, I explain the use of listener – ITestListener in a program mentioned below
Step 1) Create a class “ListenerDemo” that implements ‘ITestListener’. Add methods like onTestFailure, onTestSkipped, onTestStart, onTestSuccessto this class
Step 2) Create another class “ListenerTestCases” for the login process automation. Selenium will execute these ‘TestCases’ to login automatically.
Step 3) Next, implement this listener in our regular project class i.e. “ListenerTestCases “. There are two different ways to connect to the class and interface.
The first way is to use Listeners annotation (@Listeners) as shown below:
Step 4): Execute the “ListenerTestCases” class. Methods in class “TestPass ” are called automatically according to the behaviour of methods annotated as @Test.
Step 5): Verify the Output that logs display at the console.
import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;
public class ListenerDemo implements ITestListener {
// When Test case get failed, this method is called.
public void onTestFailure(ITestResult Result) {
System.out.println("The name of the testcase failed is :" + Result.getName());
}
// When Test case get Skipped, this method is called.
public void onTestSkipped(ITestResult Result) {
System.out.println("The name of the testcase Skipped is :" + Result.getName());
}
// When Test case get Started, this method is called.
public void onTestStart(ITestResult Result) {
System.out.println(Result.getName() + " test case started");
}
// When Test case get passed, this method is called.
public void onTestSuccess(ITestResult Result) {
System.out.println("The name of the testcase passed is :" + Result.getName());
}
}
In the below test, there are 2 test cases. One Test passes and another fails. When we are executing ListenerTestCases, it will call the ListenersDemo internally.
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
importorg.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;
@Listeners(com.selenium.testng.TestNGDemo.ListenerDemo.class)
public class ListenerTestCases {
static WebDriver driver;
@Test
public void TestPass() {
System.setProperty("webdriver.chrome.driver",
"C:\\Users\\Vibha\\Desktop\\SeleniumKT\\chromedriver_win32\\chromedriver.exe");
driver= new ChromeDriver();
driver.get("https://opensource-demo.orangehrmlive.com/");
driver.findElement(By.name("txtUsername")).sendKeys("Admin");
driver.findElement(By.name("txtPassword")).sendKeys("admin123");
driver.findElement(By.id("btnLogin")).submit();
String dashboardTitle = driver.findElement(By.id("welcome")).getText();
Assert.assertTrue(dashboardTitle.contains("Welcome"));
}
@Test
public void TestFail() {
System.setProperty("webdriver.chrome.driver",
"C:\\Users\\Vibha\\Desktop\\SeleniumKT\\chromedriver_win32\\chromedriver.exe");
driver= new ChromeDriver();
driver.get("https://opensource-demo.orangehrmlive.com/");
driver.findElement(By.name("txtUsername")).sendKeys("Admin");
driver.findElement(By.name("txtPassword")).sendKeys("admin123");
driver.findElement(By.id("btnLogin")).submit();
String dashboardTitle = driver.findElement(By.id("welcome")).getText();
Assert.assertTrue(dashboardTitle.contains("Hello"));
}
}
Output
TestFail test case started
The name of the testcase failed is :TestFail
TestPass test case started
The name of the testcase passed is :TestPass
PASSED: TestPass
FAILED: TestFail
java.lang.AssertionError: did not expect to find [true] but found [false]
The output of the above program is
To execute this program, we need to Right-click and select Run as – TestNG.
There is another way to execute the Listener class, which is using testng.xml. There is a Listener class named “ListenerDemo” where the implementation of various methods of the listener is present. If we want to run the tests using testng.xml, then there is no need of mentioning Listeners in the ListenerTestCases class.
<?xml version = "1.0"encoding = "UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name = "TestSuite">
<listeners>
<listener class-name ="com.selenium.testng.TestNGDemo.ListenerDemo"/>
</listeners>
<test name ="Test">
<classes>
<class name="com.selenium.testng.TestNGDemo.ListenerTestCases"/>
</classes>
</test>
</suite>
The test execution result will look like something shown below.
TestNG generates various types of reports under the test-output folder. Open “emailable-report.html”, as this is an HTML report open it with the browser. It will look like something below.
TestNG also produces an “index.html” report, and it resides under the test-output folder.
There is another example of Listener –ITestResult.
We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
QA Automation Expert 
