Execute Testng.xml using batch file

HOME

In previous tutorial we have seen executing testng.xml tests from command line. Now here we will look into executing testng.xml using batch file (.bat) file.

A batch file (.bat) is used in DOS and Windows, which is an unformatted text file that consists of a series of commands to be executed by the command line interpreter.

Pre-Requisite:

  1. Selenium
  2. TestNG
  3. Maven
  4. Java 11
  5. Maven Complier Plugin
  6. Maven Surefire Plugin
  7. Notepad

Let us first create some tests in a class.

import static org.testng.Assert.assertTrue;

import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.MatcherAssert.assertThat;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;

public class TestNGRunFromCommandLine {
	
	 WebDriver driver;
	 
	    @BeforeTest
	    public void setUp() {
	        System.setProperty("webdriver.gecko.driver",
	                "C:\\Users\\Vibha\\Software\\geckodriver\\geckodriver.exe");
	 
	        driver = new FirefoxDriver();
	        driver.get("https://opensource-demo.orangehrmlive.com/"); 
	        driver.manage().window().maximize();
	    }
	 
	    @Test(description = "This test validates title of login functionality", priority = 0)
	    public void verifyLoginPage() {
	 
	        String expectedTitle = driver.findElement(By.xpath("//*[@id='logInPanelHeading']")).getText(); 
	        System.out.println("Title :" + expectedTitle);
	        assertTrue(expectedTitle.equalsIgnoreCase("LOGIN Panel12"));
	    }
	 
	    @Test(description = "This test validates  successful login to Home page", priority = 1)
	    public void verifyHomePage() throws InterruptedException {
	 
	        System.out.println("Username Entered");
	        driver.findElement(By.name("txtUsername")).sendKeys("Admin");
	 
	        System.out.println("Password Entered");
	        driver.findElement(By.name("txtPassword")).sendKeys("admin123");         
	        driver.findElement(By.id("btnLogin")).submit();
	        
	        Thread.sleep(2000);
	 
	        String newPageText = driver.findElement(By.xpath("//*[@id='content']/div/div[1]/h1")).getText();
	        System.out.println("newPageText :" + newPageText);
	        assertThat(newPageText, containsString("Dashboard"));
	    }
	 
	    @AfterTest
	    public void teardown() {
	 
	        driver.quit();
	    }
	 
	}

The below is the testng.xml file which has a class that we have created above and we will be invoking this xml file using batch file (.bat).

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test name="Test">
    <classes>
      <class name="TestNG_Demo.TestNGRunFromCommandLine"/>
    </classes>
  </test> 
</suite> 

 How to create a batch file?

Step 1: Open notepad
Step 2: Paste the below lines of code – You may need to add your project location. In the example, project location is set as C:\Users\Vibha\Projects\Vibha_Personal\ParallelTestsTestNG
Step 3: Save the file as ‘TestNGProject.bat’ in location that you want to save.

cd C:\Users\Vibha\Projects\Vibha_Personal\ParallelTestsTestNG
mvn compile test

Now, to run the tests, double click on the TestNGProject.bat file and all the commands mentioned in the file will be executed one by one.

As we know, TestNG generates lot of Reports automatically. We are going to look into 2 reports – emailable-report.html and index.html. The reports are generated under surefire-reports folder within target directory.

Emailable-Report.html

Index.html

Hope this article helps you to invoke your tests using .bat file. 

Run TestNG tests from Command Line

HOME

The previous tutorial has explained the Integration of Selenium with TestNG and the tests are executed through either TestNG Suite or testng.xml. This tutorial explains the steps to run the TestNG Tests through the command line.

Pre-Requisite

  1. Selenium
  2. TestNG
  3. Maven
  4. Java 11
  5. Maven Compiler Plugin
  6. Maven Surefire Plugin

Imagine we need to run the TestNG Tests in CI/CD pipelines like Jenkins or GitLab, then we can’t right click and select TestNG Suite or tesng.xml to run the tests. In such situations, the tests can be executed through command line.

We need to add plugins to pom.xml to compile the test code and then run the tests. To know more about Maven Surefire Plugin for TestNG, refer this blog.

 <build>
      <plugins>

         <!--  Compiler Plugin -->
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.8.1</version>
            <configuration>
               <source>11</source>
               <target>11</target>
            </configuration>
         </plugin>
         
         <!--  Plugin used to execute tests -->
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            <configuration>
               <suiteXmlFiles>
                  <suiteXmlFile>testng.xml</suiteXmlFile>
               </suiteXmlFiles>
            </configuration>
         </plugin>
      </plugins>
   </build>
</project>

It is needed to add testng.xml to suiteXmlFile to start the execution of the tests.

<suiteXmlFiles>
       <suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>

Create sample class which has @Test methods. In the example below we have created a class as below:

import static org.testng.Assert.assertTrue;

import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.MatcherAssert.assertThat;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;

public class TestNGRunFromCommandLine {
	
	 WebDriver driver;
	 
	    @BeforeTest
	    public void setUp() {
	        System.setProperty("webdriver.gecko.driver",
	                "C:\\Users\\Vibha\\Software\\geckodriver\\geckodriver.exe");
	 
	        driver = new FirefoxDriver();
	        driver.get("https://opensource-demo.orangehrmlive.com/");
	 
	        driver.manage().window().maximize();
	    }
	 
	    @Test(description = "This test validates title of login functionality", priority = 0)
	    public void verifyLoginPage() {
	 
	        String expectedTitle = driver.findElement(By.xpath("//*[@id='logInPanelHeading']")).getText(); 
	        System.out.println("Title :" + expectedTitle);
	        assertTrue(expectedTitle.equalsIgnoreCase("LOGIN Panel"));
	    }
	 
	    @Test(description = "This test validates  successful login to Home page", priority = 1)
	    public void verifyHomePage() {
	 
	        System.out.println("Username Entered");
	        driver.findElement(By.name("txtUsername")).sendKeys("Admin");
	 
	        System.out.println("Password Entered");
	        driver.findElement(By.name("txtPassword")).sendKeys("admin123");
	         
	        driver.findElement(By.id("btnLogin")).submit();        
 
	        String newPageText = driver.findElement(By.xpath("//*[@id='content']/div/div[1]/h1")).getText();
	        System.out.println("newPageText :" + newPageText);
	        assertThat(newPageText, containsString("Dashboard"));
	    }
	 
	    @AfterTest
	    public void teardown() {
	 
	        driver.quit();
	    }
	}

The below is the testng.xml file, which will execute all the tests that are available under ‘TestNGRunFromCommandLine‘ class.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test name="Test">
    <classes>
      <class name="TestNG_Demo.TestNGRunFromCommandLine"/>
    </classes>
  </test>
</suite>

Below commands are used to execute ‘testng.xml’ file from command line. First, we need to go the place where pom.xml of the project is placed. Then use the mvn compile test to compile the code and execute the TestNG tests.

cd C:\Users\Vibha\Projects\Vibha_Personal\ParallelTestsTestNG
mvn compile test

After executing above command, it should execute the tests that we have specified in testng.xml file. Below is the screen shot after the execution of the tests.

This execution generate various TestNG Reports. We are concerned about emailable-report.html and index.html.

Emailable-Report.html

Emailable report is a type of summary report that one can transfer to other people in the team through any medium. Click on option “emailable-report.html”. Click on option web browser. The output reports in TestNG reporting will looks like below:

Index.html

Index report contains the index-like structure of different parts of the report, such as failed tests, test file, passed test, etc.

Right click on the index.html from the project directory. Select option open with web browser option.

The result will look like this:

Congratulations. This tutorial has explain running the tests of TestNG using Command Line. Happy Learning!!

Parallel testing of DataProviders in TestNG

HOME

The previous tutorial has explained the DataProviders in TestNG. The DataProvider in TestNG is a way to pass the parameters in the test functions. Using DataProvider in TestNG, we can easily inject multiple values into the same test case. It comes inbuilt in TestNG and is popularly used in data-driven frameworks.

 It is an option for the parallel execution of tests in TestNG. 

It is advisable to create 2 classes – one class contains the Test cases and another class define TestNG parameters – DataProviders.

Let us create a class for the DataProvider method with all the Test Data as shown below:

import org.testng.annotations.DataProvider;

public class DataProviderDemo {	
	
	 @DataProvider(name = "testData", parallel=true)
	 public Object[][] dataProvFunc() {
	       return new Object[][] {           
	    	   {"","","Username cannot be empty"},    	  
	    	   {"","Test","Username cannot be empty"},
	    	   {"$%1234","2345%$","Invalid credentials"}          
	    	 };
	    }
	}

 An extra parameter “parallel” is required to initiate parallel execution in TestNG using dataprovider.

Below is the test which uses the parameter from dataprovider and run the tests parallelly.

A new ThreadLocal is instantiated for each test class since it’s in the BeforeClass annotation.

private static final ThreadLocal<WebDriver> WEB_DRIVER_THREAD_LOCAL = new ThreadLocal<WebDriver>();

Below is the complete test code:

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;

public class DataProviderParallelTests {
	
	public WebDriver driver;
	private static final ThreadLocal<WebDriver> WEBDRIVER_THREADLOCAL = new ThreadLocal<WebDriver>();
	
	 @BeforeMethod
    public void setUp(){

        System.setProperty("webdriver.chrome.driver",
                "C:\\Users\\Vibha\\Software\\chromedriver\\chromedriver.exe");
        driver = new ChromeDriver();
        WEBDRIVER_THREADLOCAL.set(driver);
        System.out.println("Before method Thread Id:" + Thread.currentThread().getId());
        
    }
	
	@Test(dataProvider = "testData", dataProviderClass = DataProviderDemo.class)
    public void invalidLoginTest(String username, String password, String errorMessage) throws InterruptedException {
		     
	    driver = WEBDRIVER_THREADLOCAL.get();
	    driver.manage().window().maximize();
        driver.get("https://opensource-demo.orangehrmlive.com/");
     
        Thread.sleep(2000);
        driver.findElement(By.name("txtUsername")).sendKeys(username);
        System.out.println("Username :" + username);
        
        Thread.sleep(2000);
        driver.findElement(By.name("txtPassword")).sendKeys(password);
        System.out.println("password :" + password);
        
        Thread.sleep(2000);
        String expectedError = driver.findElement(By.id("spanMessage")).getText();
        System.out.println("Error Message :" + expectedError);
        Assert.assertTrue(expectedError.contains(errorMessage));

    }
		 
	@AfterMethod
	public void tear_down() {
		 
		 WebDriver driver = WEBDRIVER_THREADLOCAL.get();
		 System.out.println("After method Thread Id:" + Thread.currentThread().getId());
	        if (driver != null) {
	            driver.quit();
	     }
    }	
}

testng.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite" thread-count="2" data-provider-thread-count="2">
  <test name="Test">
    <classes>
      <class name="DataProvider.DataProviderParallelTests"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

In this file, data-provider-thread-count is set to 2, then two browsers will be opened and the first two tests will run from the list. 

Run the test script from testng.xml, Right-Click on the XML, and select Run As -> TestNG Suite.

The execution status shown below shows that 2 threads are active at a time, which execute 2 set of dataprovider parameters – Thread 14 and Thread 15. Once the tests are finished for Thread 14 and Thread 15, they are closed and a new Thread 15 again initiated to start test execution of 3rd parameter.

TestNG generates multiple test reports under folder test-output. We are mainly concerned about 2 reports – emailable-report.html and index.html.

Emailable-report.html

Emailable reports are a type of summary reports that one can transfer to other people in the team through any medium. 

Index.html

Index report contains the index-like structure of different parts of the report, such as failed tests, test files, passed tests, etc. We can divide this report into two parts. The left part contains the index, and this is the reason it is called an index report, while the right part contains the explored content of that index.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

DataProvider in TestNG using Excel

HOME

In the previous tutorial, I have explained the DataProvider in TestNG without using Excel. In this tutorial, I will explain the use of Excel in DataProvider for TestNG.

I have created an excel – SearchInBing.xlsx and placed on Desktop. You can create a TestData folder in your project and place the excel file within it. So, my datasheet looks like below-

Next, we will create a DataProvider method that will use another method to read the excel file & create a 2D object from the row & column values of the excel and return the same value, so that our test script can use it. The code for it would look like below-

@DataProvider(name = "excelData")
	public Object[][] excelDataProvider() throws IOException {
		// We are creating an object from the excel sheet data by calling a method that
		// reads data from the excel stored locally in our system
		Object[][] arrObj = getExcelData(
				"C:\\Users\\Vibha\\Desktop\\SearchInBing.xlsx",
				"Details");
		return arrObj;
	}

	// This method handles the excel - opens it and reads the data from the
	// respective cells using a for-loop & returns it in the form of a string array
	public String[][] getExcelData(String fileName, String sheetName) throws IOException {
		String[][] data = null;
		try {
			
			FileInputStream fis = new FileInputStream(fileName);
			XSSFWorkbook workbook = new XSSFWorkbook(fis);
			XSSFSheet sheet = workbook.getSheet(sheetName);
			XSSFRow row = sheet.getRow(0);
			int noOfRows = sheet.getPhysicalNumberOfRows();
			int noOfCols = row.getLastCellNum();
			Cell cell;
			data = new String[noOfRows - 1][noOfCols];

			for (int i = 1; i < noOfRows; i++) {
				for (int j = 0; j < noOfCols; j++) {
					row = sheet.getRow(i);
					cell = row.getCell(j);
					data[i - 1][j] = cell.getStringCellValue();
				}
			}
		} catch (Exception e) {
			System.out.println("The exception is: " + e.getMessage());
		}
		return data;
	}

Now, create a class which contains the test code. By default, the data provider will be looked for in the current test class or one of its base classes. If you want to put your data provider in a different class, it needs to be a static method or a class with a non-arg constructor, and you specify the class where it can be found in the dataProviderClass attribute.

public class DataProviderWithExcelDemo {

	WebDriver driver;

	@BeforeMethod
	public void setUp() {
		System.out.println("Start test");
		System.setProperty("webdriver.chrome.driver",
				"C:\\Users\\Vibha\\Software\\chromedriver_win32_93.0.4577.15\\chromedriver.exe");
		driver = new ChromeDriver();
		driver.get("https://www.bing.com");
		driver.manage().window().maximize();

	}

	@Test(dataProvider = "excelData", dataProviderClass = ExcelDataProvider.class)
	public void search(String keyWord1, String keyWord2) {

		WebElement txtBox = driver.findElement(By.id("sb_form_q"));
		txtBox.sendKeys(keyWord1, " ", keyWord2);
		System.out.println("Keyword entered is : " + keyWord1 + " " + keyWord2);
		txtBox.sendKeys(Keys.ENTER);
		System.out.println("Search results are displayed.");
	}

	@AfterMethod
	public void burnDown() {
		driver.quit();
	}

}

To run the code, right click and Select – TestNG Test.

The Execution status will look like as shown below

This test execution generate reports under test-output folder.

We are concerned about two reports – index.html and emailable-report.html.

Below is the image of emailable-report.html.

Below is the image of index.html.

See how easy it is to read data from excel and use in the Test Code using DataProvider.

I hope you have enjoyed this tutorial. Happy Learning!!

DataProviders in TestNG

HOME

In the last tutorial, I have explain the Parameters in TestNG which passes different test data to the test case as arguments. Similar to TestNG Parameters, DataProviders are a means to pass data to test scripts in TestNG. In this tutorial, I will explain about the DataProviders in TestNG.

What is DataProvider in TestNG?

The DataProvider in TestNG is another way to pass the parameters in the test function, the other one being TestNG parameters. Using DataProvider in TestNG, we can easily inject multiple values into the same test case. It comes inbuilt in TestNG and is popularly used in data-driven frameworks.

Syntax of DataProvider

@DataProvider (name = "name_of_dataprovider")
public Object[][] dpMethod() {
    return new Object [][] { values}
}
  •  A Data Provider is a method on the class that returns an array of array of objects.  This method is annotated with @DataProvider
  • A @Test method specifies its Data Provider with the dataProvider attribute. This name must correspond to a method on the same class annotated with @DataProvider(name=”…”) with a matching name.
  • TestNG dataprovider returns a 2d list of objects..An array of array of objects (Object[][]) where the first dimension’s size is the number of times the test method will be invoked and the second dimension size contains an array of objects that must be compatible with the parameter types of the test method.
  • DataProviders are not declared on top of the functions like TestNG parameters but have a method of their own, which in regular speaking terms called a dataprovider method. For example, dpMethod here.
  • The dataprovider name calls the dataprovider method, and if there is no name specified by the tester, then the dataprovider method is the default name used in the receiving @Test case.
  • Data providers can run in parallel with the attribute parallel.

Below is the basic example of using DataProvider in TestNG.

import org.openqa.selenium.By;
import org.openqa.selenium.Keys;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class DataProviderDemo {

	WebDriver driver;

	@DataProvider(name = "testData")
	public Object[][] dataProvFunc() {
		return new Object[][] { { "Selenium" }, { "TestNG" } };
	}

	@BeforeMethod
	public void setUp() {

		System.out.println("Start the test");
		System.setProperty("webdriver.chrome.driver",
				"C:\\Users\\Vibha\\Software\\chromedriver\\chromedriver.exe");
		driver = new ChromeDriver();
		driver.get("https://www.bing.com/");
		driver.manage().window().maximize();

	}

	// Passing the dataProvider to the test method through @Test annotation
	@Test(dataProvider = "testData")
	public void search(String keyWord) {
		WebElement txtBox = driver.findElement(By.id("sb_form_q"));
		txtBox.sendKeys(keyWord);
		System.out.println("Keyword entered is : " + keyWord);
		txtBox.sendKeys(Keys.ENTER);
		System.out.println("Search result is displayed.");
	}

	@AfterMethod
	public void burnDown() {
		driver.quit();

		System.out.println("End the test");
	}

}

In the above example, I am passing two search keywords, viz “Selenium” and “TestNG” to the test method using the DataProvider method. You can run the code and check the output. It will be as shown below-

Here, Test is executed with two values, but we have run the test only once.

Inheriting DataProvider in TestNG

It is messy to have supporting methods like DataProvider and test code in one class. It is always preferred to declare the test case in one class and define TestNG parameters like DataProviders in another class. By default, the data provider will be looked for in the current test class or one of its base classes. If you want to put your data provider in a different class, it needs to be a static method or a class with a non-arg constructor, and you specify the class where it can be found in the dataProviderClass attribute.

Let us create separate classes for the DataProvider method and the test method, as shown below:

DataProvider Class

public class DPDemo {

	@DataProvider(name = "testData")
	public Object[][] dataProvFunc() {
		return new Object[][] { 
          { "Selenium" }, { "TestNG" }, { "Automation" } };
	}
}

We can see that all we did was create a DataProvider method in a Class and create a new class for Test Code.

public class DataProviderInheritanceDemo {

	WebDriver driver;

	@BeforeMethod
	public void setUp() {

		System.out.println("Start the test");
		System.setProperty("webdriver.chrome.driver",
				"C:\\Users\\Vibha\\Software\\chromedriver\\chromedriver.exe");
		driver = new ChromeDriver();
		driver.get("https://www.bing.com/");
		driver.manage().window().maximize();

	}

	// Passing the dataProvider to the test method through @Test annotation
	@Test(dataProvider = "testData", dataProviderClass = DPDemo.class)
	public void search(String keyWord) {
		WebElement txtBox = driver.findElement(By.id("sb_form_q"));
		txtBox.sendKeys(keyWord);
		System.out.println("Keyword entered is : " + keyWord);
		txtBox.sendKeys(Keys.ENTER);
		System.out.println("Search result is displayed.");
	}

	@AfterMethod
	public void burnDown() {
		driver.quit();

		System.out.println("End the test");
	}

}

As you can see, to handle the inheritance, all we did was add an attribute to the test method (highlighted above), which specifies the class that has the DataProvider method. 

Passing Multiple Parameter Values in TestNG DataProviders

Passing multiple values is pretty similar to passing numerous parameters. The only difference is that we will pass various values to a single parameter so that a string of input(s) is sent in one go.

Let us quickly understand this concept with the help of the code as shown below.

DataProvider Class

public class DPDemo {

	@DataProvider(name = "testData")
	public Object[][] dataProvFunc() {
		return new Object[][] { { "Automation Tester", "2-5 years" }, { "Performance Tester", "3+ years" },
				{ "DevOps", "5+ years" } };
	}
}

Test Code – DataProviderInheritanceDemo

public class DataProviderInheritanceDemo {

	WebDriver driver;

	@BeforeMethod
	public void setUp() {

		System.out.println("Start the test");
		System.setProperty("webdriver.chrome.driver",
				"C:\\Users\\Vibha\\Software\\chromedriver\\chromedriver.exe");
		driver = new ChromeDriver();
		driver.get("https://www.bing.com/");
		driver.manage().window().maximize();

	}

	// Passing the dataProvider to the test method through @Test annotation
	@Test(dataProvider = "testData", dataProviderClass = DPDemo.class)
	public void search(String keyWord1, String keyWord2) {

		WebElement txtBox = driver.findElement(By.id("sb_form_q"));
		txtBox.sendKeys(keyWord1, keyWord2);
		System.out.println("Keyword entered is : " + keyWord1 + " " + keyWord2);
		txtBox.sendKeys(Keys.ENTER);
		System.out.println("Search result is displayed.");
	}

	@AfterMethod
	public void burnDown() {
		driver.quit();

		System.out.println("End the test");
	}
}

Run the test script, and you will see both the values for the TestNG parameters being passed in one go, the output for it would be as follows-

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

How to create and run TestNG.xml of a TestNG class

HOME

There are 2 ways to run TestNG tests – using Run As TestNG tests and from testng.xml.

In this tutorial, I’m explaining how to create and run tests using TestNG.xml.

Step 1 – In the below project, there is no testng.xml present.

Step 2 – Right click on class name “API_Test.java” and navigate “TestNG–>Convert to TestNG“.

Step 3 – Now a new window for testng.xml will be generated as shown below. Click the Finish Button.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test thread-count="5" name="Test">
    <classes>
      <class name="com.example.RestAssured_TestNG_Demo.API_Test"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

Step 4 – A xml file named “testng.xml” will be generated in project hierarchy.

Step 5 – Right click on this testng.xml and select Run As–> Testng Suite

Step 6 – You will view the result in two different tabs: Console and “Results of running suite”. Below is the image of Rest of running suite tab.

Multiple Classes

Let us imagine a sceanrio where there are 3 classes and we want to run them all together, you can done that by creating a testng.xm and mention all 3 classes in that testng.xml.

Select all 3 classes and right click and navigate “TestNG–>Convert to TestNG

This is how the testng.xml will look like for multiple classes.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test thread-count="5" name="Test">
    <classes>
      <class name="com.example.RestAssured_TestNG_Demo.API_Test1"/>
      <class name="com.example.RestAssured_TestNG_Demo.Test3"/>
      <class name="com.example.RestAssured_TestNG_Demo.Test2"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

Right click on this testng.xml and select Run As–> Testng Suite.  You will get the result in two tabs: Console and “Results of running suite”. Rhis shows that all the tests present withing the three classes are executed.

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

Hard Assert and Soft Assert in TestNG

 HOME

In this tutorial, will discuss about Hard Assert and Soft Assert in TestNG. Before starting with Hard and Soft Assert, go through What is Assert in TestNG.

What is Hard Assert?

Hard Assertion throws AssertionError immediately when an Assert Condition fails and moves to next @Test method

Suppose, there are 2 assertions in a Test and the first assertion fails, then HardAssertion do not execute the second Assertion Condition and declare the test as failed

As you can see in the below example, there are 2 assert conditions under Test – AssertionFailure(). As first Assert Condition fails, it moved directly to second test without executing another Assert Condition

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.Assert;
import org.testng.annotations.Test;
 
public class HardAssertionDemo {

     @Test
     public void AssertionFailure() {
           System.setProperty("webdriver.gecko.driver", "src\\test\\resources\\webdrivers\\window\\geckodriver.exe");
           WebDriver driver = new FirefoxDriver();
 
           driver.get("https://duckduckgo.com/");
           String actualTitle = "DuckDuckGo — Privacy, simplified";
 
           String expectedTitle = driver.getTitle();
           String expectedText = driver.findElement(By.xpath("/html/body/div/div[2]/div/div[3]/div[1]")).getText();
 
           /* Hard Assert */
           Assert.assertEquals(expectedTitle, actualTitle, "Incorrect page title");
           Assert.assertEquals("Privacy Protection For Any Device", expectedText);
     }
 
     @Test
     public void print() {
           System.out.println("Hard Assertion is displayed");
     }
}

What is Soft Assert?

To overcome above mentioned problem, there is another type of assertion called Soft Assert

Soft Assert does not throw an exception when an Assert Condition fails and continue with the next step after the Assert Condition.

Soft assert does not include by default in TestNG. For this, you need to include the package org.testng.asserts.SoftAssert;

SoftAssert softAssertion = new SoftAssert();

 Create an object of SoftAssertion to run Assert Conditions

Below is an example of Soft Assert

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.Test;
import org.testng.asserts.SoftAssert;
 
public class SoftAssertionDemo {

     @Test
     public void AssertionFailure() {
           SoftAssert softAssertion = new SoftAssert();
 
           System.setProperty("webdriver.gecko.driver", "src\\test\\resources\\webdrivers\\window\\geckodriver.exe");
           WebDriver driver = new FirefoxDriver();
 
           driver.get("https://duckduckgo.com/");
 
           String actualTitle = "DuckDuckGo — Privacy, simplified";
 
           String expectedTitle = driver.getTitle();
           String expectedText1 = driver.findElement(By.xpath("/html/body/div/div[2]/div/div[3]/div[1]")).getText();
 
           /* Soft Assert */
           softAssertion.assertEquals(expectedTitle, actualTitle, "Incorrect page title");
           softAssertion.assertEquals("Privacy Protection For Any Device", expectedText1);
     }
 
     @Test
     public void print() {
           System.out.println("Soft Assertion is displayed");
     } 
 
}

AssertAll

If there is any exception and you want to throw it then you need to use assertAll() method as a last statement in the @Test and test suite again continue with next @Test as it is. 

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.Test;
import org.testng.asserts.SoftAssert;
 
public class SoftAssertionDemo {
 
     @Test
     public void AssertionFailure() {
           SoftAssert softAssertion = new SoftAssert();
 
           System.setProperty("webdriver.gecko.driver", "src\\test\\resources\\webdrivers\\window\\geckodriver.exe");
           WebDriver driver = new FirefoxDriver();
           driver.get("https://duckduckgo.com/");
 
           String actualTitle = "DuckDuckGo — Privacy, simplified";
           String expectedTitle = driver.getTitle();
           String expectedText1 = driver.findElement(By.xpath("/html/body/div/div[2]/div/div[3]/div[1]")).getText();
 
           /* Soft Assert */
           softAssertion.assertEquals(expectedTitle, actualTitle, "Incorrect page title");
           softAssertion.assertEquals("Privacy Protection For Any Device", expectedText1);
 
           softAssertion.assertAll();
     }
 
     @Test
     public void print() {
           System.out.println("Soft Assertion is displayed");
     } 
}

TestNG Listeners in Selenium

 HOME

Listener is defined as interface that modifies the default TestNG’s behavior. There are several interfaces that allow you to modify TestNG’s behavior that are called “TestNG Listeners”.  It allows customizing TestNG reports or logs. There are many types of TestNG  listeners available. Here are a few listeners:

  • IAnnotationTransformer 
  • IAnnotationTransformer2 
  • IHookable 
  • IInvokedMethodListener 
  • IMethodInterceptor 
  • IReporter 
  • ISuiteListener 
  • ITestListener 

When you implement one of these interfaces, you can let TestNG know about it with either of the following ways:

  • Using in your testng.xml file.
  • Using the @Listeners annotation on any of your test classes.

ITestListener has the following methods

  • OnStart  Invoked before running all the test methods belonging to the classes inside the tag and calling all their Configuration methods.
  • onTestSuccess onTestSuccess method is called on the success of any Test.
  • onTestFailure onTestFailure method is called on the failure of any Test.
  • onTestSkipped– onTestSkipped method is called on skipped of any Test.
  • onTestFailedButWithinSuccessPercentage– method is called each time Test fails but is within success percentage. Invoked each time a method fails but has been annotated with successPercentage and this failure still keeps it within the success percentage requested.
  • onFinish– Invoked after all the test methods belonging to the classes inside the tag have run and all their Configuration methods have been called.
  • onTestStart Invoked each time before a test will be invoked. The ITestResult is only partially filled with the references to class, method, start millis and status.

Here, I explain the use of listener – ITestListener  in a program mentioned below

Step 1) Create class “ListenerDemo” that implements ‘ITestListener’. Add methods like onTestFailure, onTestSkipped, onTestStart, onTestSuccess to this class

Step 2) Create another class “ListenerTestCases” for the login process automation. Selenium will execute this ‘TestCases’ to login automatically.

Step 3) Next, implement this listener in our regular project class i.e. ” ListenerTestCases “. There are two different ways to connect to the class and interface.

The first way is to use Listeners annotation (@Listeners) as shown below:

@Listeners(com.selenium.testng.TestNGDemo.ListenerDemo.class)

Step 4): Execute the “ListenerTestCases” class. Methods in class “TestPass ” are called automatically according to the behavior of methods annotated as @Test.

Step 5): Verify the Output that logs displays at the console.

import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;
 
public class ListenerDemo implements ITestListener {
 
            // When Test case get failed, this method is called.
            public void onTestFailure(ITestResult Result) {
                        System.out.println("The name of the testcase failed is :" + Result.getName());
            }
 
            // When Test case get Skipped, this method is called.
            public void onTestSkipped(ITestResult Result) {
                        System.out.println("The name of the testcase Skipped is :" + Result.getName());
            }
 
            // When Test case get Started, this method is called.
            public void onTestStart(ITestResult Result) {
                        System.out.println(Result.getName() + " test case started");
            }
 
            // When Test case get passed, this method is called.
            public void onTestSuccess(ITestResult Result) {
                        System.out.println("The name of the testcase passed is :" + Result.getName());
            } 
}

In the below test, there are 2 test cases. One Test passes and another fails. When we are executing ListenerTestCases, it will call the ListenersDemo internally.

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
importorg.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;
 
@Listeners(com.selenium.testng.TestNGDemo.ListenerDemo.class)
public class ListenerTestCases {
 
            static WebDriver driver;
 
            @Test
            public void TestPass() {
                        System.setProperty("webdriver.chrome.driver",
                                    "C:\\Users\\Vibha\\Desktop\\SeleniumKT\\chromedriver_win32\\chromedriver.exe");
                        driver= new ChromeDriver();
                        driver.get("https://opensource-demo.orangehrmlive.com/");
                        driver.findElement(By.name("txtUsername")).sendKeys("Admin");
                        driver.findElement(By.name("txtPassword")).sendKeys("admin123");
                        driver.findElement(By.id("btnLogin")).submit();
                        String dashboardTitle = driver.findElement(By.id("welcome")).getText();
                        Assert.assertTrue(dashboardTitle.contains("Welcome"));
            }
 
            @Test
            public void TestFail() {
                        System.setProperty("webdriver.chrome.driver",
                                    "C:\\Users\\SingVi04\\Desktop\\SeleniumKT\\chromedriver_win32\\chromedriver.exe");
                        driver= new ChromeDriver();
                        driver.get("https://opensource-demo.orangehrmlive.com/");
                        driver.findElement(By.name("txtUsername")).sendKeys("Admin");
                        driver.findElement(By.name("txtPassword")).sendKeys("admin123");
                        driver.findElement(By.id("btnLogin")).submit();
                        String dashboardTitle = driver.findElement(By.id("welcome")).getText();
                        Assert.assertTrue(dashboardTitle.contains("Hello"));
            }
}

Output
TestFail test case started
The name of the testcase failed is :TestFail
TestPass test case started
The name of the testcase passed is :TestPass
PASSED: TestPass
FAILED: TestFail
java.lang.AssertionError: did not expect to find [true] but found [false]

To execute this program, we need to Right-click and select Run as – TestNG.

There is another way to execute the Listener class, which is using testng.xml. There is a Listener class named “ListenerDemo” where the implementation of various methods of the listener is present. If we want to run the tests using testng.xml, then there is no need of mentioning Listeners in the ListenerTestCases class.

<?xml version = "1.0"encoding = "UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name = "TestSuite">
<listeners>
<listener class-name ="com.selenium.testng.TestNGDemo.ListenerDemo"/>
</listeners>
 
<test name ="Test">
<classes>
<class name="com.selenium.testng.TestNGDemo.ListenerTestCases"/>
</classes>
</test>
</suite>

The test execution result will look like something shown below.

TestNG generates the various types of reports under the test-output folder. Open “emailable-report.html”, as this is an HTML report open it with the browser. It will look like something below.

TestNG also produce “index.html” report, and it resides under the test-output folder

There is another example of Listener –ITestResult.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

How to pass Parameters in TestNG

 HOME

One of the important features of TestNG is ability to pass different test data to a test case as arguments that is called parametrization

There are mainly two ways through which we can provide parameter values to TestNG tests.

  1. Through testng.xml XML configuration file
  2.   Through DataProviders

In this tutorial, we will discuss about using testng.xml for parameterization. If we need to pass some simple values such as String or Integer types to the test methods at runtime, there is a something called @Parameter where parameter values through TestNG XML configuration files pass to test.

@Parameters("value")

Lets explain how we can use parameters

Step 1 – Create a java test class, say, TestNGParameterizationDemo.java

Step 2 – Add test method parameterizedTest() to the test class. This method takes a string as input parameter

Add the annotation @Parameters(“browser”) to this method. The parameter pass a value from testng.xml

Step 3 – Create a TestNG.xml and pass the value of parameter in  this configuration file.

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.Parameters;
import org.testng.annotations.Test;
 
public class TestNGParameterizationDemo {
            WebDriver driver;

            @Test
            @Parameters("browser")
            public void parameterizedTest(String browser) {
                        if (browser.equalsIgnoreCase("firefox")) {
 
                                    // set path to geckodriver.exe and create gecko instance
                                    System.setProperty("webdriver.gecko.driver",
                                                            "C:\\Users\\Vibha\\Desktop\\SeleniumKT\\geckodriver-v0.27.0-win64\\geckodriver.exe");
                                    driver = new FirefoxDriver();
                                    System.out.println("Browser Started:" + browser);
                        }
 
                        // Check if parameter passed as 'chrome'
                        else if (browser.equalsIgnoreCase("chrome")) {
 
                                    // set path to chromedriver.exe and create chrome instance
                                    System.setProperty("webdriver.chrome.driver",
                                                "C:\\Users\\Vibha\\Desktop\\SeleniumKT\\chromedriver_win32\\chromedriver.exe");
                                    driver = new ChromeDriver();
                                    System.out.println("Browser Started:" + browser);
                        }
            }
}

Output
Browser Started:Chrome

TestNG.xml looks like as shown below. Here, parameter name is browser name value for browser is “Chrome”. So, this “Chrome” value is pass to Test as parameter and as a result a Google Chrome browser opens.

<?xml version = "1.0"encoding = "UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name = "TestSuite">
  <test name="ChromeTest">
<parameter name="browser"value="Chrome" />
<classes>
<class name="com.selenium.testng.TestNGDemo.TestNGParameterizationDemo">
</class>
</classes>
</test>
</suite>

Cross Browser Testing using Selenium

 HOME

What is Cross Browser Testing?

Cross Browser is a technique in which a web application tests on different browsers and operating systems.  Cross Browser testing, make sure that the site rendered the same in every browser.

Suppose, we have to execute 25 tests cases to test a web application on Google Chrome Browser and it takes around 4 hrs to execute these tests. However, we do not know if a user is going to access the web application on which browser. Therefore, now the same set of test cases need to executes on Firefox, Edge, Internet Explorer, Safari and Opera.

Therefore, now we need to execute 25*6=150 test cases and test execution hour changes from 4 to 24 hrs to check that the web application is working as expected. What is the best approach to handle this situation is to automate these tests and perform cross browser testing

Why do we need to perform cross browser testing?

Each website is built by anyone or combination of these technologies – HTML, CSS and Javascript. Each browser uses different rendering engines to compute these technologies. Chrome uses Blink, WebKit on iOS, V8 JavaScript engine, Firefox uses Gecko, Edge usesChromium-based with Blink and V8 engines, Safari uses Webkit rendering engine, IE uses Trident and so on.

1) Font size, image orientation and alignment mismatch in different browsers

2) Different browser is compatible with different operating systems

3) CSS,HTML validation difference can be there

Lets see an example of Cross Browser testing using TestNG

Step 1 – We have used Selenium Webdriver with TestNG to automate the test cases to run on browsers – Chrome, Firefox and Edge

Step 2 – The test are running on all 3 browsers in parallel as we have used  parallel=“tests”

Step 3 – thread-count=”3″ means that 3 threads will start and each browser will use a thread

Step 4 –   @Parameters(“browser”) – parameter will be passed from testng.xml

import java.util.concurrent.TimeUnit;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.edge.EdgeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Parameters;
import org.testng.annotations.Test;
import junit.framework.Assert;
 
public class TestNGCrossBrowserDemo {
      WebDriver driver;
 
       @BeforeClass
       @Parameters("browser")
       public void setup(String browser) throws Exception {
 
          // Check if parameter passed from TestNG is 'firefox'
          if (browser.equalsIgnoreCase("firefox")) {
 
                // set path to geckodriver.exe and create gecko instance
                 System.setProperty("webdriver.gecko.driver",
                                                            "C:\\Users\\Vibha\\Desktop\\SeleniumKT\\geckodriver-v0.27.0-win64\\geckodriver.exe");
                 driver = new FirefoxDriver();
                 System.out.println("Browser Started:" + browser);
          }
 
          // Check if parameter passed as 'chrome'
           else if (browser.equalsIgnoreCase("chrome")) {
 
                // set path to chromedriver.exe and create chromeinstance
                  System.setProperty("webdriver.chrome.driver",
                                                "C:\\Users\\Vibha\\Desktop\\SeleniumKT\\chromedriver_win32\\chromedriver.exe");
                   driver = new ChromeDriver();
                   System.out.println("Browser Started:" + browser);
          }
 
          // Check if parameter passed as 'edge'
           else if (browser.equalsIgnoreCase("Edge")) {
 
               // set path to chromedriver.exe and create edge instance
                System.setProperty("webdriver.edge.driver",
                                                      "C:\\Users\\Vibha\\Desktop\\SeleniumKT\\edgedriver_win64\\msedgedriver.exe");
               driver = new EdgeDriver();
                System.out.println("Browser Started:" + browser);
           }
 
           else {
                  // If no browser passed throw exception
                    throw new Exception("Browser is not correct");
           }
        driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
        driver.manage().window().maximize();
       }
 
        @Test
        public void testTitle() throws InterruptedException {
                        driver.get("https://configureselenium.blogspot.com/2019/10/testng-what-is-testng.html");
              String actualTitle = driver.getTitle();
              System.out.println("Title- " + actualTitle);
 
              System.out.println("Test Case One with Thread Id:- " + Thread.currentThread().getId());
               Assert.assertEquals("TestNG Framework - Introduction to TestNG", actualTitle);
            }
 
      @AfterClass
      public void afterTest() {

       // close the browser
       driver.quit();
    }
}

Output

Browser Started:Edge
Browser Started:Chrome
Title- TestNG Framework - Introduction to TestNG
Test Case One with Thread Id:- 15
Title- TestNG Framework - Introduction to TestNG
Test Case One with Thread Id:- 13Browser Started:Firefox
Title- TestNG Framework - Introduction to TestNG
Test Case One with Thread Id:- 14

We need to specify the values of browser in the TestNG XML file that will pass to the test case file.

<?xml version="1.0"encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="TestSuite"thread-count="3" parallel="tests">
<test name="ChromeTest">
<parameter name="browser"value="Chrome" />
<classes>
<class name="com.selenium.testng.TestNGDemo.TestNGCrossBrowserDemo">
</class>
</classes>
</test>
<test name="FirefoxTest">
<parameter name="browser"value="Firefox" />
<classes>
<class name="com.selenium.testng.TestNGDemo.TestNGCrossBrowserDemo">
</class>
</classes>
</test>
<test name="EdgeTest">
<parameter name="browser"value="Edge" /> 
<classes>
<class name="com.selenium.testng.TestNGDemo.TestNGCrossBrowserDemo">
</class>
</classes>
</test>
</suite>

To execute this program, we need to Right click on the program and select Run as – TestNG Test

The test execution result looks like as shown below. It shows the test execution status of all the tests. As, in this program, 3 tests are executed and all 3 of them passes. The same result can be depicted from below image

TestNG generates various type of reports under test-output folder like emailable-report.html, index.html, testng-results.xml

We are interested in ’emailable-report.html’ report. Open ’emailable-report.html’, as this is a html report open it with browser. Below image shows emailable-report.html.

TestNG also produce “index.html” report and it resides under test-output folder. Below image shows index.html report.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!