What is Parallel Testing?
Parallelism or multi-threading in software terms is defined as the ability of the software, operating system, or program to execute multiple parts or sub-components of another program simultaneously.
Parallel testing helps to reduce execution time and efforts and results in faster time to delivery. In a scenario where we have two versions of software available, and we need to check its stability and compatibility, we can run the two versions simultaneously and find issues at a much faster rate.
How to run Parallel Tests with Selenium
TestNG provides multiple ways to execute tests in separate threads. In testng.xml, if we set ‘parallel’ attribute on the tag to ‘methods’, testNG will run all the ‘@Test’ methods in tag in a separate thread.
The project Structure looks like shown below:-
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
<test name = "Test Demo" parallel = "methods" thread-count ="2">
<classes>
<class name="TestNGParallelExample.ParallelTestsExample"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->
The attribute thread-count allows you to specify how many threads should be allocated for this execution.
parallel = “methods” means that the methods will run parallel
The parallel attribute can be extended for multiple values, as below:
· Methods: Helps run methods in separate threads
· Tests: Help to run all methods belonging to the same tag in the same thread, means tests will run sequentially
· Classes: Helps to run all methods belonging to a class in a single thread
· Instances: Helps run all methods in the same instance in the same thread
How to run Tests parallel
Right click on TestNG.xml and Run AS TestNG Suite. If you will run the Test Class – ParallelTestDemo.java as Right click and then Run As TestNG Tests, then the methods will run sequentially.
In this Example, three test cases or methods will run parallel and enter search word like TestNG Framework, Maven, and Selenium WebDriver in Google.
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;
public class ParallelTestsExample {
public static WebDriver driver;
@BeforeTest
public void setUp(){
System.setProperty("webdriver.chrome.driver",
"C:\\Users\\Vibha\\Software\\chromedriver\\chromedriver.exe");
}
@Test
public void invalidLoginTest() throws InterruptedException {
System.out.println("Test Case 1 with Thread Id - "+Thread.currentThread().getId());
driver = new ChromeDriver();
driver.get("https://opensource-demo.orangehrmlive.com/");
driver.findElement(By.name("txtUsername")).sendKeys("admin123123");
driver.findElement(By.name("txtPassword")).sendKeys("adm");
driver.findElement(By.id("btnLogin")).click();
String expectedError = driver.findElement(By.id("spanMessage")).getText();
Assert.assertTrue(expectedError.contains("Invalid credentials"));
}
@Test
public void blankLoginTest() throws InterruptedException {
System.out.println("Test Case 2 with Thread Id - "+Thread.currentThread().getId());
driver = new ChromeDriver();
driver.get("https://opensource-demo.orangehrmlive.com/");
driver.findElement(By.name("txtUsername")).sendKeys("");
driver.findElement(By.name("txtPassword")).sendKeys("");
driver.findElement(By.id("btnLogin")).click();
String expectedError = driver.findElement(By.id("spanMessage")).getText();
Assert.assertTrue(expectedError.contains("Username cannot be empty"));
}
@Test
public void validLoginTest() throws InterruptedException {
System.out.println("Test Case 3 with Thread Id - "+Thread.currentThread().getId());
driver = new ChromeDriver();
driver.get("https://opensource-demo.orangehrmlive.com/");
driver.findElement(By.name("txtUsername")).sendKeys("Admin");
driver.findElement(By.name("txtPassword")).sendKeys("admin123");
driver.findElement(By.id("btnLogin")).click();
String expectedTitle = driver.findElement(By.xpath("//*[@id='content']/div/div[1]/h1")).getText();
Assert.assertTrue(expectedTitle.contains("Dashboard"));
}
@Test
public void forgotLinkTest() throws InterruptedException {
System.out.println("Test Case 4 with Thread Id - "+Thread.currentThread().getId());
driver = new ChromeDriver();
driver.get("https://opensource-demo.orangehrmlive.com/");
String expectedLink = driver.findElement(By.id("forgotPasswordLink")).getText();
Assert.assertTrue(expectedLink.contains("Forgot your password?"));
}
@AfterTest
public void tear_down() {
driver.close();
}
}
Execution
Here it can be seen that 2 tests were running on thread no – 19 and 20 and other 2 tests again also run on Thread 18 and 19 because we have mentioned thread-count as 2. Out of all 4 tests, browser for only 1 test is closed and rest 3 browsers are left open.
First thread initialized a browser and set a value to static WebDriver reference. Second thread initialized another browser and set a new value to the same static WebDriver reference and this will impact value set by first thread as it is a static. All threads wanted to close same browser that is the reason there is one configuration method failure as one browser is closed another threads will not find sessions to close the browsers. Browser was closed already so last 3 tests did not able to close the browser.
To overcome this issue, will use ThreadLocal<WebDriver>. The complete program looks like as below:
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;
public class ParallelTestsExampleWithThread {
public WebDriver driver;
private static final ThreadLocal<WebDriver> WEBDRIVER_THREADLOCAL = new ThreadLocal<WebDriver>();
@BeforeMethod
public void setUp(){
System.setProperty("webdriver.chrome.driver",
"C:\\Users\\SingVi04\\Vibha\\Software\\chromedriver\\chromedriver.exe");
driver = new ChromeDriver();
WEBDRIVER_THREADLOCAL.set(driver);
System.out.println("Before method Thread Id:" + Thread.currentThread().getId());
}
@Test
public void invalidLoginTest() {
System.out.println("Test Case 1 with Thread Id - "+Thread.currentThread().getId());
driver = WEBDRIVER_THREADLOCAL.get();
driver.get("https://opensource-demo.orangehrmlive.com/");
driver.findElement(By.name("txtUsername")).sendKeys("admin123123");
driver.findElement(By.name("txtPassword")).sendKeys("adm");
driver.findElement(By.id("btnLogin")).click();
String expectedError = driver.findElement(By.id("spanMessage")).getText();
Assert.assertTrue(expectedError.contains("Invalid credentials"));
}
@Test
public void blankLoginTest() {
System.out.println("Test Case 2 with Thread Id - "+Thread.currentThread().getId());
driver = WEBDRIVER_THREADLOCAL.get();
driver.get("https://opensource-demo.orangehrmlive.com/");
driver.findElement(By.name("txtUsername")).sendKeys("");
driver.findElement(By.name("txtPassword")).sendKeys("");
driver.findElement(By.id("btnLogin")).click();
String expectedError = driver.findElement(By.id("spanMessage")).getText();
Assert.assertTrue(expectedError.contains("Username cannot be empty"));
}
@Test
public void validLoginTest() {
System.out.println("Test Case 3 with Thread Id - "+Thread.currentThread().getId());
driver = WEBDRIVER_THREADLOCAL.get();
driver.get("https://opensource-demo.orangehrmlive.com/"
driver.findElement(By.name("txtUsername")).sendKeys("Admin");
driver.findElement(By.name("txtPassword")).sendKeys("admin123");
driver.findElement(By.id("btnLogin")).click();
String expectedTitle = driver.findElement(By.xpath("//*[@id='content']/div/div[1]/h1")).getText();
Assert.assertTrue(expectedTitle.contains("Dashboard"));
}
@Test
public void forgotLinkTest() {
System.out.println("Test Case 4 with Thread Id - "+Thread.currentThread().getId());
driver = WEBDRIVER_THREADLOCAL.get();
driver.get("https://opensource-demo.orangehrmlive.com/");
String expectedLink = driver.findElement(By.id("forgotPasswordLink")).getText();
Assert.assertTrue(expectedLink.contains("Forgot your password?"));
}
@AfterMethod
public void tear_down() {
WebDriver driver = WEBDRIVER_THREADLOCAL.get();
System.out.println("After method Thread Id:" + Thread.currentThread().getId());
driver.close();
}
}
Report Generation
TestNG generates 2 reports – emailable-report.html and index.html
Emailable-Report.html
Go to test-output folder and open emailable-report.html
This report gives a summary of all the tests executed, passed, failed, skipped and retried with their respective execution time.
Index.html
This report provides the detailed description of the tests like no of tests present, no of methods, time taken by each step, total time taken by each steps, testng.xml data and soon.
Run Tests Sequentially
If you will run the Test Class – ParallelTestDemo.java as Right click and then Run As TestNG Tests, then the methods will run sequentially. Here all tests are run with Thread 1 whereas with parallel execution tests were run with different threads.
We can make parallel = none, if don’t want to run them parallel. It is shown below that all the tests are running on Thread 1 that means once a test ends then another test starts on that thread.
<suite name="TestSuite" thread-count="3" parallel="none" >
Test Reports
Emailable-Report.html
Index.html
Congratulations. We are able to run methods parallelly using TestNG.
QA Automation Expert 
It's quite informative..can you help with how to prioritize test cases while running..
LikeLike
Thanks Abha. There will be a separate blog on prioritizing the test cases using TestNG. Will keep you posted
LikeLike
how can i disable some of my test cases in testng. i don't want to delete them… Thanks for your help in advance
LikeLike
Hi, we can disable test cases in TestNG. There is parameter like @Test(enabled = false) which will disable the execution of tests in TestNG. For more detail, can refer – https://configureselenium.blogspot.com/2020/02/how-to-disable-selenium-test-cases.html
LikeLike