Testng.xml

vibssingh

TestNG Framework: How to run Parallel Tests in Selenium with TestNG

HOME


What is Parallel Testing?

Parallelism or multi-threading in software terms is defined as the ability of the software, operating system, or program to execute multiple parts or sub-components of another program simultaneously. 

Parallel testing helps to reduce execution time and efforts and results in faster time to delivery.  In a scenario where we have two versions of software available, and we need to check its stability and compatibility, we can run the two versions simultaneously and find issues at a much faster rate.

How to run Parallel Tests with Selenium?

TestNG provides multiple ways to execute tests in separate threads. In testng.xml, if we set ‘parallel’ attribute on the tag to ‘methods’, testNG will run all the ‘@Test’ methods in tag in a separate thread.

The project Structure looks like shown below:-

To start with, add the below mentioned dependencies to POM.xml (Maven project)

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>ParallelTestsTestNG</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>ParallelTestsTestNG</name>
    <url>http://maven.apache.org</url>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <selenium.version>4.21.0</selenium.version>
        <testng.version>7.10.2</testng.version>
        <maven.compiler.plugin.version>3.13.0</maven.compiler.plugin.version>
        <maven.compiler.source.version>17</maven.compiler.source.version>
        <maven.compiler.target.version>17</maven.compiler.target.version>
        <maven.surefire.plugin.version>3.2.5</maven.surefire.plugin.version>
        <java.version>17</java.version>
    </properties>

    <dependencies>

        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>${selenium.version}</version>
        </dependency>

        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>${testng.version}</version>
            <scope>test</scope>
        </dependency>

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>${maven.surefire.plugin.version}</version>
                <configuration>
                    <suiteXmlFiles>
                        <suiteXmlFile>testng.xml</suiteXmlFile>
                    </suiteXmlFiles>
                </configuration>
                <dependencies>
                    <dependency>
                        <groupId>org.apache.maven.surefire</groupId>
                        <artifactId>surefire-testng</artifactId>
                        <version>${maven.surefire.plugin.version}</version>
                    </dependency>
                </dependencies>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${maven.compiler.plugin.version}</version>
                <configuration>
                    <source>${maven.compiler.source.version}</source>
                    <target>${maven.compiler.target.version}</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

Let us create a class with multiple tests. In the below Example, we have created three test cases or methods . We want to run these methods parallelly. To achieve this, we need to add the below command in testng.xml

parallel="methods"

Below is an example of parallel tests.

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.testng.Assert;
import org.testng.annotations.*;

import java.time.Duration;

public class ParallelTestsExample1 {


    @Test
    public void invalidLoginTest() {

        System.out.println("Test Case 1 with Thread Id - "+Thread.currentThread().getId());
        ChromeOptions options = new ChromeOptions();
        options.addArguments("--start-maximized");
        WebDriver driver = new ChromeDriver(options);
        driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(5));
        driver.get("https://opensource-demo.orangehrmlive.com/web/index.php/auth/login");
        driver.findElement(By.name("username")).sendKeys("admin123123");
        driver.findElement(By.name("password")).sendKeys("adm");
        driver.findElement(By.xpath("//*[@class='oxd-form']/div[3]/button")).click();
        String expectedError = driver.findElement(By.xpath("//*[@class='orangehrm-login-error']/div[1]/div[1]/p")).getText();
        Assert.assertTrue(expectedError.contains("Invalid credentials"));

    }

    @Test
    public void validLoginTest() throws InterruptedException {

        System.out.println("Test Case 2 with Thread Id - "+Thread.currentThread().getId());
        ChromeOptions options = new ChromeOptions();
        options.addArguments("--start-maximized");
        WebDriver driver = new ChromeDriver(options);
        driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(5));
        driver.get("https://opensource-demo.orangehrmlive.com/web/index.php/auth/login");
        driver.findElement(By.name("username")).sendKeys("Admin");
        driver.findElement(By.name("password")).sendKeys("admin123");
        driver.findElement(By.xpath("//*[@class='oxd-form']/div[3]/button")).click();
        String expectedTitle = driver.findElement(By.xpath("//*[@class='oxd-topbar-header-breadcrumb']/h6")).getText();
        Assert.assertTrue(expectedTitle.contains("Dashboard"));
    }

    @Test
    public void forgotLinkTest() {

        System.out.println("Test Case 3 with Thread Id - "+Thread.currentThread().getId());
        ChromeOptions options = new ChromeOptions();
        options.addArguments("--start-maximized");
        WebDriver driver = new ChromeDriver(options);
        driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(5));
        driver.get("https://opensource-demo.orangehrmlive.com/web/index.php/auth/login");

        String expectedLink = driver.findElement(By.className("orangehrm-login-forgot-header")).getText();
        Assert.assertTrue(expectedLink.contains("Forgot your password?"));
    }


}

Now, let us create a testng.xml. Right click on the project and select TestNG -> Convert to TestNG.

The attribute thread-count allows you to specify how many threads should be allocated for this execution.

parallel = “methods” means that the methods will run parallel

The parallel attribute can be extended for multiple values, as below:

·         Methods: Helps run methods in separate threads

·         Tests: Help to run all methods belonging to the same tag in the same thread, means tests will run sequentially

·         Classes: Helps to run all methods belonging to a class in a single thread

·         Instances: Helps run all methods in the same instance in the same thread

testng.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test name="Parallel Tests" parallel = "methods" thread-count="3">
    <classes>
      <class name="com.example.parallel.ParallelTestsExample"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

How to run the tests?

Right-click on testng.xml and select Run AS -> TestNG Suite. If you will run the Test Class – ParallelTestDemo.java as Right click and then Run As TestNG Tests, then the methods will run sequentially.

Execution

Here it can be seen that 3 tests were running on thread no – 19, 20 and 21. Out of all 3 tests, browser for only 1 test is closed and rest 2 browsers are left open.

First thread initialized a browser and set a value to static WebDriver reference. Second thread initialized another browser and set a new value to the same static WebDriver reference and this will impact value set by first thread as it is a static. All threads wanted to close same browser that is the reason there is one configuration method failure as one browser is closed another threads will not find sessions to close the browsers. Browser was closed already so last 2 tests did not able to close the browser.

To overcome this issue, will use ThreadLocal<WebDriver>. The complete program looks like as below:

First, I will create a HelperClass which contains the initialization of driver and closing the driver. I like to keep the tests only in Test Class. This is not mandatory. You can combine the code of both classes in one also.

HelperClass

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;

import java.time.Duration;

public class BaseClass {

    private static final ThreadLocal<WebDriver> driver = new ThreadLocal<WebDriver>();

    @BeforeMethod
    public void setDriver()  {
        
        ChromeOptions options = new ChromeOptions();
        options.addArguments("--start-maximized");
        driver.set(new ChromeDriver(options));
        driver.get().manage().timeouts().implicitlyWait(Duration.ofSeconds(5));
        driver.get().get("https://opensource-demo.orangehrmlive.com/web/index.php/auth/login");
        System.out.println("Before method Thread Id:" + Thread.currentThread().getId());

    }

    public WebDriver getDriver() {
        return driver.get();
    }
    
    @AfterMethod
    public  void closeBrowser() {
        System.out.println("After method Thread Id:" + Thread.currentThread().getId());
        driver.get().quit();
        driver.remove();
    }
}

ParallelTestsExample

import org.openqa.selenium.By;
import org.testng.Assert;
import org.testng.annotations.Test;

public class ParallelTestsExample extends BaseClass{

    @Test
    public void invalidLoginTest() {

        System.out.println("Test Case 1 with Thread Id - "+Thread.currentThread().getId());

        getDriver().findElement(By.name("username")).sendKeys("admin123123");
        getDriver().findElement(By.name("password")).sendKeys("adm");
        getDriver().findElement(By.xpath("//*[@class='oxd-form']/div[3]/button")).click();
        String expectedError = getDriver().findElement(By.xpath("//*[@class='orangehrm-login-error']/div[1]/div[1]/p")).getText();
        Assert.assertTrue(expectedError.contains("Invalid credentials"));

    }

    @Test
    public void validLoginTest() throws InterruptedException {

        System.out.println("Test Case 2 with Thread Id - "+Thread.currentThread().getId());

        getDriver().findElement(By.name("username")).sendKeys("Admin");
        getDriver().findElement(By.name("password")).sendKeys("admin123");
        getDriver().findElement(By.xpath("//*[@class='oxd-form']/div[3]/button")).click();
        Thread.sleep(5000);
        String expectedTitle = getDriver().findElement(By.xpath("//*[@class='oxd-topbar-header-breadcrumb']/h6")).getText();
        Assert.assertTrue(expectedTitle.contains("Dashboard"));
    }

    @Test
    public void forgotLinkTest() {

        System.out.println("Test Case 3 with Thread Id - "+Thread.currentThread().getId());

        String expectedLink = getDriver().findElement(By.className("orangehrm-login-forgot-header")).getText();
        Assert.assertTrue(expectedLink.contains("Forgot your password?"));
    }

}

The output of the above program is

Report Generation

TestNG generates 2 reports – emailable-report.html and index.html

Emailable-Report.html

Go to test-output folder and open emailable-report.html

This report gives a summary of all the tests executed, passed, failed, skipped and retried with their respective execution time.

Index.html

This report provides the detailed description of the tests like no of tests present, no of methods, time taken by each step, total time taken by each steps, testng.xml data and soon.

Run Tests Sequentially

If you will run the Test Class – ParallelTestDemo.java as Right click and then Run As TestNG Tests, then the methods will run sequentially. Here all tests are run with Thread 1 whereas with parallel execution tests were run with different threads.

We can make parallel = none, if don’t want to run them parallel. It is shown below that all the tests are running on Thread 1 that means once a test ends then another test starts on that thread.

<suite name="TestSuite" thread-count="3" parallel="none" >

Congratulations. We are able to run methods parallelly using TestNG.

vibssingh

How to pass Parameters in TestNG

 HOME

One of the important features of TestNG is the ability to pass different test data to a test case as arguments, which is called parametrization

There are mainly two ways through which we can provide parameter values to TestNG tests.

  1. Through testng.xml  XML configuration file
  2. Through DataProviders

In this tutorial, we will discuss using testng.xml for parametrization. If we need to pass some simple values such as String or Integer types to the test methods at runtime, there is something called @Parameter where parameter values through TestNG XML configuration files pass to test.

@Parameters("value")

Let us explain how we can use parameters. To start with, add the below dependencies to the POM.xml in the case of the Maven project.

 <properties>
        <selenium.version>4.14.0</selenium.version>
        <testng.version>7.8.0</testng.version>
        <webdrivermanager.version>5.5.3</webdrivermanager.version>
        <maven.compiler.source>11</maven.compiler.source>
        <maven.compiler.target>11</maven.compiler.target>
 </properties>

 <dependencies>

    <!-- Selenium -->
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-java</artifactId>
        <version>${selenium.version}</version>
    </dependency>

    <!-- TestNG -->
    <dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version>${testng.version}</version>
        <scope>test</scope>
    </dependency>

    <!-- Web Driver Manager -->
    <dependency>
        <groupId>io.github.bonigarcia</groupId>
        <artifactId>webdrivermanager</artifactId>
        <version>${webdrivermanager.version}</version>
    </dependency>

      <!-- Hamcrest Dependency  -->
      <dependency>
          <groupId>org.hamcrest</groupId>
          <artifactId>hamcrest-all</artifactId>
          <version>1.3</version>
          <scope>test</scope>
      </dependency>

 </dependencies>

Implementation Steps

Step 1 – Create a JAVA test class, say, TestNGParameterizationDemo.java

Step 2 – Add test method parameterizedTest() to the test class. This method takes a string as an input parameter

Add the annotation @Parameters(“browser”) to this method. The parameter passes a value from testng.xml

Step 3 – Create a TestNG.xml and pass the value of the parameter in this configuration file.

Below is an example that shows the use of Parameters.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Parameters;
import org.testng.annotations.Test;

import java.util.concurrent.TimeUnit;

import static org.testng.Assert.assertEquals;

public class TestNGParameterizationDemo {
    WebDriver driver;
    By userName = By.name("username");
    By passWord = By.name("password");

    By loginBtn = By.xpath("//*[@class='oxd-form']/div[3]/button");

    By loginTitle = By.xpath("//*[@class='oxd-topbar-header-breadcrumb']/h6");

    By errorMessage = By.xpath("//*[@class='orangehrm-login-error']/div[1]/div/p");


    @BeforeMethod
    @Parameters("browser")
    public void parameterizedTest(String browser) {
        if (browser.equalsIgnoreCase("firefox")) {

            WebDriverManager.firefoxdriver().setup();
            FirefoxOptions options=new FirefoxOptions();
            options.addArguments("--start-maximized");
            driver=new FirefoxDriver(options);
            System.out.println("Browser Started :" + browser);

        } else if (browser.equalsIgnoreCase("chrome")) {
            WebDriverManager.chromedriver().setup();
            ChromeOptions options=new ChromeOptions();
            options.addArguments("--start-maximized");
            driver=new ChromeDriver(options);
            System.out.println("Browser Started :" + browser);
        }

        driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
        driver.get("https://opensource-demo.orangehrmlive.com/");

    }

    @Test
    public void validCredentials()  {

        driver.findElement(userName).sendKeys("Admin");
        driver.findElement(passWord).sendKeys("admin123");
        driver.findElement(loginBtn).click();
        String newPageText = driver.findElement(loginTitle).getText();
        System.out.println("newPageText :" + newPageText);
        assertEquals(newPageText,"Dashboard");
    }

    @Test
    public void invalidCredentials() {

        driver.findElement(userName).sendKeys("1234");
        driver.findElement(passWord).sendKeys("admin3456");
        driver.findElement(loginBtn).click();
        String actualErrorMessage = driver.findElement(errorMessage).getText();
        System.out.println("Actual ErrorMessage :" + actualErrorMessage);
        assertEquals(actualErrorMessage,"Invalid credentials");

    }

    @AfterMethod
    public  void closeBrowser() {
        driver.quit();
    }
}

TestNG.xml looks like this, as shown below. Here, the parameter name is the browser name value for the browser is “Chrome”. So, this “Chrome” value is passed to Test as a parameter, and as a result, a Google Chrome browser opens. Similarly, the same tests are run using Firefox, as it is mentioned in the testng.xml.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">

<suite name="Suite ">

    <test name="Chrome Test">
        <parameter name="browser" value="chrome" />
        <classes>
            <class name="TestNGParameterizationDemo" />
        </classes>
    </test> <!-- Test -->

    <test name="Firefox Test">
        <parameter name="browser" value="firefox" />
        <classes>
            <class name="TestNGParameterizationDemo" />
        </classes>
    </test> <!-- Test -->
</suite> <!-- Suite -->

The output of the above program is

The execution generates reports – Index.html and Emailable-Report.html. The reports are generated under the target folder.

Index.html

Emailable-Report.html

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

vibssingh

How to create and run TestNG.xml of a TestNG class

Last Updated On

HOME

There are 2 ways to run TestNG tests – using Run As TestNG tests and from testng.xml.

In this tutorial, I’m explaining how to create and run tests using TestNG.xml.

Step 1 – In the below project, there is no testng.xml present.

Step 2 – Right click on class name “API_Test.java” and navigate “TestNG–>Convert to TestNG“.

Step 3 – Now a new window for testng.xml will be generated as shown below. Click the Finish Button.

Below is the sample testng.xml.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test thread-count="5" name="Test">
    <classes>
      <class name="com.example.RestAssured_TestNG_Demo.API_Test"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

Step 4 – A xml file named “testng.xml“ will be generated in project hierarchy.

Step 5 – Right click on this testng.xml and select Run As–> Testng Suite. 

Step 6 – You will view the result in two different tabs: Console and “Results of running suite”. Below is the image of Rest of running suite tab.

Multiple Classes

Let us imagine a scenario where there are 3 classes and we want to run them all together, you can done that by creating a testng.xml and mention all 3 classes in that testng.xml.

Select all 3 classes and right click and navigate “TestNG–>Convert to TestNG“.

This is how the testng.xml will look like for multiple classes.

Below is the sample testng.xml.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test thread-count="5" name="Test">
    <classes>
      <class name="com.example.RestAssured_TestNG_Demo.API_Test1"/>
      <class name="com.example.RestAssured_TestNG_Demo.Test3"/>
      <class name="com.example.RestAssured_TestNG_Demo.Test2"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

Right click on this testng.xml and select Run As–> Testng Suite.  You will get the result in two tabs: Console and “Results of running suite”. This shows that all the tests present within the three classes are executed.

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

vibssingh

TestNG Listeners in Selenium

 HOME

The listener is defined as an interface that modifies the default TestNG’s behaviour. There are several interfaces that allow you to modify TestNG’s behaviour that are called “TestNG Listeners”.  It allows customizing TestNG reports or logs. There are many types of TestNG listeners available. Here are a few listeners:

  • IAnnotationTransformer 
  • IAnnotationTransformer2 
  • IHookable 
  • IInvokedMethodListener 
  • IMethodInterceptor 
  • IReporter 
  • ISuiteListener 
  • ITestListener 

When you implement one of these interfaces, you can let TestNG know about it in either of the following ways:

  • Using in your testng.xml file.
  • Using the @Listeners annotation on any of your test classes.

ITestListener has the following methods

  • OnStart –  Invoked before running all the test methods belonging to the classes inside the tag and calling all their Configuration methods.
  • onTestSuccess – onTestSuccess method is called on the success of any Test.
  • onTestFailure – onTestFailure method is called on the failure of any Test.
  • onTestSkipped – onTestSkipped method is called on skipped of any Test.
  • onTestFailedButWithinSuccessPercentage– method is called each time Test fails but is within success percentage. Invoked each time a method fails but has been annotated with successPercentage and this failure still keeps it within the success percentage requested.
  • onFinish – Invoked after all the test methods belonging to the classes inside the tag have run and all their Configuration methods have been called.
  • onTestStart – Invoked each time before a test will be invoked. The ITestResult is only partially filled with the references to class, method, start millis, and status.

Here, I explain the use of listener – ITestListener  in a program mentioned below

Step 1) Create a class “ListenerDemo” that implements ‘ITestListener’. Add methods like onTestFailure, onTestSkipped, onTestStart, onTestSuccess to this class

Step 2) Create another class “ListenerTestCases” for the login process automation. Selenium will execute these ‘TestCases’ to login automatically.

Step 3) Next, implement this listener in our regular project class i.e. “ListenerTestCases “. There are two different ways to connect to the class and interface.

The first way is to use Listeners annotation (@Listeners) as shown below:

@Listeners(com.selenium.testng.TestNGDemo.ListenerDemo.class)

Step 4): Execute the “ListenerTestCases” class. Methods in class “TestPass ” are called automatically according to the behaviour of methods annotated as @Test.

Step 5): Verify the Output that logs display at the console.

import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;
 
public class ListenerDemo implements ITestListener {
 
            // When Test case get failed, this method is called.
            public void onTestFailure(ITestResult Result) {
                        System.out.println("The name of the testcase failed is :" + Result.getName());
            }
 
            // When Test case get Skipped, this method is called.
            public void onTestSkipped(ITestResult Result) {
                        System.out.println("The name of the testcase Skipped is :" + Result.getName());
            }
 
            // When Test case get Started, this method is called.
            public void onTestStart(ITestResult Result) {
                        System.out.println(Result.getName() + " test case started");
            }
 
            // When Test case get passed, this method is called.
            public void onTestSuccess(ITestResult Result) {
                        System.out.println("The name of the testcase passed is :" + Result.getName());
            } 
}

In the below test, there are 2 test cases. One Test passes and another fails. When we are executing ListenerTestCases, it will call the ListenersDemo internally.

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
importorg.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;
 
@Listeners(com.selenium.testng.TestNGDemo.ListenerDemo.class)
public class ListenerTestCases {
 
            static WebDriver driver;
 
            @Test
            public void TestPass() {
                        System.setProperty("webdriver.chrome.driver",
                                    "C:\\Users\\Vibha\\Desktop\\SeleniumKT\\chromedriver_win32\\chromedriver.exe");
                        driver= new ChromeDriver();
                        driver.get("https://opensource-demo.orangehrmlive.com/");
                        driver.findElement(By.name("txtUsername")).sendKeys("Admin");
                        driver.findElement(By.name("txtPassword")).sendKeys("admin123");
                        driver.findElement(By.id("btnLogin")).submit();
                        String dashboardTitle = driver.findElement(By.id("welcome")).getText();
                        Assert.assertTrue(dashboardTitle.contains("Welcome"));
            }
 
            @Test
            public void TestFail() {
                        System.setProperty("webdriver.chrome.driver",
                                    "C:\\Users\\Vibha\\Desktop\\SeleniumKT\\chromedriver_win32\\chromedriver.exe");
                        driver= new ChromeDriver();
                        driver.get("https://opensource-demo.orangehrmlive.com/");
                        driver.findElement(By.name("txtUsername")).sendKeys("Admin");
                        driver.findElement(By.name("txtPassword")).sendKeys("admin123");
                        driver.findElement(By.id("btnLogin")).submit();
                        String dashboardTitle = driver.findElement(By.id("welcome")).getText();
                        Assert.assertTrue(dashboardTitle.contains("Hello"));
            }
}

Output
TestFail test case started
The name of the testcase failed is :TestFail
TestPass test case started
The name of the testcase passed is :TestPass
PASSED: TestPass
FAILED: TestFail
java.lang.AssertionError: did not expect to find [true] but found [false]

The output of the above program is

To execute this program, we need to Right-click and select Run as – TestNG.

There is another way to execute the Listener class, which is using testng.xml. There is a Listener class named “ListenerDemo” where the implementation of various methods of the listener is present. If we want to run the tests using testng.xml, then there is no need of mentioning Listeners in the ListenerTestCases class.

<?xml version = "1.0"encoding = "UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name = "TestSuite">
<listeners>
<listener class-name ="com.selenium.testng.TestNGDemo.ListenerDemo"/>
</listeners>
 
<test name ="Test">
<classes>
<class name="com.selenium.testng.TestNGDemo.ListenerTestCases"/>
</classes>
</test>
</suite>

The test execution result will look like something shown below.

TestNG generates various types of reports under the test-output folder. Open “emailable-report.html”, as this is an HTML report open it with the browser. It will look like something below.

TestNG also produces an “index.html” report, and it resides under the test-output folder.

There is another example of Listener –ITestResult.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

vibssingh

Cross Browser Testing using Selenium and TestNG

 HOME

What is Cross Browser Testing?

Cross Browser is a technique in which a web application tests on different browsers and operating systems.  Cross Browser testing, make sure that the site rendered the same in every browser.

Suppose, we have to execute 25 tests cases to test a web application on Google Chrome Browser and it takes around 4 hrs to execute these tests. However, we do not know if a user is going to access the web application on which browser. Therefore, now the same set of test cases need to executes on Firefox, Edge, Internet Explorer, Safari and Opera.

Therefore, now we need to execute 25*6=150 test cases and test execution hour changes from 4 to 24 hrs to check that the web application is working as expected. What is the best approach to handle this situation is to automate these tests and perform cross browser testing

Why do we need to perform cross browser testing?

Each website is built by anyone or combination of these technologies – HTML, CSS and Javascript. Each browser uses different rendering engines to compute these technologies. Chrome uses Blink, WebKit on iOS, V8 JavaScript engine, Firefox uses Gecko, Edge uses Chromium-based with Blink and V8 engines, Safari uses Webkit rendering engine, IE uses Trident and so on.

1) Font size, image orientation and alignment mismatch in different browsers

2) Different browser is compatible with different operating systems

3) CSS,HTML validation difference can be there

Lets see an example of Cross Browser testing using Selenium and TestNG.

Step 1 – Add the below dependencies to the POM.xml, in case of Maven project.

<dependencies>
  
      <dependency>
          <groupId>org.seleniumhq.selenium</groupId>
          <artifactId>selenium-java</artifactId>
          <version>3.141.59</version>
      </dependency>
      
      <dependency>
          <groupId>io.github.bonigarcia</groupId>
          <artifactId>webdrivermanager</artifactId>
          <version>5.2.1</version>
       </dependency>

      <dependency>
           <groupId>org.testng</groupId>
           <artifactId>testng</artifactId>
           <version>7.5</version>
           <scope>test</scope>
      </dependency>

  </dependencies>

Step 2 – We have used Selenium WebDriver with TestNG to automate the test cases to run on 3 different browsers – Chrome, Firefox and Edge

Step 3 – The test are running on all 3 browsers in parallel as we have used  parallel=“tests”

Step 4 – thread-count=”3″ means that 3 threads will start and each browser will use a thread

Step 5 –  @Parameters(“browser”) – parameter will be passed from testng.xml. To know more about @Parameters in TestNG, please refer to this tutorial.

Let us create a helper class which contains the methods to initialize the browser and close the browser.

import java.util.concurrent.TimeUnit;
import org.openqa.selenium.WebDriver;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Parameters;

import io.github.bonigarcia.wdm.WebDriverManager;

public class BaseClass {
	
	WebDriver driver;
	  
    @BeforeMethod
    @Parameters("browser")
    public void setup(String browser) throws Exception {

       if (browser.equalsIgnoreCase("firefox")) {
          
    	    driver = WebDriverManager.firefoxdriver().create();
            System.out.println("Browser Started:" + browser);
       
       } else if (browser.equalsIgnoreCase("chrome")) {
        	
        	 driver = WebDriverManager.chromedriver().create();
             System.out.println("Browser Started:" + browser);
       } else if (browser.equalsIgnoreCase("edge")) {
            
             driver = WebDriverManager.edgedriver().create();
             System.out.println("Browser Started:" + browser);
       } else {               
                 throw new Exception("Browser is not correct");
        }
       
       driver.get("https://opensource-demo.orangehrmlive.com/");
       driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
       driver.manage().window().maximize();
    }
	

	    @AfterMethod
	    public  void closeBrowser() {
	    	
	    	driver.quit();
	    	
			  
	    }
	}

CrossBrowserTests

import static org.testng.Assert.assertTrue;
import org.openqa.selenium.By;
import org.testng.Assert;
import org.testng.annotations.Test;

public class CrossBrowserTests extends BaseClass{
	
    @Test
    public void invalidLoginTest() throws InterruptedException {
    	
    	System.out.println("Test Case1");
               
        driver.get("https://opensource-demo.orangehrmlive.com/");
        driver.manage().window().maximize();      
        driver.findElement(By.name("txtUsername")).sendKeys("admin123123");
        driver.findElement(By.name("txtPassword")).sendKeys("adm");
        driver.findElement(By.id("btnLogin")).click();
        String expectedError = driver.findElement(By.id("spanMessage")).getText();
        Assert.assertTrue(expectedError.contains("Invalid credentials"));

    }

    @Test
    public void verifyLinkedIn() {

    	System.out.println("Test Case2");
    	
        driver.manage().window().maximize();
        Boolean linkedInIcon = driver.findElement(By.xpath("//*[@id='social-icons']/a[1]/img")).isEnabled();
        assertTrue(linkedInIcon);
    }

    
    @Test
    public void validLoginTest() throws InterruptedException {
    	
    	System.out.println("Test Case3");

        driver.findElement(By.name("txtUsername")).sendKeys("Admin");
        driver.findElement(By.name("txtPassword")).sendKeys("admin123");
        driver.findElement(By.id("btnLogin")).click();
        String expectedTitle = driver.findElement(By.xpath("//*[@id='content']/div/div[1]/h1")).getText();
        Assert.assertTrue(expectedTitle.contains("Dashboard"));
    }

}

We need to specify the values of browser in the TestNG XML file that will pass to the test case file.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="tests" thread-count="3">
  <test name="Chrome Test">
   <parameter name="browser" value="chrome" />
    <classes>
      <class name="com.example.crossbrowser.CrossBrowserTests"/>
    </classes>
  </test> <!-- Test -->
  
    <test name="firefox Test">
   <parameter name="browser" value="firefox" />
    <classes>
      <class name="com.example.crossbrowser.CrossBrowserTests"/>
    </classes>
  </test> <!-- Test -->
  
    <test name="Edge Test">
   <parameter name="browser" value="edge" />
    <classes>
      <class name="com.example.crossbrowser.CrossBrowserTests"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

To execute this program, we need to Right click on the program and select Run as – TestNG Test

The test execution result looks like as shown below. It shows the test execution status of all the tests. As, in this program, 3 tests are executed and all 3 of them passes. The same result can be depicted from below image.

TestNG Report Generation

TestNG generates various type of reports under test-output folder like emailable-report.html, index.html, testng-results.xml

We are interested in ’emailable-report.html’ report. Open ’emailable-report.html’, as this is a html report open it with browser. Below image shows emailable-report.html.

TestNG also produce “index.html” report and it resides under test-output folder. Below image shows index.html report.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!