Integration Of Jenkins With Selenium WebDriver

HOME

What is Jenkins?

Jenkins is a well-known open-source tool that aids in the implementation of Continuous Integration (CI) and Continuous Deployment/Continuous Delivery (CD) processes by automating parts of the software development pipeline such as building, testing, and deployment.

What is Selenium?

Selenium is an open-source automation tool that has been widely used for testing web applications.  It is basically a suite of software or a set of JAR files to automate web browsers across various browsers. Selenium is used by many companies, but a few to mention are Netflix, Google, HubSpot, Fitbit, and more.  Selenium supports multiple languages like Python, Pearl, Ruby, .Net, Java, C#, PHP. 

Selenium supports parallel test execution. That means we can open multiple browsers at the same time and execute the test cases. This decrease the execution time significantly. Selenium supports Cross browser test execution.

Jenkins Integration With Selenium 

It is recommended to go through these tutorials:-

Download and install Jenkins on Windows10

Configure JAVA_HOME and MAVEN_HOME

Implementation Steps

Step 1 – Create a project for running the tests using Selenium WebDriver and TestNG

Step 2 – Create the Test Code

Below is the sample POM.xml.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>com.example</groupId>
	<artifactId>SeleniumTestNG_Demo</artifactId>
	<version>0.0.1-SNAPSHOT</version>

	<properties>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
		<selenium.version>4.3.0</selenium.version>
		<testng.version>7.6.0</testng.version>
		<webdrivermanager.version>5.2.1</webdrivermanager.version>
		<apache.common.version>2.4</apache.common.version>
		<maven.compiler.plugin.version>3.10.1</maven.compiler.plugin.version>
		<maven.compiler.source.version>11</maven.compiler.source.version>
		<maven.compiler.target.version>11</maven.compiler.target.version>
		<maven.surefire.plugin.version>3.0.0-M7</maven.surefire.plugin.version>

	</properties>

	<dependencies>

		<!-- Selenium -->
		<dependency>
			<groupId>org.seleniumhq.selenium</groupId>
			<artifactId>selenium-java</artifactId>
			<version>${selenium.version}</version>
		</dependency>

		<!-- TestNG -->
		<dependency>
			<groupId>org.testng</groupId>
			<artifactId>testng</artifactId>
			<version>${testng.version}</version>
			<scope>test</scope>
		</dependency>

		<!-- Web Driver Manager -->
		<dependency>
			<groupId>io.github.bonigarcia</groupId>
			<artifactId>webdrivermanager</artifactId>
			<version>${webdrivermanager.version}</version>
		</dependency>

		<dependency>
			<groupId>org.hamcrest</groupId>
			<artifactId>hamcrest</artifactId>
			<version>2.2</version>
			<scope>test</scope>
		</dependency>
			
		<!-- Allure Report -->	
		<dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-testng</artifactId>
        <version>${allure.testng.version}</version>
        <scope>test</scope>
    </dependency>

	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-compiler-plugin</artifactId>
				<version>${maven.compiler.plugin.version}</version>
				<configuration>
					<source>${maven.compiler.source.version}</source> 
					<target>${maven.compiler.target.version}</target> 
				</configuration>
			</plugin>
			 <plugin>
              <groupId>org.apache.maven.plugins</groupId>
              <artifactId>maven-surefire-plugin</artifactId>
              <version>${maven.surefire.plugin.version}</version>
              <configuration>
                    <suiteXmlFiles>
                        <suiteXmlFile>testng.xml</suiteXmlFile>
                    </suiteXmlFiles>
             </configuration>          
        </plugin>
      </plugins>
  </build>
</project>

BasePage

package com.example;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.support.PageFactory;

public class BasePage {
	
	  public WebDriver driver;

	  public BasePage(WebDriver driver) {
		  this.driver = driver;
		  PageFactory.initElements(driver,this);
	}

}

HomePage

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;

public class HomePage extends BasePage {

	public HomePage(WebDriver driver) {
		super(driver);

	}

	  @FindBy(xpath = "//*[@id='app']/div[1]/div[1]/header/div[1]/div[1]/span/h6")
	  public  WebElement homePageUserName;

	  public String getHomePageText() {
	       return homePageUserName.getText();
   }

}

LoginPage

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;

public class LoginPage extends BasePage{
	
	 public LoginPage(WebDriver driver) {
		 super(driver);
		
    }
	
	@FindBy(name = "username")
    public WebElement userName;
 
    @FindBy(name = "password")
    public WebElement password;
 
    @FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/form/div[3]/button")
    public WebElement login;
 
    @FindBy(xpath = "//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/div/div[1]/div[1]/p")
    public  WebElement errorMessage;
   
    
    public void login(String strUserName, String strPassword) {
    	 
    	userName.sendKeys(strUserName);
    	password.sendKeys(strPassword);
    	login.click();
 
    }
  
   
    public String getErrorMessage() {
        return errorMessage.getText();
    }
    
 
}

BaseTests

import java.time.Duration;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import io.github.bonigarcia.wdm.WebDriverManager;

public class BaseTests {
	
	public static WebDriver driver;
	public final static int TIMEOUT = 10;    
 
	@BeforeMethod
    public void setup() {
    	WebDriverManager.chromedriver().setup();
	    driver = new ChromeDriver();
	    driver.manage().window().maximize();
	    driver.get("https://opensource-demo.orangehrmlive.com/");	    
	    driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(TIMEOUT));

    }
   
    @AfterMethod
    public void tearDown() {
        driver.quit();
    }
    
}

LoginPageTests

import org.testng.Assert;
import org.testng.annotations.Test;

public class LoginPageTests extends BaseTests{
	 

    @Test
    public void invalidCredentials() {
   
	    LoginPage objLoginPage = new LoginPage(driver);
    	objLoginPage.login("admin$$", "admin123");
    	 
    	// Verify Error Message
    	 Assert.assertEquals("Invalid credentials",objLoginPage.getErrorMessage());
    
    }
    

    @Test
    public void validLogin() {
   
	    LoginPage objLoginPage = new LoginPage(driver);
    	objLoginPage.login("Admin", "admin123");
    	 
    	HomePage objHomePage = new HomePage(driver);
    	
    	// Verify Home Page
    	Assert.assertEquals("Dashboard",objHomePage.getHomePageText());
    
    }
     
}

Step 3 – Start Jenkins

Open the Jenkins dashboard – http://localhost:8081/.  In the Jenkins dashboard, click New Item to create a new project.  

Step 4 – Create a FreeStyle project in Jenkins

Specify the name of the project and click the Freestyle Project option. Click the OK button.

Step 5 – Mention the description

In the General section, enter the project description in the Description box.

Step 6 – Source Management

In the Source Code Management section, select None.

       

Step 7 – Select Invoke top-level Maven targets in build steps

In the Build Steps section, select Invoke top-level Maven targets.

Step 8 – Provide the build steps

The Build Steps window will extend. Mention the below details:-

Maven Version – MAVEN_HOME

Goals – clean test

Click on the Advanced button.

Step 9 – Mention the path of the project

Specify the full path to pom.xml in POM. Click on the Apply Button and then on the Save button.

Step 10 – Verify the the project is created in Jenkins

The below screen shows that a new project with the name of SeleniumTestNGDemo is created.

Step 11 – Execute the tests

Click on the Build Now option to run the tests.

Step 12 – View the execution result

Click on the Console Output to see the test execution result.

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

Additional Tutorials on Jenkins

Advertisement

Gradle – Extent Report Version 5 for Cucumber, Selenium, and TestNG

HOME

The previous tutorial explained the generation of Extent Reports Version 5 for Cucumber 7 and TestNG in a Maven project. In this tutorial, I will explain the steps to create an Extent Report Version 5 for Cucumber, Selenium, and TestNG in a Gradle project.

Pre Requisite

  1. Java 8 or above installed
  2. Eclipse or IntelliJ IDE installed
  3. Gradle Installed
  4. Environment variables JAVA_HOME and GRADLE_HOME are correctly configured

In this tutorial, I’ll create a BDD Framework for the testing of web applications using Cucumber, and Selenium WebDriver with TestNG. This framework consists of:-

  1. Cucumber Java- 7.6.0
  2. Cucumber JUnit– 7.6.0
  3. Java 11
  4. TestNG – 7.6.0
  5. Gradle – 7.5.1
  6. Selenium – 4.3.0
  7. ExtentReport – 5.0.9
  8. GrassHopper Cucumber Adapter – 1.7.0

Implementation Steps

  1. Add ExtentReport dependency to the build.gradle
  2. Add ExtentCucumberAdapter plugin to task cucumber
  3. Add Cucumber, Selenium and TestNG, and dependencies in build.gradle
  4. Create Locator and Action classes and Step Definition corresponding to the feature file
  5. Create extent.properties file in the resources folder and paste the below code
  6. Execute the Tests
  7. View the Extent Report

There is a tutorial that explains the Integration of Cucumber, Selenium, and TestNG in a Gradle project. Please refer to this tutorial – Gradle Project with Cucumber, Selenium and TestNG.

Step 1 – Add Extent Report dependency to the build.gradle

To create an Extent Report, we need to add the below-mentioned dependency in the build.gradle

implementation 'tech.grasshopper:extentreports-cucumber7-adapter:1.7.0'
implementation 'com.aventstack:extentreports:5.0.9'  

Step 2 – Add ExtentCucumberAdapter plugin to task cucumber

task cucumber() {
    dependsOn assemble, compileTestJava
    doLast {
        javaexec {         
            main = "io.cucumber.core.cli.Main"
            classpath = configurations.cucumberRuntime + sourceSets.main.output + sourceSets.test.output
            args = ['--plugin', 'pretty', 
            '--plugin', 'io.qameta.allure.cucumber7jvm.AllureCucumber7Jvm',
            '--plugin', 'com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter:',
            '--glue', 'com.example.definitions', 'src/test/resources']
        }
    }
}

Step 3 – Add Cucumber, Selenium, and TestNG dependencies in build.gradle

dependencies {

    testImplementation 'io.cucumber:cucumber-java:7.6.0'
    testImplementation 'io.cucumber:cucumber-junit:7.6.0'
    
     //TestNG  
     testImplementation 'org.testng:testng:7.6.0'
    
    //ExtentReport    
    implementation 'tech.grasshopper:extentreports-cucumber7-adapter:1.7.0'
    implementation 'com.aventstack:extentreports:5.0.9' 

    // This dependency is used by the application.
    implementation 'com.google.guava:guava:30.1.1-jre'
    implementation 'org.seleniumhq.selenium:selenium-java:4.4.0'
    implementation 'io.github.bonigarcia:webdrivermanager:5.3.0'
}

The complete build.gradle is shown below:

plugins {
    // Apply the application plugin to add support for building a CLI application in Java.
    id 'application'
}

repositories {
    // Use Maven Central for resolving dependencies.
    mavenCentral()
}

java {
    sourceCompatibility = 11
    targetCompatibility = 11
}

dependencies {
   
    // Use TestNG framework, also requires calling test.useTestNG() below
    testImplementation 'io.cucumber:cucumber-java:7.6.0'
    testImplementation 'io.cucumber:cucumber-testng:7.6.0'
             
    //TestNG  
     testImplementation 'org.testng:testng:7.6.0'
      
    //ExtentReport    
     implementation 'tech.grasshopper:extentreports-cucumber7-adapter:1.7.0'
     implementation 'com.aventstack:extentreports:5.0.9'  
      
     //Others  
     implementation 'com.google.guava:guava:31.0.1-jre'
     implementation 'org.seleniumhq.selenium:selenium-java:4.4.0'
     implementation 'io.github.bonigarcia:webdrivermanager:5.3.0'

}

application {
    // Define the main class for the application.
    mainClass = 'com.example.App'
}

tasks.named('test') {
    // Use TestNG for unit tests.
    useTestNG()
}

configurations {
    cucumberRuntime {
        extendsFrom testImplementation
    }
}

task cucumber() {
    dependsOn assemble, compileTestJava
    doLast {
        javaexec {         
            main = "io.cucumber.core.cli.Main"
            classpath = configurations.cucumberRuntime + sourceSets.main.output + sourceSets.test.output
            args = ['--plugin', 'pretty', 
            '--plugin', 'io.qameta.allure.cucumber7jvm.AllureCucumber7Jvm',
            '--plugin', 'com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter:',
            '--glue', 'com.example.definitions', 'src/test/resources']
        }
    }
}

Step 4 – Create Locator and Action classes and Step Definition corresponding to the feature file

As mentioned above, there is another tutorial that explains the project structure as well as the feature file and corresponding Step Definitions, please refer to this tutorial – Gradle Project with Cucumber, Selenium, and TestNG.

Step 5 – Create extent.properties file in the resources folder and paste the below code

#Extent Report
extent.reporter.spark.start=true
extent.reporter.spark.out=Reports/Spark.html
 
#PDF Report
extent.reporter.pdf.start=true
extent.reporter.pdf.out=PdfReport/ExtentPdf.pdf
 
#HTML Report
extent.reporter.html.start=true
extent.reporter.html.out=HtmlReport/ExtentHtml.html
 
#FolderName
basefolder.name=ExtentReports/SparkReport_
basefolder.datetimepattern=d_MMM_YY HH_mm_ss
 
#Screenshot
screenshot.dir=/Screenshots/
screenshot.rel.path=../Screenshots/
 
#Base64
extent.reporter.spark.base64imagesrc=true
 
#System Info
systeminfo.os=windows
systeminfo.version=10

Step 6 – Execute the Tests

Go to the app project and run the tests, using the below command

gradle cucumber

The output of the above program is

Step 7: View the ExtentReports

Refresh the project and will see a new folder – SparkReport_ which further contains 4 folders -Html Report, Pdf Report, Reports, and Screenshots.

The Extent Report will be present in the Report’s folder with the name Spark.html. PDF Report is present in the Pdf Report folder and HTML Report is present in the HTML report folder. We can see that the Screenshot’s folder is empty because we have used the base64imagesrc feature, which resulted in no physical screenshots. The screenshots are embedded in the reports.

Right-click and open the ExtentHtml.html report with the Web Browser. The report also has a summary section that displays the summary of the execution. The summary includes the overview of the pass/fail using a pictogram, start time, end time, and pass/fail details of features as shown in the image below.

ExtentHtml.html

The failed test has a screenshot embedded in it. Double-click on mase64image, and it will open the screenshot in full screen.

Screenshot of failed Test Case

PDF Report

To know more about PDF Report generation, please refer to this tutorial – PDF ExtentReport for Cucumber and TestNG.

Spark Report

Right-click and open the Spark.html report with the Web Browser.

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

Selenium 4 Grid – Parallel Testing

HOME

The previous tutorial explained running the tests in Selenium4 Grid using the Standalone option. One of the major advantages of Selenium Grid is the ability to run tests parallelly on multiple browsers simultaneously. Parallel testing helps to reduce execution time and efforts and results in a faster time to delivery.

In this tutorial, we will run the same set of tests on Chrome, Firefox, and Edge browsers sequentially initially to confirm that we can perform Cross Browser testing also

It is very important to construct our tests thread-safe in order to run them in parallel without a problem

This example uses Selenium 4 with TestNG.

Implementation Steps

1. Download Selenium Grid 4

The latest version of Selenium 4 is 4.3.0 and the same can be downloaded from the official website of Selenium

2. Download various Browser driver exe

It is recommended to download the exe of various browsers in the same location where the Selenium WebDriver jar file is present. This is because Selenium 4 Alpha has the ability to automatically detect the WebDrivers present on the node machine. For this example, I have downloaded Chrome, Firefox, and edge drivers.

3. Start Selenium Server Jar

Open a command line terminal. Go to the location where these files are present

cd C:/Users/Vibha/Software/SeleniumGrid

Use the below command to run selenium-server standalone jar files.

java -jar selenium-server-4.3.0.jar standalone --port 4445

It’s optional to mention the port number at the end of the command. By default, it will use port 4444. It is good practice to mention the port at the end to avoid any conflict between the ports.

4. Open Selenium Console

The server is listening on http://172.30.96.1:4445/ui/index.html  which is the same as shown in the above image. This is a dynamic address, so make sure to get the address from the logs when Selenium Grid is started.

The Grid automatically identifies that the WebDrivers for Chrome and Firefox are present on the system.

5. Add dependencies to the project

In this example, we are using a Maven project, so are adding the dependencies to the POM.xml.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.example</groupId>
    <artifactId>Selenium4Parallel</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>11</maven.compiler.source>
        <maven.compiler.target>11</maven.compiler.target>
    </properties>

    <dependencies>

        <!-- Selenium 4 Dependency -->
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>4.1.2</version>
        </dependency>

        <!-- Selenium WebDriver Manager -->
        <dependency>
            <groupId>io.github.bonigarcia</groupId>
            <artifactId>webdrivermanager</artifactId>
            <version>5.1.0</version>
        </dependency>

        <!-- TestNG Dependency -->
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>7.4.0</version>
            <scope>test</scope>
        </dependency>

    </dependencies>

</project>

6. Create a Test Code

Creating an instance of the Remote WebDriver and passing the selenium endpoint and chrome options defined in it.

To run a Remote WebDriver client, we first need to connect to the RemoteWebDriver. We do this by pointing the URL to the address of the server running our tests. In order to customize our configuration, we set desired capabilities.

Below is an example of instantiating a remote WebDriver object pointing to our remote web server running our tests on Chrome, Firefox, and Edge. We have used @Parameters Annotation to pass the browser names to the tests.

import org.openqa.selenium.By;
import org.openqa.selenium.Capabilities;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.edge.EdgeOptions;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Parameters;
import java.net.MalformedURLException;
import java.net.URL;
import java.time.Duration;

public class BaseTest {


    protected static ThreadLocal<RemoteWebDriver> driver = new ThreadLocal<RemoteWebDriver>();
    public static String remote_url = "http://localhost:4445/";
    public Capabilities capabilities;

    @Parameters({"browser"})
    @BeforeMethod
    public void setDriver(String browser) throws MalformedURLException {
    	
    	System.out.println("Test is running on "+browser);

        if(browser.equals("firefox")) {
            capabilities = new FirefoxOptions();
        } else if (browser.equals("chrome")) {
            capabilities = new ChromeOptions();
        } else if (browser.equals("edge")) {
        	capabilities = new EdgeOptions();
        }

        driver.set(new RemoteWebDriver(new URL(remote_url), capabilities));
        driver.get().get("https://opensource-demo.orangehrmlive.com/");
        driver.get().manage().window().maximize();
        driver.get().manage().timeouts().pageLoadTimeout(Duration.ofSeconds(10));
        WebDriverWait wait = new WebDriverWait(driver.get(), Duration.ofSeconds(10));
        wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//*[@id='divUsername']/span")));
    }

    public WebDriver getDriver() {
        return driver.get();
    }

    @AfterMethod
    public  void closeBrowser() {
        driver.get().quit();
        driver.remove();
    }
}

Selenium4ParallelDemo

This class contains the various tests that need to be executed.

  • In BaseTest class, I created ThreadLocal <>() webdriver (ThreadLocalMap) for thread-safe test execution
  • I got the TestNG parameter (browser) with @Parameter annotation.
  • BaseTest returns browser Capabilities based on browser name.
  • In BaseTest class, the getDriver() method returns the created driver.
  • Selenium4ParallelDemo class extends TestBase class and comprises their test code.

import org.openqa.selenium.By;
import org.testng.annotations.Test;
import static org.testng.Assert.assertEquals;
import static org.testng.Assert.assertTrue;

public class Selenium4ParallelDemo extends BaseTest {

    @Test
    public void validCredentials()  {

        getDriver().findElement(By.xpath("//*[@id='txtUsername']")).sendKeys("Admin");
        getDriver().findElement(By.xpath("//*[@id='txtPassword']")).sendKeys("admin123");
        getDriver().findElement(By.xpath("//*[@id='btnLogin']")).click();
        String newPageText = getDriver().findElement(By.xpath("//*[@id='content']/div/div[1]/h1")).getText();
        System.out.println("newPageText :" + newPageText);
        assertEquals(newPageText,"Dashboard");

    }

    @Test
    public void invalidCredentials() {
    	
        getDriver().findElement(By.xpath("//*[@id='txtUsername']")).sendKeys("1234");
        getDriver().findElement(By.xpath("//*[@id='txtPassword']")).sendKeys("12342");
        getDriver().findElement(By.xpath("//*[@id='btnLogin']")).click();
        String actualErrorMessage = getDriver().findElement(By.xpath("//*[@id='spanMessage']")).getText();
        System.out.println("Actual ErrorMessage :" + actualErrorMessage);
        assertEquals(actualErrorMessage,"Invalid credentials");

    }

    @Test
    public void loginPageHeading() {

        String loginText = getDriver().findElement(By.xpath("//*[@id='logInPanelHeading']")).getText();
        System.out.println("Actual loginText :" + loginText);
        assertEquals(loginText,"LOGIN Panel");

    }

    @Test
    public void forgotPasswordPageHeading()  {

        getDriver().findElement(By.xpath("//*[@id='forgotPasswordLink']/a")).click();
        String forgetPasswordTitle= getDriver().findElement(By.xpath(" //*[@id='content']/div[1]/div[2]/h1")).getText();
        System.out.println("Actual Page Title of Forgot Password Page :" + forgetPasswordTitle);
        assertEquals(forgetPasswordTitle,"Forgot Your Password?");
    }

    @Test
    public void verifyLinkedIn() {

        Boolean linkedInIcon = getDriver().findElement(By.xpath("//*[@id='social-icons']/a[1]/img")).isEnabled();
        System.out.println("Actual linkedIn Text :" + linkedInIcon);
        assertTrue(linkedInIcon);

    }
}

7. Create a testng.xml

It is very easy to create testng.xml in the case of Eclipse. Right-click on the project, and select TestNG -> Convert to TestNG. It will create a basic testng.xml structure. We need to add the parameter name and value to it.

We are planning to run the test on 3 different browsers, so we are passing the name of the browser from this testng.xml using the parameter. This testng.xml will run the test sequentially. There is a minor change needs to be done in testng.xml that will run the tests parallelly.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">

<suite name="Suite">
  <test thread-count="5" name="Chrome Test">
    <parameter name ="browser" value="chrome"/>
    <classes>
      <class name="org.example.Selenium4ParallelDemo"/>
    </classes>
  </test> <!-- Test -->
  
   <test thread-count="5" name="Firefox Test">
    <parameter name ="browser" value="firefox"/>
    <classes>
      <class name="org.example.Selenium4ParallelDemo"/>
    </classes>
  </test> <!-- Test -->
  
   <test thread-count="5" name="Edge Test">
    <parameter name ="browser" value="edge"/>
    <classes>
      <class name="org.example.Selenium4ParallelDemo"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

8. Run the tests from testng.xml

Right-Click on the testng.xml and select Run As ->TestNG Suite.

9. Navigate to the sessions tab on the Selenium Grid UI upon running the command

It would reflect an active session.

10. Review the test execution result

There are 2 ways to see if the tests are getting executed or not. First, we can check in the command line. We can see the logs there as shown below.

The complete test execution result can be found in the console too.

The tests are TestNG ones, so we can also check the Result of Running Suite tab also.

11. TestNG Report Generation

TestNG generates the test reports in the test-output folder.

We are interested in 2 reports – index.html and emailable-report.html.

Index.html

The below image shows that the tests are run sequentially.

Emailable-Report.html

This report is the summary report. It contains the summary of all the tests executed like, the number of tests passed, skipped, retried, and failed, and the execution time taken by each test.

Parallel Testing

Update the testng.xml for parallel testing. Add parallel=”tests” in the XML.

Updated testng.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">

<suite name="Suite" parallel="tests">
  <test thread-count="5" name="Chrome Test">
    <parameter name ="browser" value="chrome"/>
    <classes>
      <class name="org.example.Selenium4ParallelDemo"/>
    </classes>
  </test> <!-- Test -->
  
   <test thread-count="5" name="Firefox Test">
    <parameter name ="browser" value="firefox"/>
    <classes>
      <class name="org.example.Selenium4ParallelDemo"/>
    </classes>
  </test> <!-- Test -->
  
   <test thread-count="5" name="Edge Test">
    <parameter name ="browser" value="edge"/>
    <classes>
      <class name="org.example.Selenium4ParallelDemo"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

The below image shows that the tests were running on multiple browsers at the same time.

The console also shows that tests were started on all three browsers at the same time.

Index.html shows that methods run in chronological order. It can be seen that the setDriver() method for all 3 browsers was the first one to be executed.

Best Practices:

  1. It is advisable to use nodes other than 4444 to run the tests. By using the different port numbers, we prevent the risk that the port is already in use on your system.
  2. It is suggested to use RemoteWebDriver objects, as it is used to set which node (or machine) our test will run against.

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

How to run parameterized Selenium test using JUnit5

HOME

The previous tutorial has shown the various parameterized tests in JUnit5. This tutorial shows how to run a test multiple times with a different set of data. This helps to reduce the duplication of code. This is a very common scenario in any testing. Imagine, we want to test the requirement for a login page that uses a username and password to log in to the application. Username and password must satisfy some conditions like username can be only alphabets and no numeric and special characters. There could be multiple sets of data that can be used to test this requirement.

Pre-Requisite:

  1. Selenium – 3.141.59
  2. Maven
  3. Java 11
  4. JUnit Jupiter Engine – 5.8.2
  5. JUnit Jupiter API- 5.8.2

JUnit5 provides a lot of ways to parameterize a test – @ValueSource, @EnumSource, @MethodSource, @CsvSource, @CsvFileSource, and @ArgumentsSource.

Let us see an example where the test is not parameterized. In the below example, we want to verify the different error messages generated by passing incorrect values to username and password.

This is the base class – Login which contains the test method that uses a different set of test data.

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;
import org.openqa.selenium.support.PageFactory;

public class LoginPage {

    WebDriver driver ;

    @FindBy(name="txtUsername")
    WebElement username;

    @FindBy(name="txtPassword")
    WebElement password;

    @FindBy(id="btnLogin")
    WebElement loginButton;

    @FindBy(id="spanMessage")
    WebElement actualErrorMessage;


    public LoginPage(WebDriver driver) {

        this.driver = driver;

        // This initElements method will create all WebElements
        PageFactory.initElements(driver, this);
    }

    public void setUserName(String strUserName) {
        username.sendKeys(strUserName);
    }

    // Set password in password textbox
    public void setPassword(String strPassword) {
        password.sendKeys(strPassword);
    }

    // Click on login button
    public void clickLogin() {
        loginButton.click();
    }

    // Get the error message
    public String getErrorMessage() {
        return actualErrorMessage.getText();
    }

    public void login(String strUserName, String strPasword) {

        // Fill user name
        this.setUserName(strUserName);

        // Fill password
        this.setPassword(strPasword);

        // Click Login button
        this.clickLogin();
    }

}

The below example shows 4 tests using a common test with 4 different sets of data.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import static org.junit.jupiter.api.Assertions.assertEquals;

public class NonParameterizedLoginTest  {

    WebDriver driver;
    LoginPage login;

    @BeforeEach
    void setUp() {

        WebDriverManager.chromedriver().setup();
        ChromeOptions chromeOptions = new ChromeOptions();
        driver = new ChromeDriver(chromeOptions);
        driver.manage().window().maximize();
        driver.get("https://opensource-demo.orangehrmlive.com/");

    }

    @Test
    void invalidCredentials1() {

        login = new Login(driver);
        login.login("Admin","Admin");
        String actualErrorMessage = login.getErrorMessage();
        assertEquals("Invalid credentials", actualErrorMessage);

    }

    @Test
    void invalidCredentials2() {

        login = new Login(driver);
        login.login("","Admin");
        String actualErrorMessage = login.getErrorMessage();
        assertEquals("Username cannot be empty", actualErrorMessage);

    }

    @Test
    void invalidCredentials3() {

        login = new Login(driver);
        login.login("Admin","");
        String actualErrorMessage = login.getErrorMessage();
        assertEquals("Password cannot be empty", actualErrorMessage);

    }

    @Test
    void invalidCredentials4() {

        login = new Login(driver);
        login.login("","");
        String actualErrorMessage = login.getErrorMessage();
        assertEquals("Username cannot be empty", actualErrorMessage);

    }


    @AfterEach
    void tearDown() {
        if (driver != null) {
            driver.close();
        }
    }
    
}

We can see that the same method is called multiple times. This is a duplication of code. The output of the above program is

Now, we will parametrize the same test. To do so, we need to add a dependency to the POM.xml.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.example</groupId>
    <artifactId>JUnit5Demo</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>11</maven.compiler.source>
        <maven.compiler.target>11</maven.compiler.target>
    </properties>

    <dependencies>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>5.8.2</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.8.2</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>3.141.59</version>
        </dependency>

        <dependency>
            <groupId>io.github.bonigarcia</groupId>
            <artifactId>webdrivermanager</artifactId>
            <version>5.1.0</version>
        </dependency>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-params</artifactId>
            <version>5.8.2</version>
            <scope>test</scope>
        </dependency>

    </dependencies>

    <build>
        <plugins>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M5</version>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M5</version>

                <dependencies>
                    <dependency>
                        <groupId>org.junit.jupiter</groupId>
                        <artifactId>junit-jupiter-engine</artifactId>
                        <version>5.8.2</version>
                    </dependency>
                </dependencies>
            </plugin>
        </plugins>
    </build>

</project>

There are multiple ways to parameterize the test. To start with:

  1. Replace @Test annotation with @ParameterizedTest annotation provided by the JUnit5 framework.
  2. Add parameters to the loginTest() method. In this example, we will add a username and a password parameter.
  3. Add the parameters source. In this example, we will use the @CsvFileSource annotation.

In this example, will retrieve the data from CSV. This CSV file is placed under src/test/resources. Below is the example of the credentials.csv file.

To know all the different types of parameterization methods, please refer to this tutorial. This tutorial will show the 2 most common ways to parameterize tests in JUnit5.

1.@CsvSource

@CsvSource allows us to express argument lists as comma-separated values (i.e., CSV String literals). Each string provided via the value attribute in @CsvSource represents a CSV record and results in one invocation of the parameterized test. An empty, quoted value (”) results in an empty String. This can be seen in the below example.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvFileSource;
import org.junit.jupiter.params.provider.CsvSource;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import static org.junit.jupiter.api.Assertions.assertEquals;

public class ParameterizedLoginPageTest {

    WebDriver driver;

    LoginPage loginPage;

    @BeforeEach
     void setUp() {

        WebDriverManager.chromedriver().setup();
        ChromeOptions chromeOptions = new ChromeOptions();
        driver = new ChromeDriver(chromeOptions);
        driver.manage().window().maximize();
        driver.get("https://opensource-demo.orangehrmlive.com/");

    }

    @ParameterizedTest
    @CsvSource({
            "admin123,admin123,Invalid credentials",
            "'',admin123,Username cannot be empty",
            "Admin,'',Password cannot be empty",
            "'','',Username cannot be empty"
    })

    void invalidCredentials1(String username, String password, String errorMessage) {

        loginPage = new LoginPage(driver);
        loginPage.login(username,password);
        String actualErrorMessage = loginPage.getErrorMessage();
        assertEquals(errorMessage, actualErrorMessage);

    }

    @AfterEach
     void tearDown() {
        driver.close();
    }
}

The output of the above program is

2. @CsvFileSource

@CsvFileSource lets us use comma-separated value (CSV) files from the classpath or the local file system.

We can see in the below example, that we have skipped the first line from the credentials.csv file as it is the heading of the file. invalidCredentials() method got 4 different set of the test data from CSV file using parameterization. JUnit5 ignores the headers via the numLinesToSkip attribute.

In @CsvFileSource, an empty, quoted value (“”) results in an empty String in JUnit5.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvFileSource;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import static org.junit.jupiter.api.Assertions.assertEquals;

public class ParameterizedLoginPageTest {

    WebDriver driver;

    LoginPage loginPage;

    @BeforeEach
     void setUp() {

        WebDriverManager.chromedriver().setup();
        ChromeOptions chromeOptions = new ChromeOptions();
        driver = new ChromeDriver(chromeOptions);
        driver.manage().window().maximize();
        driver.get("https://opensource-demo.orangehrmlive.com/");

    }

    @ParameterizedTest
    @CsvSource({
            "admin123,admin123,Invalid credentials",
            "'',admin123,Username cannot be empty",
            "Admin,'',Password cannot be empty",
            "'','',Username cannot be empty"
    })


    @ParameterizedTest
    @CsvFileSource(files = "src/test/resources/credentials.csv", numLinesToSkip = 1)
    void invalidCredentials(String username, String password, String errorMessage) {

        loginPage = new LoginPage(driver);
        loginPage.login(username,password);
        String actualErrorMessage = loginPage.getErrorMessage();
        assertEquals(errorMessage, actualErrorMessage);

    }


    @AfterEach
     void tearDown() {
        driver.close();
    }
}
     

The result of the above program is

Congratulations!! We have seen how Selenium tests are parameterized in JUnit5. Happy Learning.

Run Selenium Tests in GitLab CI/CD

HOME

This tutorial explains the process to run the Selenium Tests in the GitLab pipeline. This is a very important step towards achieving CI/CD. Ideally, the tests need to run after any change (minor/major) before merging the latest change to the master branch. Suppose there are 100 changes merged to the master branch in a day and it is expected to run the tests every time before deployment. In this case, any QA won’t want to start the tests manually 100 times in a day. Now, what should be done to overcome this problem. Now, adding tests to the GitLab pipeline comes into the picture. We can add a test stage to the pipeline and the tests will run automatically when the pipeline run.

Pre-Requisite:

  1. Selenium
  2. TestNG/JUnit (for Assertions)
  3. Java 11
  4. Maven/ Gradle
  5. GitLab Account

What is GitLab CI/CD Workflow?

Once the proposed changes are built, then push the commits to a feature branch in a remote repository that’s hosted in GitLab. The push triggers the CI/CD pipeline for your project. Then, GitLab CI/CD runs automated scripts (sequentially or in parallel) to build as well as to test the application. After a successful run of the test scripts, GitLab CI/CD deploys your changes automatically to any environment (DEV/QA/UAT/PROD). But if the test stage is failed in the pipeline, then the deployment is stopped.

After the implementation works as expected:

  • Get the code reviewed and approved.
  • Merge the feature branch into the default branch.
    • GitLab CI/CD deploys your changes automatically to a production environment.

To use GitLab CI/CD, we need to keep 2 things in mind:

a) Make sure a runner is available in GitLab to run the jobs. If there is no runner, install GitLab Runner and register a runner for your instance, project, or group.

b) Create a .gitlab-ci.yml file at the root of the repository. This file is where CI/CD jobs are defined.

The Selenium tests run on a headless browser in the pipeline.

What is a headless browser?

A headless browser is like any other browser, but without a Head/GUI (Graphical User Interface).  A headless browser is used to automate the browser without launching the browser. While the tests are running, we could not see the browser, but we can see the test results coming on the console.

To explain, I have created 2 Selenium tests and used TestNG for asserting the tests. The tests will run on a headless Chrome browser. One more thing to keep in mind is that when tests run on a headless Chrome browser, the window screen won’t be full screen. So, the screen needs to be maximized explicitly otherwise some of the tests would fail.

Step 1 – Create a new Maven Project

Step 2- Add the dependencies to the POM.xml

Add the below-mentioned dependencies that need to add to the project to the pom.xml in Maven Project.

 <dependencies>

        <dependency>
            <groupId>io.github.bonigarcia</groupId>
            <artifactId>webdrivermanager</artifactId>
            <version>5.1.0</version>
        </dependency>

        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>3.141.59</version>
        </dependency>

        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>7.5</version>
            <scope>test</scope>
        </dependency>
                   
    </dependencies>
    <build>
     <plugins>
     
     <!--  Compiler Plugin -->
    
    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.8.1</version>
        <configuration>
           <source>11</source>
           <target>11</target>
        </configuration>
    </plugin>
     
      <!--  Plugin used to execute tests -->
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>3.0.0-M5</version>
        <configuration>
          <suiteXmlFiles>
            <suiteXmlFile>testng.xml</suiteXmlFile>
          </suiteXmlFiles>
        </configuration>
      </plugin>
     </plugins>
  </build>
</project>

As explained in one of the previous tutorial, it is needed to add the maven-surefire-plugin to run the TestNG tests through the command line.

Step 3 – Create the Test Code

This is the BaseTest Class where the WebDriver is initialized, headless mode, full screen, and at the end close the WebDriver.

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;

import io.github.bonigarcia.wdm.WebDriverManager;

public class BaseTest {
	
	WebDriver driver;
	
	@BeforeMethod
	public void beforeTests() {
        WebDriverManager.chromedriver().setup();
        ChromeOptions options=new ChromeOptions();
        options.setHeadless(true);
        options.addArguments("window-size=1920,1200");
        driver=new ChromeDriver(options);
        driver.get("https://opensource-demo.orangehrmlive.com/");
    }

	@AfterMethod
    public void afterTests() {
        driver.quit();
    }

}

There are 2 different pages that need to be tested – LoginPage and ForgetPasswordPage

LoginPage contains the tests to log in to the application. After successful login, the application moves to the next webpage – HomePage. You can see that BaseTest class is extended in both the Test classes.

import org.openqa.selenium.By;
import org.testng.Assert;
import org.testng.annotations.Test;

@Test
public class LoginPage extends BaseTest{
	
	@Test
	public void validCredentials() throws InterruptedException {

	        driver.findElement(By.name("txtUsername")).sendKeys("Admin");	        
	        driver.findElement(By.name("txtPassword")).sendKeys("admin123");
	        driver.findElement(By.id("btnLogin")).click();
	         String newPageText = driver.findElement(By.xpath("//*[@id='content']/div/div[1]/h1")).getText();
	        System.out.println("newPageText :" + newPageText);
	        Assert.assertEquals(newPageText,"Dashboard");

	}

}

ForgetPasswordPage

import org.openqa.selenium.By;
import org.testng.Assert;
import org.testng.annotations.Test;

@Test
public class ForgetPasswordPage extends BaseTest{
	
	@Test
	public void getHeading() {
		
		Thread.sleep(1000);
		driver.findElement(By.xpath("//*[@id='forgotPasswordLink']/a")).click();
		String heading = driver.findElement(By.xpath("//*[@id='content']/div[1]/div[2]/h1")).getText();
		System.out.println("Title : "+ heading );
		Assert.assertEquals(heading, "Forgot Your Password?");
	}
	
}

Step 4 – Create testng.xml to run the tests

Now, let’s create a testng.xml to run the TestNG tests. If JUnit is used instead of TestNG, then this step is not needed.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test name="Test">
    <classes>
      <class name="org.example.DockerDemo.LoginPage"/>
      <class name="org.example.DockerDemo.ForgetPasswordPage"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

Step 5 – Run the tests through command line

Now, let us execute the tests through the command line. Go to the place where the pom.xml of the project is placed and use the below command to run the tests. This step makes sure that all the tests are running as expected.

mvn compile test

GitLab Section

Step 6 – Create a blank project in GitLab

To know, how to create a blank new project in GitLab, please refer to this tutorial.

Step 7 – Push the project from local repository to Gitlab Repository

To know, how to push the changes in GitLab, please refer to this tutorial.

Step 8 – Create a .gitlab-ci.yml file in the project in GitLab

There are many ways to create a new file in GitLab. One of the ways is to create a file as shown in the below image.

It is a YAML file where you configure specific instructions for GitLab CI/CD. In the .gitlab-ci.yml, we can define:

  • The scripts you want to run.
  • Other configuration files and templates you want to include.
  • Dependencies and caches.
  • The commands you want to run in sequence and those you want to run in parallel.
  • The location to deploy your application to.
  • Whether you want to run the scripts automatically or trigger any of them manually.
image: markhobson/maven-chrome

stages:
  - test

variables:
  MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"

test:
  stage: test
  allow_failure: true

# Run the tests  
  script:
    - mvn $MAVEN_OPTS clean package
    - mvn compile test

# Store artifacts
  artifacts:
    when: always
    name: "report"
    paths:
    - target/surefire-reports/*
    expire_in: 1 h

Image – markhobson/maven-chrome is used in this test. It is a docker image for Java automated UI tests.

This gitlab-ci.yml has only 1 stage – a test that contains the command to run the tests as well as also create an artifact that contains all the surefire reports which can be saved as Test Evidence.

Step 9 – Run the tests in the GitLab pipeline

Now, when a new change is committed, a pipeline kicks off and it runs all the tests.

Step 10 – Check the status of the pipeline

Once the Status of the pipeline changes to either failed or passed.. that means the tests are already executed.

As you can see the Status is failed here which means that the execution is completed. Let us see the logs of the execution it shows that out of 2 tests, 1 test passed and 1 test failed. This shows that tests are running successfully in the GitLab pipeline.

As I have added an artifact also in the gitalb-ci.yml, which is highlighted in the image. This artifact creates a folder with the name “report” and the reports in this folder come from the path /target/surefire-reports. This artifact gives us the option to download the reports or browse the report. This report will be available for 1 hour only as mentioned in the gitlab-ci.yml.

Step 11 – Download the report

Once, will click on the download button, it will download “report.zip”. Unzip the folder and it looks like something as shown below:

Example of Emailable-Report.html

Example of Index.html

Congratulations. This tutorial has explained the steps to run Selenium tests in GitLab CI/CD. Happy Learning!!

Run TestNG tests from Command Line

HOME

The previous tutorial has explained the Integration of Selenium with TestNG and the tests are executed through either TestNG Suite or testng.xml. This tutorial explains the steps to run the TestNG Tests through the command line.

Pre-Requisite

  1. Selenium
  2. TestNG
  3. Maven
  4. Java 11
  5. Maven Compiler Plugin
  6. Maven Surefire Plugin

Imagine we need to run the TestNG Tests in CI/CD pipelines like Jenkins or GitLab, then we can’t right click and select TestNG Suite or tesng.xml to run the tests. In such situations, the tests can be executed through command line.

We need to add plugins to pom.xml to compile the test code and then run the tests. To know more about Maven Surefire Plugin for TestNG, refer this blog.

 <build>
      <plugins>

         <!--  Compiler Plugin -->
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.8.1</version>
            <configuration>
               <source>11</source>
               <target>11</target>
            </configuration>
         </plugin>
         
         <!--  Plugin used to execute tests -->
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            <configuration>
               <suiteXmlFiles>
                  <suiteXmlFile>testng.xml</suiteXmlFile>
               </suiteXmlFiles>
            </configuration>
         </plugin>
      </plugins>
   </build>
</project>

It is needed to add testng.xml to suiteXmlFile to start the execution of the tests.

<suiteXmlFiles>
       <suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>

Create sample class which has @Test methods. In the example below we have created a class as below:

import static org.testng.Assert.assertTrue;

import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.MatcherAssert.assertThat;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;

public class TestNGRunFromCommandLine {
	
	 WebDriver driver;
	 
	    @BeforeTest
	    public void setUp() {
	        System.setProperty("webdriver.gecko.driver",
	                "C:\\Users\\Vibha\\Software\\geckodriver\\geckodriver.exe");
	 
	        driver = new FirefoxDriver();
	        driver.get("https://opensource-demo.orangehrmlive.com/");
	 
	        driver.manage().window().maximize();
	    }
	 
	    @Test(description = "This test validates title of login functionality", priority = 0)
	    public void verifyLoginPage() {
	 
	        String expectedTitle = driver.findElement(By.xpath("//*[@id='logInPanelHeading']")).getText(); 
	        System.out.println("Title :" + expectedTitle);
	        assertTrue(expectedTitle.equalsIgnoreCase("LOGIN Panel"));
	    }
	 
	    @Test(description = "This test validates  successful login to Home page", priority = 1)
	    public void verifyHomePage() {
	 
	        System.out.println("Username Entered");
	        driver.findElement(By.name("txtUsername")).sendKeys("Admin");
	 
	        System.out.println("Password Entered");
	        driver.findElement(By.name("txtPassword")).sendKeys("admin123");
	         
	        driver.findElement(By.id("btnLogin")).submit();        
 
	        String newPageText = driver.findElement(By.xpath("//*[@id='content']/div/div[1]/h1")).getText();
	        System.out.println("newPageText :" + newPageText);
	        assertThat(newPageText, containsString("Dashboard"));
	    }
	 
	    @AfterTest
	    public void teardown() {
	 
	        driver.quit();
	    }
	}

The below is the testng.xml file, which will execute all the tests that are available under ‘TestNGRunFromCommandLine‘ class.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test name="Test">
    <classes>
      <class name="TestNG_Demo.TestNGRunFromCommandLine"/>
    </classes>
  </test>
</suite>

Below commands are used to execute ‘testng.xml’ file from command line. First, we need to go the place where pom.xml of the project is placed. Then use the mvn compile test to compile the code and execute the TestNG tests.

cd C:\Users\Vibha\Projects\Vibha_Personal\ParallelTestsTestNG
mvn compile test

After executing above command, it should execute the tests that we have specified in testng.xml file. Below is the screen shot after the execution of the tests.

This execution generate various TestNG Reports. We are concerned about emailable-report.html and index.html.

Emailable-Report.html

Emailable report is a type of summary report that one can transfer to other people in the team through any medium. Click on option “emailable-report.html”. Click on option web browser. The output reports in TestNG reporting will looks like below:

Index.html

Index report contains the index-like structure of different parts of the report, such as failed tests, test file, passed test, etc.

Right click on the index.html from the project directory. Select option open with web browser option.

The result will look like this:

Congratulations. This tutorial has explain running the tests of TestNG using Command Line. Happy Learning!!

How to run Edge tests in headless mode in Selenium4

HOME

This tutorial explains the steps to run the Selenium tests for Chrome browser and Firefox in headless mode. We are going to run the tests in Selenium 4.

There are 2 ways to add dependencies to the Selenium project.

Manually add the dependencies to the project

Download Selenium Version from here

Download Microsoft Edge Binary from here

Download the latest version of WebDriverManager

Once the Selenium and WebDriverManager folders are downloaded, unzip the folder. Once the zip file is extracted, reference these jar files in the project. For this, navigate to project properties and click Build Path-> Configure Build Path in Eclipse. Click “Add External Jars“. After clicking on “Add External JARs“, selected all the extracted JARs. The JARs files are present in the project.

Add the below dependencies to pom.xml or build.gradle

Selenium 4

 <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-java -->
<dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>selenium-java</artifactId>
    <version>4.1.2</version>
</dependency>

What is headless browser?

A headless browser is like any other browser, but without a Head/GUI (Graphical User Interface).  A headless browser is used to automate the browser without launching the browser. While the tests are running, we could not see the browser, but we can see the test results coming on the console.

We know that to execute Selenium automation scripts on browsers like edge, we must download the binary files of EDGE driver – msedgedriver. After this, we need to set the path to these binaries in the automation script or add the classpath location. Here, we want to execute Selenium WebDriver automation scripts on the Microsoft Edge browser, then you need first to download msedgedriver.exe and then use the System.setProperty  method to set its path as follows:

		// Set the path of EdgeDriver
		System.setProperty("webdriver.edge.driver",
				"C:\\Users\\Vibha\\Software\\edgedriver_win64\\msedgedriver.exe");

The complete program looks like as shown below:

import org.openqa.selenium.edge.EdgeDriver;
import org.openqa.selenium.edge.EdgeOptions;

public class EdgeHeadlessInSelenium4 {

	public static void main(String[] args) {
		// Set the path of EdgeDriver
		System.setProperty("webdriver.edge.driver",
				"C:\\Users\\Vibha\\Software\\edgedriver_win64\\msedgedriver.exe");

		// Create an object of Edge Options class
		EdgeOptions edgeOptions = new EdgeOptions();

		// pass the argument –headless to Edge Options class.
		edgeOptions.addArguments("--headless");

		// Create an object of WebDriver class and pass the Edge Options object as
		// an argument
		WebDriver driver = new EdgeDriver(edgeOptions);

		System.out.println("Executing Edge Driver in Headless mode..");
		driver.get("https://duckduckgo.com/");

		System.out.println("Title of Page :" + driver.getTitle());
		System.out.println("Page URL  : " + driver.getCurrentUrl());

		// Close the driver
		driver.close();

	}

}

How to run headless Microsoft Edge Tests in Selenium using WebDriverManager?

WebDriverManager Maven Dependency

<!-- https://mvnrepository.com/artifact/io.github.bonigarcia/webdrivermanager -->
<dependency>
    <groupId>io.github.bonigarcia</groupId>
    <artifactId>webdrivermanager</artifactId>
    <version>5.1.0</version>
</dependency>

WebDriverManager has an automated way to download browser executables(exes) or binaries. It supports different browsers like Chrome, Firefox, Microsoft Edge, Internet Explorer, Opera, or PhantomJS.

WebDriverManager.edgedriver().setup: checks for the latest version of the specified WebDriver binary. If the binaries are not present on the machine, then it will download the WebDriver binaries. Next, it instantiates the Selenium WebDriver instance with the EdgeDriver.

import org.openqa.selenium.edge.EdgeDriver;
import org.openqa.selenium.edge.EdgeOptions;

import io.github.bonigarcia.wdm.WebDriverManager;

public class EdgeHeadlessWithWebDriverManagerInSelenium4 {

	public static void main(String[] args) {

		// WebDriverManager downloads Edge browser executables or binaries.
		WebDriverManager.edgedriver().setup();

		// Create an object of Edge Options class
		EdgeOptions edgeOptions = new EdgeOptions();

		// pass the argument –headless to Edge Options class.
		edgeOptions.addArguments("--headless");

		// Create an object of WebDriver class and pass the Edge Options object as
		// an argument
		WebDriver driver = new EdgeDriver(edgeOptions);

		System.out.println("Executing Edge Driver in Headless mode..");

		driver.get("https://www.bing.com/");
		System.out.println("Title of Page :" + driver.getTitle());
		System.out.println("Page URL : " + driver.getCurrentUrl());

		// Close the driver
		driver.close();

	}
}

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

How to disable infobar warning for Chrome tests in Selenium

HOME

This tutorial explains the steps to disable infobar warning generated by Selenium for running tests in Chrome. Selenium tests run on Chrome shows a warning message – Chrome is being controlled by automated test software as shown in the below image.

We want to run the Selenium tests on Chrome, but without the above-shown warning message. This can be achieved by using excludeSwitches.

chromeOptions.setExperimentalOption("excludeSwitches", Arrays.asList("enable-automation"));

The complete program is shown below:

import java.util.Arrays;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;

import io.github.bonigarcia.wdm.WebDriverManager;

public class ChromeDisableInfobars {

	public static void main(String[] args) {

		WebDriverManager.chromedriver().setup();

		// Create an object of Chrome Options class
		ChromeOptions chromeOptions = new ChromeOptions();

		// prevents Chrome from displaying the notification 'Chrome is being controlled
		// by automated software'
		chromeOptions.setExperimentalOption("excludeSwitches", Arrays.asList("enable-automation"));

		// Create an object of WebDriver class and pass the Chrome Options object as an
		// argument
		WebDriver driver = new ChromeDriver(chromeOptions);

		driver.get("https://duckduckgo.com/");

		System.out.println("Title of Page :" + driver.getTitle());

		// close the browser
		driver.quit();

	}
}

The output of the above program is

We are done. Congratulations!! Happy Learning.

How to run Firefox tests in headless mode in Selenium

HOME

This tutorial explains the steps to run the Selenium tests on Firefox browser in headless mode. We are going to run the tests in Selenium. To run the Selenium tests on Chrome browser in headless mode, refer this tutorial.

To start with, we need to add dependencies to the project.

Manually add the dependencies to the project

Download Selenium Version from here

Download Firefox Binary from here – Version 97.0.1

Download the latest version of WebDriverManager (Download this if you want to use WebDriverManager to download browser executables(exes) or binaries automatically, then skip downloading FireFox Binary).

Add the below dependencies to pom.xml or build.gradle.

Selenium 3

<!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-java -->
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>3.141.59</version>
    </dependency>

What is headless browser?

A headless browser is like any other browser, but without a GUI (Graphical User Interface).  A headless browser is used to automate the browser without launching the browser. While the tests are running, we could not see the browser, but we can see the test results coming on the console. This makes the test execution faster than normal execution. This is suitable for parallel testing as UI tests needs a lot of memory and resources.

The path of Gecko Driver (used for Firefox browser) needs to be set up in the Test using System.setProperty().Here, we use the methods setHeadless (true) of the FirfoxOptions class provided by Selenium WebDriver.

The complete program looks like as shown below:

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxOptions;

public class FirefoxOptionsHeadless1 {

	public static void main(String[] args) {

		// Set the path of GeckoDriver
		System.setProperty("webdriver.gecko.driver",
				"C:\\Users\\Vibha\\Software\\geckodriver\\geckodriver.exe");

		// Create an object of Firefox Options class
		FirefoxOptions options = new FirefoxOptions();

		// Set Firefox Headless mode as TRUE
		options.setHeadless(true);

		// Create an object of WebDriver class and pass the Firefox Options object
		// as an argument
		WebDriver driver = new FirefoxDriver(options);

		// Navigate to site url
		driver.get("https://duckduckgo.com/");

		System.out.println("Executing Firefox Driver in Headless mode..");
		System.out.println("Page Title : " + driver.getTitle());
		System.out.println("Page URL  : " + driver.getCurrentUrl());

		// Close the driver
		driver.close();
	}
}

How to run headless Firefox Tests in Selenium using WebDriverManager?

WebDriverManager

<!-- https://mvnrepository.com/artifact/io.github.bonigarcia/webdrivermanager -->
<dependency>
    <groupId>io.github.bonigarcia</groupId>
    <artifactId>webdrivermanager</artifactId>
    <version>5.1.0</version>
</dependency>

WebDriverManager.firefoxdriver().setup(): checks for the latest version of the specified WebDriver binary. If the binaries are not present on the machine, then it will download the WebDriver binaries. In this case, it is not needed to download Firefox binary and set up the path

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxOptions;

import io.github.bonigarcia.wdm.WebDriverManager;

public class FirefoxOptionsHeadless2 {

	public static void main(String[] args) {

		WebDriverManager.firefoxdriver().setup();

		// Create an object of Firefox Options class
		FirefoxOptions options = new FirefoxOptions();

		// Set Firefox Headless mode as TRUE
		options.setHeadless(true);

		// Create an object of Firefox Driver class and pass the Firefox Options object
		// as an argument
		WebDriver driver = new FirefoxDriver(options);

		// Navigate to the url
		driver.get("https://duckduckgo.com/");

		System.out.println("Executing Firefox Driver in Headless mode..");
		System.out.println("Page Title : " + driver.getTitle());
		System.out.println("Page URL  : " + driver.getCurrentUrl());

		// Close the driver
		driver.close();
	}

}

Congratulations!! We have run the tests in headless mode in FireFox.

Parallel testing of DataProviders in TestNG

HOME

The previous tutorial has explained the DataProviders in TestNG. The DataProvider in TestNG is a way to pass the parameters in the test functions. Using DataProvider in TestNG, we can easily inject multiple values into the same test case. It comes inbuilt into TestNG and is popularly used in data-driven frameworks.

 It is an option for the parallel execution of tests in TestNG. 

It is advisable to create 2 classes – one class contains the Test cases and another class defines TestNG parameters – DataProviders.

Let us create a class for the DataProvider method with all the Test Data as shown below:

import org.testng.annotations.DataProvider;

public class DataProviderDemo {	
	
	 @DataProvider(name = "testData", parallel=true)
	 public Object[][] dataProvFunc() {
	       return new Object[][] {           
	    	   {"","","Username cannot be empty"},    	  
	    	   {"","Test","Username cannot be empty"},
	    	   {"$%1234","2345%$","Invalid credentials"}          
	    	 };
	    }
	}

An extra parameter “parallel” is required to initiate parallel execution in TestNG using the data provider.

Below is the test which uses the parameter from the data provider and runs the tests parallelly.

A new ThreadLocal is instantiated for each test class, since it’s in the BeforeClass annotation.

private static final ThreadLocal<WebDriver> WEB_DRIVER_THREAD_LOCAL = new ThreadLocal<WebDriver>();

Below is the complete test code:

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;

public class DataProviderParallelTests {
	
	public WebDriver driver;
	private static final ThreadLocal<WebDriver> WEBDRIVER_THREADLOCAL = new ThreadLocal<WebDriver>();
	
	 @BeforeMethod
    public void setUp(){

        System.setProperty("webdriver.chrome.driver",
                "C:\\Users\\Vibha\\Software\\chromedriver\\chromedriver.exe");
        driver = new ChromeDriver();
        WEBDRIVER_THREADLOCAL.set(driver);
        System.out.println("Before method Thread Id:" + Thread.currentThread().getId());
        
    }
	
	@Test(dataProvider = "testData", dataProviderClass = DataProviderDemo.class)
    public void invalidLoginTest(String username, String password, String errorMessage) throws InterruptedException {
		     
	    driver = WEBDRIVER_THREADLOCAL.get();
	    driver.manage().window().maximize();
        driver.get("https://opensource-demo.orangehrmlive.com/");
     
        Thread.sleep(2000);
        driver.findElement(By.name("txtUsername")).sendKeys(username);
        System.out.println("Username :" + username);
        
        Thread.sleep(2000);
        driver.findElement(By.name("txtPassword")).sendKeys(password);
        System.out.println("password :" + password);
        
        Thread.sleep(2000);
        String expectedError = driver.findElement(By.id("spanMessage")).getText();
        System.out.println("Error Message :" + expectedError);
        Assert.assertTrue(expectedError.contains(errorMessage));

    }
		 
	@AfterMethod
	public void tear_down() {
		 
		 WebDriver driver = WEBDRIVER_THREADLOCAL.get();
		 System.out.println("After method Thread Id:" + Thread.currentThread().getId());
	        if (driver != null) {
	            driver.quit();
	     }
    }	
}

testng.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite" thread-count="2" data-provider-thread-count="2">
  <test name="Test">
    <classes>
      <class name="DataProvider.DataProviderParallelTests"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

In this file, the data-provider-thread-count is set to 2, then two browsers will be opened, and the first two tests will run from the list. 

Run the test script from testng.xml, Right-Click on the XML, and select Run As ->TestNG Suite.

The execution status shown below shows that 2 threads are active at a time, which execute 2 sets of data provider parameters – Thread 14 and Thread 15. Once the tests are finished for Thread 14 and Thread 15, they are closed and a new Thread 15 is again initiated to start the test execution of the 3rd parameter.

TestNG generates multiple test reports under the folder test-output. We are mainly concerned about 2 reports – emailable-report.html and index.html.

Emailable-report.html

Emailable reports are a type of summary report that one can transfer to other people in the team through any medium. 

Index.html

Index report contains the index-like structure of different parts of the report, such as failed tests, test files, passed tests, etc. We can divide this report into two parts. The left part contains the index, and this is the reason it is called an index report, while the right part contains the explored content of that index.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!