Cross Browser is a technique in which a web application tests on different browsers and operating systems. Cross Browser testing, make sure that the site is rendered the same in every browser and operating system, and ensure that the users have a good experience.
Why do we need to perform Cross Browser Testing?
User Experience Consistency: Different browsers can render web content differently. Cross-browser testing ensures that users have a consistent and reliable experience regardless of the browser they choose.
Browser-Specific Issues: Each browser has its quirks, bugs, and features. Different browser is compatible with different operating systems. Cross-browser testing helps identify and address browser-specific issues or inconsistencies that may affect functionality or appearance.
Prerequisite:
Python is installed on the machine
PIP is installed on the machine
PyCharm or another Python Editor is installed on the machine
This tutorial explains the process to run the Selenium Tests on multiple browsers in the GitLab pipeline. This is a very important step towards achieving CI/CD. Ideally, the tests need to run after any change (minor/major) before merging the latest change to the master branch. This makes life of a QA very easy.
Once the proposed changes are built, then push the commits to a feature branch in a remote repository that’s hosted in GitLab. The push triggers the CI/CD pipeline for your project. Then, GitLab CI/CD runs automated scripts (sequentially or in parallel) to build as well as to test the application. After a successful run of the test scripts, GitLab CI/CD deploys your changes automatically to any environment (DEV/QA/UAT/PROD). But if the test stage is failed in the pipeline, then the deployment is stopped.
After the implementation works as expected:
Get the code reviewed and approved.
Merge the feature branch into the default branch.
GitLab CI/CD deploys your changes automatically to a production environment.
To use GitLab CI/CD, we need to keep 2 things in mind:
a) Make sure a runner is available in GitLab to run the jobs. If there is no runner, install GitLab Runner and register a runner for your instance, project, or group.
b) Create a .gitlab-ci.yml file at the root of the repository. This file is where CI/CD jobs are defined.
The Selenium tests run on a headless browser in the pipeline.
How to check if GitLab Runner is configured?
Go to the project created in the GitLab. Go to the left side and click on Settings. Go to CI/CD option and a CI/CD Settings page open. Scroll down and see the Runners. By default, Shared runner will be selected for any new project.
What is a headless browser?
A headless browser is like any other browser, but without a Head/GUI (Graphical User Interface). A headless browser is used to automate the browser without launching the browser. While the tests are running, we could not see the browser, but we can see the test results coming on the console.
As explained in one of the previous tutorial, it is needed to add the maven-surefire-plugin to run the TestNG tests through the command line.
Step 3 – Create the Test Code
This is the BaseTest Class where the WebDriver is initialized, headless mode, full screen, and at the end close the WebDriver.
package com.example.tests;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.edge.EdgeOptions;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Parameters;
import java.net.URL;
import java.time.Duration;
public class BaseTests {
protected static ThreadLocal<RemoteWebDriver> driver = new ThreadLocal<RemoteWebDriver>();
public static String remote_url = "http://selenium-hub:4444";
public final static int TIMEOUT = 5;
@BeforeMethod
@Parameters("browser")
public void setUp(String browser) throws Exception {
if(browser.equalsIgnoreCase("chrome")) {
ChromeOptions options = new ChromeOptions();
options.addArguments("--start-maximized");
options.addArguments("--headless=new");
options.addArguments("--remote-allow-origins=*");
driver.set(new RemoteWebDriver(new URL(remote_url), options));
System.out.println("Browser Started :"+ browser);
} else if (browser.equalsIgnoreCase("firefox")) {
FirefoxOptions options = new FirefoxOptions();
options.addArguments("--start-maximized");
options.addArguments("-headless");
driver.set(new RemoteWebDriver(new URL(remote_url), options));
System.out.println("Browser Started :"+ browser);
} else if (browser.equalsIgnoreCase("edge")) {
EdgeOptions options = new EdgeOptions();
options.addArguments("--start-maximized");
options.addArguments("--headless=new");
driver.set(new RemoteWebDriver(new URL(remote_url), options));
System.out.println("Browser Started :"+ browser);
} else {
throw new Exception ("Browser is not correct");
}
driver.get().get("https://opensource-demo.orangehrmlive.com/");
driver.get().manage().timeouts().implicitlyWait(Duration.ofSeconds(TIMEOUT));
}
public WebDriver getDriver() {
return driver.get();
}
@AfterMethod
public void closeBrowser() {
driver.get().quit();
driver.remove();
}
}
There is a Login pages that need to be tested.
LoginPage contains the tests to log in to the application. After successful login, the application moves to the next webpage – HomePage. You can see that BaseTest class is extended here.
package com.example.tests;
import org.openqa.selenium.By;
import org.testng.annotations.Test;
import static org.testng.Assert.assertEquals;
public class LoginPageTests extends BaseTests {
By userName = By.name("username");
By passWord = By.name("password");
By loginBtn = By.xpath("//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/form/div[3]/button");
By errorMessage = By.xpath("//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/div/div[1]/div[1]/p");
By blankUsername = By.xpath("//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/form/div[1]/div/span");
By dashboardPage = By.xpath("//*[@id='app']/div[1]/div[1]/header/div[1]/div[1]/span/h6");
@Test
public void invalidCredentials() {
getDriver().findElement(userName).sendKeys("1234");
getDriver().findElement(passWord).sendKeys("12342");
getDriver().findElement(loginBtn).click();
String actualErrorMessage = getDriver().findElement(errorMessage).getText();
System.out.println("Actual ErrorMessage :" + actualErrorMessage);
assertEquals(actualErrorMessage,"Invalid credentials");
}
@Test
public void blankUsername() {
getDriver().findElement(userName).sendKeys("");
getDriver().findElement(passWord).sendKeys("12342");
getDriver().findElement(loginBtn).click();
String actualErrorMessage = getDriver().findElement(blankUsername).getText();
System.out.println("Actual ErrorMessage :" + actualErrorMessage);
assertEquals(actualErrorMessage,"Required");
}
@Test
public void successfulLogin() {
getDriver().findElement(userName).sendKeys("Admin");
getDriver().findElement(passWord).sendKeys("admin123");
getDriver().findElement(loginBtn).click();
String actualMessage = getDriver().findElement(dashboardPage).getText();
System.out.println("Message :" + actualMessage);
assertEquals(actualMessage,"Dashboard");
}
}
Step 4 – Create testng.xml to run the tests
Now, let’s create a testng.xml to run the TestNG tests. It is very easy to create testng.xml in the case of Eclipse. Right-click on the project, and select TestNG -> Convert to TestNG. It will create a basic testng.xml structure. In case of IntelliJ, create a new file with the name of testng.xml and copy the code from here.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="tests" thread-count="3">
<test name="Chrome Test">
<parameter name="browser" value="chrome"></parameter>
<classes>
<class name="com.example.tests.LoginPageTests"/>
</classes>
</test> <!-- Test -->
<test name="Firefox Test">
<parameter name="browser" value="firefox"></parameter>
<classes>
<class name="com.example.tests.LoginPageTests"/>
</classes>
</test> <!-- Test -->
<test name="Edge Test">
<parameter name="browser" value="edge"></parameter>
<classes>
<class name="com.example.tests.LoginPageTests"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->
version: 3. It is the latest version of the docker-compose files.
services(containers): This contains the list of the images and their configurations.
image: It defines which image will be used to spin up container.
ports: Published ports with host:container format.
container_name: You can give name to your containers.
depends_on: This defines the required dependency before spinning up the container. In our docker-compose.yml file, containers Chrome and Firefox are dependent upon container hub to spin up.
SE_NODE_MAX_INSTANCES: This defines how many instances of same version of browser can run over the Remote System.
SE_NODE_MAX_SESSIONS: This defines maximum number of concurrent sessions that will be allowed.
Step 6 – Create a .gitlab-ci.yml
stages:
- test
variables:
SELENIUM_SERVER_NAME: selenium-hub
SELENIUM_SERVER_URL: http://${SELENIUM_SERVER_NAME}:4444
DOCKER_HOST: tcp://docker:2375
services:
- docker:20.10.16-dind
test:
stage: test
image: docker/compose
before_script:
- docker-compose up -d selenium-hub chrome edge firefox
- sleep 10
- docker-compose run ping curl ${SELENIUM_SERVER_URL}/status
script:
- docker-compose run tests mvn clean test
artifacts:
when: always
name: "report"
paths:
- target/surefire-reports/**
expire_in: 7 days
GitLab Section
Step 7 – Create a blank project in GitLab
To know, how to create a blank new project in GitLab, please refer tothis tutorial.
Step 8 – Push the project from local repository to Gitlab Repository
To know, how to push the changes in GitLab, please refer to this tutorial.
Step 9 – Run the tests in the GitLab pipeline
Now, when a new change is committed, a pipeline kicks off and it runs all the tests.
Step 10 – Check the status of the pipeline
Once the Status of the pipeline changes to either failed or passed.. that means the tests are already executed.
As you can see the Status is failed here which means that the execution is completed. Let us see the logs of the execution it shows that out of 9 tests, all 9 are passed. This shows that tests ran successfully in the GitLab pipeline.
As I have added an artifact also in the gitalb-ci.yml, which is highlighted in the image. This artifact creates a folder with the name “report” and the reports in this folder come from the path /target/surefire-reports. This artifact gives us the option to download the reports or browse the report. This report will be available for 7 days only as mentioned in the gitlab-ci.yml.
Step 11 – Download the report
Once, will click on the download button, it will download “report.zip”. Unzip the folder and it looks like something as shown below:
Example of Emailable-Report.html
Example of Index.html
Congratulations. This tutorial has explained the steps to run Selenium tests in GitLab CI/CD. Happy Learning!!