vibssingh

Run Cross Browser Tests in GitLab CI/CD

Last Updated on

HOME

This tutorial explains the process to run the Selenium Tests on multiple browsers in the GitLab pipeline. This is a very important step towards achieving CI/CD. Ideally, the tests need to run after any change (minor/major) before merging the latest change to the master branch. This makes life of a QA very easy.

Table of Contents

  1. Prerequisite
  2. What is GitLab CI/CD Workflow?
    1. How to check if GitLab Runner is configured?
  3. What is a headless browser?
  4. Project Structure
  5. Implementation Steps
    1. Create a new Maven Project
    2. Add the dependencies to the POM.xml
    3. Create the Test Code
    4. Create testng.xml to run the tests
    5. Create docker-compose.yml file
    6. Create a .gitlab-ci.yml
  6. GitLab Section
    1. Create a blank project in GitLab
    2. Push the project from local repository to Gitlab Repository
    3. Run the tests in the GitLab pipeline
    4. Check the status of the pipeline
    5. Download the report
      1. Example of Emailable-Report.html
      2. Example of Index.html

Prerequisite

  1. Selenium
  2. TestNG (for Assertions)
  3. Java 11
  4. Maven/ Gradle
  5. GitLab Account

What is GitLab CI/CD Workflow?

Once the proposed changes are built, then push the commits to a feature branch in a remote repository that’s hosted in GitLab. The push triggers the CI/CD pipeline for your project. Then, GitLab CI/CD runs automated scripts (sequentially or in parallel) to build as well as to test the application. After a successful run of the test scripts, GitLab CI/CD deploys your changes automatically to any environment (DEV/QA/UAT/PROD). But if the test stage is failed in the pipeline, then the deployment is stopped.

After the implementation works as expected:

  • Get the code reviewed and approved.
  • Merge the feature branch into the default branch.
    • GitLab CI/CD deploys your changes automatically to a production environment.

To use GitLab CI/CD, we need to keep 2 things in mind:

a) Make sure a runner is available in GitLab to run the jobs. If there is no runner, install GitLab Runner and register a runner for your instance, project, or group.

b) Create a .gitlab-ci.yml file at the root of the repository. This file is where CI/CD jobs are defined.

The Selenium tests run on a headless browser in the pipeline.

How to check if GitLab Runner is configured?

Go to the project created in the GitLab. Go to the left side and click on Settings. Go to CI/CD option and a CI/CD Settings page open. Scroll down and see the Runners. By default, Shared runner will be selected for any new project.

What is a headless browser?

A headless browser is like any other browser, but without a Head/GUI (Graphical User Interface).  A headless browser is used to automate the browser without launching the browser. While the tests are running, we could not see the browser, but we can see the test results coming on the console.

Project Structure

Implementation Steps

Step 1 – Create a new Maven Project

Click here to know How to create a Maven project.

Below is the Maven project structure. Here,

Group Id – com.example
Artifact Id – CrossBrowser_SeleniumGrid4
Version – 0.0.1-SNAPSHOT
Package – com. example

Step 2- Add the dependencies to the POM.xml

Add the below-mentioned dependencies that need to add to the project to the pom.xml in Maven Project.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.example</groupId>
  <artifactId>CrossBrowser_GitLab</artifactId>
  <version>1.0-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>CrossBrowser_GitLab</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <selenium.version>4.11.0</selenium.version>
    <testng.version>7.8.0</testng.version>
    <maven.compiler.plugin.version>3.10.1</maven.compiler.plugin.version>
    <maven.surefire.plugin.version>3.0.0-M7</maven.surefire.plugin.version>
    <maven.compiler.source.version>11</maven.compiler.source.version>
    <maven.compiler.target.version>11</maven.compiler.target.version>
  </properties>

  <dependencies>

    <!-- Selenium 4 Dependency -->
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>${selenium.version}</version>
    </dependency>

    <!-- TestNG Dependency -->
    <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>${testng.version}</version>
      <scope>test</scope>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>${maven.compiler.plugin.version}</version>
        <configuration>
          <source>${maven.compiler.source.version}</source>
          <target>${maven.compiler.target.version}</target>
        </configuration>
      </plugin>

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>${maven.surefire.plugin.version}</version>
        <configuration>
          <suiteXmlFiles>
            <suiteXmlFile>testng.xml</suiteXmlFile>
          </suiteXmlFiles>
        </configuration>
      </plugin>

    </plugins>
  </build>
</project>

As explained in one of the previous tutorial, it is needed to add the maven-surefire-plugin to run the TestNG tests through the command line.

Step 3 – Create the Test Code

This is the BaseTest Class where the WebDriver is initialized, headless mode, full screen, and at the end close the WebDriver.

package com.example.tests;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.edge.EdgeOptions;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Parameters;

import java.net.URL;
import java.time.Duration;

public class BaseTests {
    protected static ThreadLocal<RemoteWebDriver> driver = new ThreadLocal<RemoteWebDriver>();
    public static String remote_url = "http://selenium-hub:4444";
    public final static int TIMEOUT = 5;

    @BeforeMethod
    @Parameters("browser")
    public void setUp(String browser) throws Exception {
        if(browser.equalsIgnoreCase("chrome")) {

            ChromeOptions options = new ChromeOptions();
            options.addArguments("--start-maximized");
            options.addArguments("--headless=new");
            options.addArguments("--remote-allow-origins=*");
            driver.set(new RemoteWebDriver(new URL(remote_url), options));
            System.out.println("Browser Started :"+ browser);


        } else if (browser.equalsIgnoreCase("firefox")) {
            FirefoxOptions options = new FirefoxOptions();
            options.addArguments("--start-maximized");
            options.addArguments("-headless");
            driver.set(new RemoteWebDriver(new URL(remote_url), options));
            System.out.println("Browser Started :"+ browser);


        } else if (browser.equalsIgnoreCase("edge")) {
            EdgeOptions options = new EdgeOptions();
            options.addArguments("--start-maximized");
            options.addArguments("--headless=new");
            driver.set(new RemoteWebDriver(new URL(remote_url), options));
            System.out.println("Browser Started :"+ browser);


        } else {
            throw new Exception ("Browser is not correct");
        }

        driver.get().get("https://opensource-demo.orangehrmlive.com/");
        driver.get().manage().timeouts().implicitlyWait(Duration.ofSeconds(TIMEOUT));
    }


    public WebDriver getDriver() {
        return driver.get();
    }

    @AfterMethod
    public  void closeBrowser() {
        driver.get().quit();
        driver.remove();
    }

}

There is a Login pages that need to be tested.

LoginPage contains the tests to log in to the application. After successful login, the application moves to the next webpage – HomePage. You can see that BaseTest class is extended here.

package com.example.tests;

import org.openqa.selenium.By;
import org.testng.annotations.Test;

import static org.testng.Assert.assertEquals;

public class LoginPageTests extends BaseTests {

    By userName = By.name("username");
    By passWord = By.name("password");

    By loginBtn = By.xpath("//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/form/div[3]/button");

    By errorMessage = By.xpath("//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/div/div[1]/div[1]/p");

    By blankUsername = By.xpath("//*[@id='app']/div[1]/div/div[1]/div/div[2]/div[2]/form/div[1]/div/span");

    By dashboardPage = By.xpath("//*[@id='app']/div[1]/div[1]/header/div[1]/div[1]/span/h6");

    @Test
    public void invalidCredentials()  {

        getDriver().findElement(userName).sendKeys("1234");
        getDriver().findElement(passWord).sendKeys("12342");
        getDriver().findElement(loginBtn).click();
        String actualErrorMessage = getDriver().findElement(errorMessage).getText();
        System.out.println("Actual ErrorMessage :" + actualErrorMessage);
        assertEquals(actualErrorMessage,"Invalid credentials");

    }

    @Test
    public void blankUsername()  {

        getDriver().findElement(userName).sendKeys("");
        getDriver().findElement(passWord).sendKeys("12342");
        getDriver().findElement(loginBtn).click();
        String actualErrorMessage = getDriver().findElement(blankUsername).getText();
        System.out.println("Actual ErrorMessage :" + actualErrorMessage);
        assertEquals(actualErrorMessage,"Required");

    }

    @Test
    public void successfulLogin()  {

        getDriver().findElement(userName).sendKeys("Admin");
        getDriver().findElement(passWord).sendKeys("admin123");
        getDriver().findElement(loginBtn).click();
        String actualMessage = getDriver().findElement(dashboardPage).getText();
        System.out.println("Message :" + actualMessage);
        assertEquals(actualMessage,"Dashboard");

    }
}

Step 4 – Create testng.xml to run the tests

Now, let’s create a testng.xml to run the TestNG tests. It is very easy to create testng.xml in the case of Eclipse. Right-click on the project, and select TestNG -> Convert to TestNG. It will create a basic testng.xml structure. In case of IntelliJ, create a new file with the name of testng.xml and copy the code from here.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="tests" thread-count="3">
    <test name="Chrome Test">
        <parameter name="browser" value="chrome"></parameter>
        <classes>
            <class name="com.example.tests.LoginPageTests"/>
        </classes>
    </test> <!-- Test -->
    <test name="Firefox Test">
        <parameter name="browser" value="firefox"></parameter>
        <classes>
            <class name="com.example.tests.LoginPageTests"/>
        </classes>
    </test> <!-- Test -->
    <test name="Edge Test">
        <parameter name="browser" value="edge"></parameter>
        <classes>
            <class name="com.example.tests.LoginPageTests"/>
        </classes>
    </test> <!-- Test -->
</suite> <!-- Suite -->

Step 5 – Create docker-compose.yml file

version: "3"
services:
  chrome:
    image: selenium/node-chrome:4.11.0-20230801
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=${SELENIUM_SERVER_NAME}
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443

  firefox:
    image: selenium/node-firefox:4.11.0-20230801
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=${SELENIUM_SERVER_NAME}
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443

  edge:
    image: selenium/node-edge:4.11.0-20230801
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=${SELENIUM_SERVER_NAME}
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443

  selenium-hub:
    image: selenium/hub:4.11.0-20230801
    container_name: ${SELENIUM_SERVER_NAME}
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"

  ping:
    image: alpine/curl

  tests:
    image: maven:3.6.3-jdk-11
    working_dir: /app
    volumes:
      - ${CI_PROJECT_DIR}:/app
    environment:
      ENVIRONMENT: remote
      SELENIUM_SERVER_URL: ${SELENIUM_SERVER_URL}
  • version: 3. It is the latest version of the docker-compose files.
  • services(containers): This contains the list of the images and their configurations.
  • image: It defines which image will be used to spin up container.
  • ports: Published ports with host:container format.
  • container_name: You can give name to your containers.
  • depends_on: This defines the required dependency before spinning up the container. In our docker-compose.yml file, containers Chrome and Firefox are dependent upon container hub to spin up.
  • SE_NODE_MAX_INSTANCES: This defines how many instances of same version of browser can run over the Remote System.
  • SE_NODE_MAX_SESSIONS: This defines maximum number of concurrent sessions that will be allowed.

Step 6 – Create a .gitlab-ci.yml

stages:
  - test

variables:
  SELENIUM_SERVER_NAME: selenium-hub
  SELENIUM_SERVER_URL: http://${SELENIUM_SERVER_NAME}:4444
  DOCKER_HOST: tcp://docker:2375

services:
  - docker:20.10.16-dind

test:
  stage: test
  image: docker/compose
  before_script:
    - docker-compose up -d selenium-hub chrome edge firefox
    - sleep 10
    - docker-compose run ping curl ${SELENIUM_SERVER_URL}/status
  script:
    - docker-compose run tests mvn clean test

  artifacts:
    when: always
    name: "report"
    paths:
      - target/surefire-reports/**
    expire_in: 7 days

GitLab Section

Step 7 – Create a blank project in GitLab

To know, how to create a blank new project in GitLab, please refer to this tutorial.

Step 8 – Push the project from local repository to Gitlab Repository

To know, how to push the changes in GitLab, please refer to this tutorial.

Step 9 – Run the tests in the GitLab pipeline

Now, when a new change is committed, a pipeline kicks off and it runs all the tests.

Step 10 – Check the status of the pipeline

Once the Status of the pipeline changes to either failed or passed.. that means the tests are already executed.

As you can see the Status is failed here which means that the execution is completed. Let us see the logs of the execution it shows that out of 9 tests, all 9 are passed. This shows that tests ran successfully in the GitLab pipeline.

As I have added an artifact also in the gitalb-ci.yml, which is highlighted in the image. This artifact creates a folder with the name “report” and the reports in this folder come from the path /target/surefire-reports. This artifact gives us the option to download the reports or browse the report. This report will be available for 7 days only as mentioned in the gitlab-ci.yml.

Step 11 – Download the report

Once, will click on the download button, it will download “report.zip”. Unzip the folder and it looks like something as shown below:

Example of Emailable-Report.html
Example of Index.html

Congratulations. This tutorial has explained the steps to run Selenium tests in GitLab CI/CD. Happy Learning!!

Posted in GitLabTagged , , , , , , , , Leave a comment

Leave a comment