GitLab automatically enables CI/CD pipelines for new projects. It’s just a matter of adding a new configuration file called .gitlab-ci.yml to your code repository with instructions for GitLab on what to run. So simply create the following basic workflow in your main repository directory and commit it:
The Serenity tests run on a headless browser in the pipeline.
What is a headless browser?
A headless browser is a web browser that operates without a graphical user interface (GUI). It is typically used for automated testing, web scraping, and server-side rendering of web pages. While traditional web browsers like Chrome, Firefox, or Safari have a visible interface for users to interact with, headless browsers work in the background and don’t display the web content visually.
Why do we use Headless browser for executing tests in CI/CD pipeline?
Headless browsers provide a consistent and controlled environment for running tests. They eliminate the variability introduced by different operating systems, browser versions, or screen resolutions, ensuring that tests produce consistent and reliable results across different environments.
Headless browsers can often execute tasks faster than their graphical counterparts. They don’t need to render and display web content, which can significantly reduce the execution time for automated tests or other web-related tasks, contributing to faster feedback in the CI/CD pipeline.
In the below example, our tests are in headless mode.
WebDriverManager.chromedriver().setup();
ChromeOptions ops = new ChromeOptions().setHeadless(true);
ops.addArguments("--remote-allow-origins=*");
driver = new ChromeDriver(ops);
To know, how to create a blank new project in GitLab, please refer tothis tutorial.
Step 2 – Push the project from local repository to Gitlab Repository
To know, how to push the changes in GitLab, please refer to this tutorial.
Step 3 – Create a .gitlab-ci.yml file in the project in GitLab
There are many ways to create a new file in GitLab. One of the ways is to create a file as shown in the below image.
It is a YAML file where you configure specific instructions for GitLab CI/CD. In the .gitlab-ci.yml, we can define:
The scripts you want to run.
Other configuration files and templates you want to include.
Dependencies and caches.
The commands you want to run in sequence and those you want to run in parallel.
The location to deploy your application to.
Whether you want to run the scripts automatically or trigger any of them manually.
Below is a sample example to run the Cucumber tests in the GitLab pipeline.
image: markhobson/maven-chrome
stages:
- test
variables:
MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
test:
stage: test
allow_failure: true
# Run the tests
script:
- echo "Executing BDD scenarios with maven"
- mvn clean test
# Store artifacts
artifacts:
when: always
name: "Cucumber Report"
paths:
- target/*
expire_in: 24 h
Image – markhobson/maven-chrome is used in this test. It is a docker image for Java automated UI tests.
Pipeline configuration begins with jobs. Jobs are the most fundamental element of a .gitlab-ci.yml file.
Jobs are:
Defined with constraints stating under what conditions they should be executed.
Top-level elements with an arbitrary name must contain at least the script clause.
Not limited in how many can be defined.
Jobs can output an archive of files and directories. This output is known as a job artifact. The expire_in keyword determines how long GitLab keeps the job artifacts. Here, it shows 24 hrs to retain the artifacts.
Step 4 – Run the tests in the GitLab pipeline
Now, when a new change is committed, a pipeline kicks off and it runs all the tests.
Step 5 – Check the status of the pipeline
Once the Status of the pipeline changes to either failed or passed.. that means the tests are already executed.
As you can see the Status is passed, it’s green colour. This means all the tests present in the test suite are executed and passed. If any test fails in the test suite, the final execution status will be brown. The reason for the brown colour is we have mentioned allow_failure: true.
Below shows the execution status report in the GitLab pipeline.
As I have added an artifact also in the .gitalb-ci.yml, which is highlighted in the image. This artifact creates a folder with the name “Cucumber_Report”and the reports in this folder come from the path /target/site. This artifact gives us the option to download the reports or browse the report. This report will be available for 24 hours only as mentioned in the gitlab-ci.yml.
Step 5 – Download the report
Once, will click on the download button, it will download “Cucumber_Report.zip”. Unzip the folder and it looks like something as shown below:
Example of Overview Features
Example of Overview Tags
Example of Overview Steps
Congratulations. This tutorial has explained the steps to run Cucumber tests in GitLab CI/CD. Happy Learning!!
GitLab automatically enables CI/CD pipelines for new projects. It’s just a matter of adding a new configuration file called .gitlab-ci.yml to your code repository with instructions for GitLab on what to run. So simply create the following basic workflow in your main repository directory and commit it.
Step 3 – Create a .gitlab-ci.yml file in the project in GitLab
There are many ways to create a new file in GitLab. One of the ways is to create a file as shown in the below image.
It is a YAML file where you configure specific instructions for GitLab CI/CD. In the.gitlab-ci.yml, we can define:
The scripts you want to run.
Other configuration files and templates you want to include.
Dependencies and caches.
The commands you want to run in sequence and those you want to run in parallel.
The location to deploy your application to.
Whether you want to run the scripts automatically or trigger any of them manually.
Below is a sample example to run the Gradle tests in the GitLab pipeline.
image : gradle:7.1.0-jdk11
stages:
- test
variables:
GRADLE_OPTS: "-Dorg.gradle.daemon=false"
test:
stage: test
allow_failure: true
# Run the tests
script:
- echo "Executing BDD scenarios with Gradle"
- gradle clean test
- gradle reports
# Store artifacts
artifacts:
when: always
name: "Gradle Report"
paths:
- lib/target/site/serenity/*
expire_in: 24 h
Image – gradle:7.1.0-jdk11 is used in this test. It is a docker image for Gradle with Java 11 installed in it.
Pipeline configuration begins with jobs. Jobs are the most fundamental element of a .gitlab-ci.yml file.
Jobs are:
Defined with constraints stating under what conditions they should be executed.
Top-level elements with an arbitrary name and must contain at least the script clause.
Not limited in how many can be defined.
Jobs can output an archive of files and directories. This output is known as a job artifact. The expire_in keyword determines how long GitLab keeps the job artifacts. Here, it shows 24 hrs to retain the artifacts.
Step 4 – View GitLab Pipeline
Now, when a new change is committed, a pipeline kicks off and it runs all the tests. To view the pipeline, go to the left panel and click on the Build option. There are a number of sub-options in the Build option, click on the Pipelines.
Step 5 – Run the tests in the GitLab pipeline
The below image shows that the tests are running in the GitLab pipeline.
Step 6 – Check the status of the pipeline
Once the Status of the pipeline changes to either failed or passed, that means the tests are already executed.
As you can see, the Status is passed, its green colour. This means all the tests present in the test suite are executed and passed. If any test fails in the test suite, the final execution status will be brown. The reason for the brown colour is we have mentioned allow_failure: true.
Below shows the execution status report in the GitLab pipeline.
As I have added an artifact also in the .gitalb-ci.yml, which is highlighted in the image. This artifact creates a folder with the name “Serenity_Report”and the reports in this folder come from the path /target/site. This artifact gives us the option to download the reports or browse the report. This report will be available for 24 hours only as mentioned in the gitlab-ci.yml.
Step 7 – Download the report
Once, will click on the download button, it will download “Gradle_Report.zip”. Unzip the folder and it looks like something as shown below:
Example of Index.html
Example of Serenity Summary Report
Congratulations. This tutorial has explained the steps to run Serenity tests in GitLab CI/CD. Happy Learning!!
GitLab automatically enables CI/CD pipelines for new projects. It’s just a matter of adding a new configuration file called .gitlab-ci.yml to your code repository with instructions for GitLab on what to run. So simply create the following basic workflow in your main repository directory and commit it:
By default, GitLab will have CI/CD pipelines and Auto DevOps enabled for all projects. What this means is that, when you push code to the repository, GitLab will automatically trigger the pipeline.
Step 3 – Create a .gitlab-ci.yml file in the project in GitLab
There are many ways to create a new file in GitLab. One of the ways is to create a file as shown in the below image. I have already created .gitlab-ci.yml in the project, which can be seen in the image.
It is a YAML file where you configure specific instructions for GitLab CI/CD. In the.gitlab-ci.yml, we can define:
The scripts you want to run.
Other configuration files and templates you want to include.
Dependencies and caches.
The commands you want to run in sequence and those you want to run in parallel.
The location to deploy your application to.
Whether you want to run the scripts automatically or trigger any of them manually.
Below is a sample example to run the SpringBoot project (Maven) in the GitLab pipeline.
image: maven:3.8.5-openjdk-17
stages:
- test
variables:
MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
test:
stage: test
allow_failure: true
# Run the tests
script:
- echo "Executing SpringBoot scenarios with maven"
- mvn clean test site
# Store artifacts
artifacts:
when: always
name: "SpringBoot Report"
paths:
- target/site/*
expire_in: 24 h
Image – maven:3.8.5-openjdk-17 is used in this test. It is a docker image for Maven and have Java 17 installed in it.
Pipeline configuration begins with jobs. Jobs are the most fundamental element of a .gitlab-ci.yml file.
Jobs are:
Defined with constraints stating under what conditions they should be executed.
Top-level elements with an arbitrary name and must contain at least the script clause.
Not limited in how many can be defined.
Jobs can output an archive of files and directories. This output is known as a job artifact. The expire_in keyword determines how long GitLab keeps the job artifacts. Here, it shows 24 hrs to retain the artifacts.
Step 4 – View GitLab Pipeline
Now, when a new change is committed, a pipeline kicks off and it runs all the tests. To view the pipeline, go to the left panel and click on the Build option. There are a number of sub-options in the Build option, click on the Pipelines.
Step 5 – Run the tests in the GitLab pipeline
The below image shows that the tests are running in the GitLab pipeline.
Step 6 – Manually run the tests in the GitLab pipeline
Whenever there is a new change committed, the pipeline runs automatically. Imagine the scenario when we need to run the pipeline without any new change. This can be done by clicking on the button “Run pipeline”.
Step 7 – Check the status of the pipeline
Once the Status of the pipeline changes to either failed or passed, that means the tests are already executed.
As you can see, the Status is passed, its green colour. This means all the tests present in the test suite are executed and passed. If any test fails in the test suite, the final execution status will be brown. The reason for the brown colour is we have mentioned allow_failure: true.
Below shows the execution status report in the GitLab pipeline.
As I have added an artifact also in the .gitalb-ci.yml, which is highlighted in the image. This artifact creates a folder with the name “Serenity_Report”and the reports in this folder come from the path /target/site. This artifact gives us the option to download the reports or browse the report. This report will be available for 24 hours only as mentioned in the gitlab-ci.yml.
Step 8 – Download the report
Once, will click on the download button, it will download “SpringBoot_Report.zip”. Unzip the folder and it looks like something as shown below:
Example of SureFire-Report.html
Example of Project Summary Report
Congratulations. This tutorial has explained the steps to run Serenity tests in GitLab CI/CD. Happy Learning!!
GitLab automatically enables CI/CD pipelines for new projects. It’s just a matter of adding a new configuration file called .gitlab-ci.yml to your code repository with instructions for GitLab on what to run. So simply create the following basic workflow in your main repository directory and commit it:
The Serenity tests run on a headless browser in the pipeline.
What is a headless browser?
A headless browser is like any other browser but without a Head/GUI (Graphical User Interface). A headless browser is used to automate the browser without launching the browser. While the tests are running, we could not see the browser, but we can see the test results coming on the console. The tests can run in the GitLab pipeline if the tests run in the headless mode.
To know, how to create a blank new project in GitLab, please refer tothis tutorial.
Step 2 – Push the project from the local repository to GitLab Repository
To know, how to push the changes in GitLab, please refer to this tutorial.
Step 3 – Create a .gitlab-ci.yml file in the project in GitLab
There are many ways to create a new file in GitLab. One of the ways is to create a file as shown in the below image.
It is a YAML file where you configure specific instructions for GitLab CI/CD. In the .gitlab-ci.yml, we can define:
The scripts you want to run.
Other configuration files and templates you want to include.
Dependencies and caches.
The commands you want to run in sequence and those you want to run in parallel.
The location to deploy your application to.
Whether you want to run the scripts automatically or trigger any of them manually.
Below is a sample example to run the Serenity tests in the GitLab pipeline.
image: markhobson/maven-chrome
stages:
- test
variables:
MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
test:
stage: test
allow_failure: true
# Run the tests
script:
- echo "Executing BDD scenarios with maven"
- mvn clean verify
# Store artifacts
artifacts:
when: always
name: "Serenity Report"
paths:
- target/site/*
expire_in: 24 h
Image – markhobson/maven-chrome is used in this test. It is a docker image for Java automated UI tests.
This command lets the pipeline continue running subsequent jobs, even if the previous job is failed.
allow_failure: true
The below command is used to execute the tests from Maven in the docker image.
mvn clean verify
The below command means that the artifact should be generated every time the pipeline is run, irrespective of the fact if the job is successful or failed.
artifacts:
when: always
The below command provides the name of the artifact. If this is not used, then the artifacts file is named artifacts, which becomes artifacts.zip when downloaded.
artifacts:
name: "Serenity Report"
The below command provides the path of the files that should be present in the artifact.
artifacts:
paths:
- target/site/*
Pipeline configuration begins with jobs. Jobs are the most fundamental element of a .gitlab-ci.yml file.
Jobs are:
Defined with constraints stating under what conditions they should be executed.
Top-level elements with an arbitrary name and must contain at least the script clause.
Not limited in how many can be defined.
Jobs can output an archive of files and directories. This output is known as a job artifact. The expire_in keyword determines how long GitLab keeps the job artifacts. Here, it shows 24 hrs to retain the artifacts.
Step 4 – Run the tests in the GitLab pipeline
Now, when a new change is committed, a pipeline kicks off and it runs all the tests.
Step 5 – Check the status of the pipeline
Once the Status of the pipeline changes to either failed or passed, that means the tests are already executed.
As you can see, the Status is passed, its green colour. This means all the tests present in the test suite are executed and passed. If any test fails in the test suite, the final execution status will be brown. The reason for the brown colour is we have mentioned allow_failure: true.
Below is the execution status report in the GitLab pipeline.
As I have added an artifact also in the .gitalb-ci.yml, which is highlighted in the image. This artifact creates a folder with the name “Serenity_Report”and the reports in this folder come from the path /target/site. This artifact gives us the option to download the reports or browse the report. This report will be available for 24 hours only as mentioned in the gitlab-ci.yml.
Step 6 – Download the report
Once, will click on the download button, it will download “Serenity_Report.zip”. Unzip the folder and it looks like something as shown below:
Example of Index.html
Example of Serenity Summary Report
Congratulations. This tutorial has explained the steps to run Serenity tests in GitLab CI/CD. Happy Learning!!
This tutorial walks you through the process of running Robot Framework tests in GitLab pipelines. This is a very important step in achieving CI/CD. Ideally, tests should be run after each change (minor/major) before the latest change ismergedinto the master branch. Let’sassumethat100 changes are merged into the master branch a day and tests are run every time before deployment. In this case, there is no QA manuallystarting100tests a day. Now, what should be done to overcome this problem. Nowaddatest to your GitLab pipeline. Adding a test stage to your pipeline automatically runs the tests when you run the pipeline.
Once the proposed changes are built, then push the commits to a feature branch in a remote repository that’s hosted in GitLab. The push triggers the CI/CD pipeline for your project. Then, GitLab CI/CD runs automated scripts (sequentially or in parallel) to build as well as to test the application. After a successful run of the test scripts, GitLab CI/CD deploys your changes automatically to any environment (DEV/QA/UAT/PROD). But if the test stage is failed in the pipeline, then the deployment is stopped.
To use GitLab CI/CD, we need to keep 2 things in mind:
a) Make sure a runner is available in GitLab to run the jobs. If there is no runner, install GitLab Runner and register a runnerfor your instance, project, or group.
b) Create a .gitlab-ci.yml file at the root of the repository. This file is where CI/CD jobs are defined.
The Selenium tests run on a headless browser in the pipeline.
What is a headless browser?
A headless browser is like any other browser but without a Head/GUI (Graphical User Interface). A headless browser is used to automate the browser without launching the browser. While the tests are running, we could not see the browser, but we can see the test results coming on the console.
Prerequisite
Install Python
Install PIP
Install Robot Framework
Install Robot framework Selenium Library
Install PyCharm IDE
GitLab Account
Implementation Steps:
Step 1 – Create a new Project
Step 1.1 – Open PyCharm and create a new project. Go to File and select New Project from the main menu.
Step 1.2 – Choose the project location. Click the Browse button next to the Location field and specify the directory for your project.
Deselect the Create a main.py welcome script checkbox because you will create a new Python file for this tutorial.
Click on the Create Button.
Step 1.3 – A new dialog will appear asking to Open the project using any one of the given options. I have selected New Window as I like to have separate windows for each project.
Below is the image of the new project created in PyCharms.
Step 2 – Create 2 new directories in the new project
Right-Click on the project, select New->Directory, and provide the name as TestCases and Resources
Step 3 – Create Test Files
This directory contains multiple test case files consisting of test steps.
Right-click on the new directory select New File and provide the name LoginPageTests.robot and ForgetPasswordTests.robot as shown below:
Below is the code for LoginPageTests.robot
*** Settings ***
Documentation Tests to login to Login Page
Library SeleniumLibrary
Test Setup Open the Browser with URL
Test Teardown Close Browser Session
Resource ../Resources/GenericResources.robot
Resource ../Resources/LoginResources.robot
Resource ../Resources/DashboardResources.robot
*** Test Cases ***
Validate Unsuccessful Login using invalid credentials
LoginResources.Fill the login form ${valid_username} ${invalid_password}
LoginResources.Verify the error message is correct
Validate Unsuccessful Login for blank username
LoginResources.Fill the login form ${blank_username} ${valid_password}
LoginResources.Verify the error message is displayed for username
Validate Unsuccessful Login for blank password
LoginResources.Fill the login form ${valid_username} ${blank_password}
LoginResources.Verify the error message is displayed for password
Validate successful Login
LoginResources.Fill the login form ${valid_username} ${valid_password}
DashboardResources.Verify Dashboard page opens
Below is the code for ForgetPasswordTests.robot
*** Settings ***
Documentation Tests to validate Forgot Your Password Page functionality
Library SeleniumLibrary
Test Setup Open the Browser with URL
Test Teardown Close Browser Session
Resource ../Resources/GenericResources.robot
Resource ../Resources/LoginResources.robot
Resource ../Resources/ForgetPasswordResources.robot
*** Test Cases ***
Validate Reset Password Functionality
LoginResources.Go to Forgot Your Password Page
ForgetPasswordResources.Verify Forgot Your Password Page opens
ForgetPasswordResources.Fill the Forgot Password Page
ForgetPasswordResources.Verify the message
Validate Cancel Functionality
LoginResources.Go to Forgot Your Password Page
ForgetPasswordResources.Verify Forgot Your Password Page opens
ForgetPasswordResources.Cancel the Reset Password
ForgetPasswordResources.Verify that Login Page is displayed
Step 4 – Create a Resource file for each page
It maintains the files which contain page elements as well as corresponding keywords.
Right-click on the new directory select New File and provide the name as LoginResources.robot, DashboardResources.robot, GenericResources.robot, and ForgetPasswordResources.robot as shown below:
GenericResources.robot contains the keywords that are common to all the tests, like opening of the browser or closing of the browser.
*** Settings ***
Documentation A resource file with reusable keywords and variables.
Library SeleniumLibrary
*** Variables ***
${valid_username} Admin
${valid_password} admin123
${invalid_username} 1234
${invalid_password} 45678
${blank_username}
${blank_password}
${url} https://opensource-demo.orangehrmlive.com/web/index.php/auth/login
${browser_name} Chrome
*** Keywords ***
Open the Browser with URL
Open Browser ${url} headlesschrome
Maximize Browser Window
Set Selenium Implicit Wait 5
Close Browser Session
Close Browser
Below is the code for LoginResources.robot
*** Settings ***
Documentation All the page objects and keywords of landing page
Library SeleniumLibrary
*** Variables ***
${login_error_message} css:.oxd-alert-content--error
${dashboard_title} css:.oxd-topbar-header-breadcrumb-module
${missing_username_error_message} xpath://*[@class='oxd-form']/div[1]/div/span
${missing_password_error_message} xpath://*[@class='oxd-form']/div[2]/div/span
${forgot_password_link} xpath://div[@class='orangehrm-login-forgot']/p
*** Keywords ***
Fill the login form
[Arguments] ${username} ${password}
Input Text css:input[name=username] ${username}
Input Password css:input[name=password] ${password}
Click Button css:.orangehrm-login-button
Verify the error message is correct
Element Text Should Be ${login_error_message} Invalid credentials
Verify the error message is displayed for username
Element Text Should Be ${missing_username_error_message} Required
Verify the error message is displayed for password
Element Text Should Be ${missing_password_error_message} Required
Go to Forgot Your Password Page
Click Element ${forgot_password_link}
Below is the code for DashboardResources.robot
*** Settings ***
Documentation All the page objects and keywords of Dashboard page
Library SeleniumLibrary
*** Variables ***
${dashboard_title} css:.oxd-topbar-header-breadcrumb-module
*** Keywords ***
Verify Dashboard page opens
Element Text Should Be ${dashboard_title} Dashboard
Below is the code for ForgetPasswordResources.robot
*** Settings ***
Documentation All the page objects and keywords of Forget Password page
Library SeleniumLibrary
*** Variables ***
${forgot_page_title} css:.orangehrm-forgot-password-title
${username} css:.oxd-input--active
${reset_btn} css:.orangehrm-forgot-password-button--reset
${cancel_btn} css:.orangehrm-forgot-password-button--cancel
${reset_message} xpath://div[@class='orangehrm-card-container']/h6
${login_page_title} xpath://*[@class='orangehrm-login-slot']/h5
*** Keywords ***
Verify Forgot Your Password Page opens
Element Text Should Be ${forgot_page_title} Reset Password
Fill the Forgot Password Page
Input Text ${username} abc@gmail.com
Click Button ${reset_btn}
Verify the message
Element Text Should Be ${reset_message} Reset Password link sent successfully
Cancel the Reset Password
Click Button ${cancel_btn}
Verify that Login Page is displayed
Element Text Should Be ${login_page_title} Login
All the below-mentioned keywords are derived from SeleniumLibrary. The functionality of keywords mentioned above:
1. Create Webdriver − The keyword creates an instance of Selenium WebDriver.
2. Go To – This keyword navigates the current browser window to the provided url.
3. Maximize Browser Window – This keyword maximizes the current browser window.
4. Set Selenium Implicit Wait – This keyword sets the implicit wait value used by Selenium.
5. Input Text − This keyword is used to type the given text in the specified textbox identified by the locator name:username.
6. Input Password – This keyword is used to type the given text in the specified password identified by the locator name:password.
The difference compared to Input Text is that this keyword does not log the given password on the INFO level.
7. Click button – This keyword is used to click the button identified by the locator. In this case, it is “Login” button.
8. Element Text Should Be – This keyword is used to verify that the current page contains the exact text identified by the locator. Here, we are checking the exact text “Invalid Credentials”.
To run this script, go to the command line and go to directory tests.
Step 5 – Execute the tests
We need the below command to run the Robot Framework script.
robot .
The output of the above program is
GitLab Section
Step 6 – Create a blank project in GitLab
To know, how to create a blank new project in GitLab, please refer tothis tutorial.
Step 7 – Push the project from the local repository to the Gitlab Repository
To know, how to push the changes in GitLab, please refer to this tutorial.
Step 8 – Create a .gitlab-ci.yml file in the project in GitLab
There are many ways to create a new file in GitLab. One of the ways is to create a file as shown in the below image.
It is a YAML file where you configure specific instructions for GitLab CI/CD. In the .gitlab-ci.yml, we can define:
The scripts you want to run.
Other configuration files and templates you want to include.
Dependencies and caches.
The commands you want to run in sequence and those you want to run in parallel.
The location to deploy your application to.
Whether you want to run the scripts automatically or trigger any of them manually.
Below is the sample pipeline.
stages:
- test
image: ppandiyan/robotframework
run-tests:
stage: test
script:
- robot --outputdir testOutput .
artifacts:
paths:
- testOutput
expire_in: 1 day
Image – ppandiyan/robotframework is used in this test. This docker file contains headlesschrome, headlessfirefox and Python2.7.
This gitlab-ci.yml has only 1 stage – a test that contains the command to run the tests as well as also create an artifact that contains all the surefire reports which can be saved as Test Evidence.
Step 9 – Run the tests in the GitLab pipeline
Now, when a new change is committed, a pipeline kicks off and it runs all the tests.
Step 10 – Check the status of the pipeline
Once the Status of the pipeline changes to either failed or passed that means the tests are already executed.
Let us see the logs of the execution it shows that all the tests are passed here.
As I have added an artifact also in the gitalb-ci.yml, which is highlighted in the image. This artifact creates a folder with the name “testOutput” and the folder contains reports, logs as well as output files. This artifact gives us the option to download the reports or browse the report. This report will be available for 1 day only as mentioned in the gitlab-ci.yml.
Step 11 – Download the report
Once, will click on the download button, it will download “report.zip”. Unzip the folder and it looks like something as shown below:
Example of Report.html
Example of Log.html
Congratulations. This tutorial has explained the steps to run Robot Framework tests in GitLab CI/CD. Happy Learning!!
Docker is an open platform for developing, shipping, and running applications. Docker provides the ability to package and run an application in a loosely isolated environment called a container. A container is a runnable instance of an image. You can create, start, stop, move, or delete a container using the Docker API or CLI. You can connect a container to one or more networks, attach storage to it, or even create a new image based on its current state.
By default, a container is relatively well isolated from other containers and its host machine It contains multiple components such as Docker Daemon, Docker Clients, Docker Registry, and Docker Compose. It works on a client-server architecture. The Docker client communicates with the Docker Daemon handles the complex part of building, running, and distributing the Docker containers.
Docker Desktop is an easy-to-install application for your Mac or Windows environment that enables you to build and share containerized applications and microservices. Docker Desktop includes the Docker daemon (dockerd), the Docker client (docker), Docker Compose, Docker Content Trust, Kubernetes, and Credential Helper.
Why use Selenium with Docker for web application testing
When Selenium is used with Docker, there is no need to install the necessary browsers and browser drivers on the host machine. Overall, it is significantly quicker to get started with Selenium web automation testing using pre-made Docker containers. A range of Docker images (with Selenium) on the Docker Hub can be used by running a few commands on the terminal. Some of these images on the Docker Hub were developed and maintained by Selenium HQ.
One of the most common uses of Docker containers in selenium testing is cross-browser testing. Setting up test setups with all of the Browser-OS combinations gets difficult. These can be set up on the go and taken down once the tests are completed using Docker containers.
Another use of Docker containers in Selenium testing is parallel testing. Because there are so many sorts of tests to conduct, doing them sequentially takes a long time. As a result, parallelizing the testing saves time and provides faster feedback. Parallel testing, on the other hand, necessitates a significant amount of infrastructure setup. This would have a direct bearing on the price. Multiple containers can be launched on a single server using docker containers, making the most of the underlying hardware while also allowing for concurrent testing
Let’s discuss how to set up docker and run Selenium tests on it.
Setting up Window Docker
Step 1 – Download the Docker Installer
Docker provides an installer for Windows which can be downloaded from the official docker website.
Step 2 – Install Docker
Launch the installer by double-clicking on it. Select the Enable Hyper-V Windows Features option on the configuration page.
If the user account and admin accounts are different, the user account must be added to the docker-users group as shown below:
To do that, you will need to run Computer Management as an administrator and navigate to Local Users and Groups > Groups > docker-users. Then right-click to add the user to the group. You will need to log and log back in for the changes to take effect.
Step 3 – Start Docker Desktop
After the installation process is complete, the tool does not start automatically. To start the Docker tool, search for the tool, and select Docker Desktop in your desktop search results or Docker Desktop can be started from the start menu.
Docker is free for small businesses, personal usage, education, and non-commercial purposes.
Step 4 – Verify the installation of Desktop Docker
To see if Docker is correctly configured, run the following line in Command Prompt. The version of Docker installed on the system is provided.
docker --version
Running Selenium Tests in Docker
The Docker Desktop can execute a few docker images after it is installed. You can either build a Docker image from the scratch or start with a pre-configured base image from the Docker hub and add to it.
Run the following command to download a copy of the image onto the system.
docker pull selenium/standalone-firefox
Step 2 – Running the Selenium Webdriver Docker container
Upon pulling the selenium/standalone-firefox image onto the system, start the container by running the following command:
docker run -d -p 4444:4444 -v /dev/shm:/dev/shm selenium/standalone-firefox
The above command starts a container from the image specified in detached mode (background mode). It also maps Port 4444 on the container to Port 4444 on your local browser.
Open the browser and navigate to http://localhost:4444/. It reflects Selenium Grid UI, as shown below.
I have created a base class where the WebDriver is initialized and at the end is closed.
import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.After;
import org.junit.Before;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.remote.RemoteWebDriver;
import java.net.MalformedURLException;
import java.net.URL;
public class BaseTest {
protected static ThreadLocal<RemoteWebDriver> driver = new ThreadLocal<>();
public static String remote_url_firefox = "http://localhost:4444/wd/hub";
@Before
public void setDriver() throws MalformedURLException {
WebDriverManager.firefoxdriver().setup();
FirefoxOptions options = new FirefoxOptions();
options.addArguments("window-size=1920,1200");
driver.set(new RemoteWebDriver(new URL(remote_url_firefox), options));
driver.get().get("https://opensource-demo.orangehrmlive.com/");
}
public WebDriver getDriver() {
return driver.get();
}
@After
public void closeBrowser() {
driver.get().quit();
driver.remove();
}
}
The below class contains the various tests. The tests are related to verifying the login to the application, verifying the title of the current page, verifying the error message generated on providing the invalid credentials, verifying the LinkedIn link, and verifying the heading of forgot password page.
This tutorial explains the process to run the Selenium Tests in the GitLab pipeline. This is a very important step towards achieving CI/CD. Ideally, the tests need to run after any change (minor/major) before merging the latest change to the master branch. Suppose there are 100 changes merged to the master branch in a day. It is expected to run the tests every time before deployment. In this case, any QA won’t want to start the tests manually 100 times in a day. Now, what should be done to overcome this problem. Now, adding tests to the GitLab pipeline comes into the picture. We can add a test stage to the pipeline and the tests will run automatically when the pipeline run.
Build the proposed changes. Then, push the commits to a feature branch. This branch should be in a remote repository that’s hosted in GitLab. The push triggers the CI/CD pipeline for your project. Then, GitLab CI/CD runs automated scripts (sequentially or in parallel) to build as well as to test the application. After a successful run of the test scripts, GitLab CI/CD deploys your changes automatically to any environment (DEV/QA/UAT/PROD). But if the test stage is failed in the pipeline, then the deployment is stopped.
After the implementation works as expected:
Get the code reviewed and approved.
Merge the feature branch into the default branch.
GitLab CI/CD deploys your changes automatically to a production environment.
To use GitLab CI/CD, we need to keep 2 things in mind:
a) Make sure a runner is available in GitLab to run the jobs. If there is no runner, install GitLab Runner and register a runner for your instance, project, or group.
b) Create a .gitlab-ci.yml file at the root of the repository. This file is where CI/CD jobs are defined.
The Selenium tests run on a headless browser in the pipeline.
What is a headless browser?
A headless browser is like any other browser, but without a Head/GUI (Graphical User Interface). A headless browser is used to automate the browser without launching the browser. While the tests are running, we cannot see the browser. However, we can see the test results coming on the console.
To explain, I have created 2 Selenium tests and used TestNG for asserting the tests. The tests will run on a headless Chrome browser. One more thing to keep in mind is that when tests run on a headless Chrome browser, the window screen won’t be full screen. So, the screen needs to be maximized explicitly otherwise some of the tests would fail.
Implementation Steps
Step 1 – Create a new Maven Project
Step 2- Add the dependencies to the POM.xml
Add the below-mentioned dependencies that need to add to the project to the pom.xml in Maven Project.
There are 2 different pages that need to be tested – LoginPage and ForgetPasswordPage
LoginPage contains the tests to log in to the application. After successful login, the application moves to the next webpage – HomePage. You can see that BaseTest class is extended in both the Test classes.
import org.openqa.selenium.By;
import org.testng.Assert;
import org.testng.annotations.Test;
@Test
public class ForgetPasswordPage extends BaseTest{
@Test
public void getHeading() {
Thread.sleep(1000);
driver.findElement(By.xpath("//*[@id='forgotPasswordLink']/a")).click();
String heading = driver.findElement(By.xpath("//*[@id='content']/div[1]/div[2]/h1")).getText();
System.out.println("Title : "+ heading );
Assert.assertEquals(heading, "Forgot Your Password?");
}
}
Step 4 – Create testng.xml to run the tests
Now, let’s create a testng.xml to run the TestNG tests. If JUnit is used instead of TestNG, then this step is not needed.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
<test name="Test">
<classes>
<class name="org.example.DockerDemo.LoginPage"/>
<class name="org.example.DockerDemo.ForgetPasswordPage"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->
Step 5 – Run the tests through command line
Now, let us execute the tests through the command line. Go to the place where the pom.xml of the project is placed and use the below command to run the tests. This step makes sure that all the tests are running as expected.
mvn compile test
GitLab Section
Step 6 – Create a blank project in GitLab
To know, how to create a blank new project in GitLab, please refer tothis tutorial.
Step 7 – Push the project from local repository to Gitlab Repository
To know, how to push the changes in GitLab, please refer to this tutorial.
Step 8 – Create a .gitlab-ci.yml file in the project in GitLab
There are many ways to create a new file in GitLab. One of the ways is to create a file as shown in the below image.
It is a YAML file where you configure specific instructions for GitLab CI/CD. In the .gitlab-ci.yml, we can define:
The scripts you want to run.
Other configuration files and templates you want to include.
Dependencies and caches.
The commands you want to run in sequence and those you want to run in parallel.
The location to deploy your application to.
Whether you want to run the scripts automatically or trigger any of them manually.
image: markhobson/maven-chrome
stages:
- test
variables:
MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
test:
stage: test
allow_failure: true
# Run the tests
script:
- mvn $MAVEN_OPTS clean package
- mvn compile test
# Store artifacts
artifacts:
when: always
name: "report"
paths:
- target/surefire-reports/*
expire_in: 1 h
Image – markhobson/maven-chrome is used in this test. It is a docker image for Java automated UI tests.
This gitlab-ci.yml has only 1 stage. It is a test stage that contains the command to run the tests. It also creates an artifact that contains all the surefire reports. These reports can be saved as Test Evidence.
Step 9 – Run the tests in the GitLab pipeline
Now, when a new change is committed, a pipeline kicks off and it runs all the tests.
Step 10 – Check the status of the pipeline
Once the Status of the pipeline changes to either failed or passed.. that means the tests are already executed.
As you can see the Status is failed here which means that the execution is completed. Let us see the logs of the execution it shows that out of 2 tests, 1 test passed and 1 test failed. This shows that tests are running successfully in the GitLab pipeline.
As I have added an artifact also in the gitalb-ci.yml, which is highlighted in the image. This artifact creates a folder with the name “report” and the reports in this folder come from the path /target/surefire-reports. This artifact gives us the option to download the reports or browse the report. This report will be available for 1 hour only as mentioned in the gitlab-ci.yml.
Step 11 – Download the report
Once, will click on the download button, it will download “report.zip”. Unzip the folder and it looks like something as shown below:
Example of Emailable-Report.html
Example of Index.html
Congratulations. This tutorial has explained the steps to run Selenium tests in GitLab CI/CD. Happy Learning!!
QA Automation Expert 