GitHub Actions is a powerful tool that can be used to automate your software development workflows. In this blog post, we will show you how to get started with GitHub Actions for Java Spring Boot.
Why GitHub?
The flexible aspects of GitHub Actionsenable users to create powerful, fast, and efficient automated testing workflows in CI/CD environments.
CI/CD pipelines have contributed to the success of the DevOps cycle in all software development projects. This is a holistic process that bridges development and operations. Continuous integration helps development teams deploy code efficiently, and continuous delivery automates code deployment.
Implementation Steps
Step 1 – Create GitHub Actions and Workflows
I have a repository available on GitHub – SpringBoot_Serenity_Cucumber_JUnit4_Demo as shown in the below image. Go to the “Actions” tab. Click on the “Actions” tab.
Step 2 – Select the type of Actions
You will see that GitHub recommends Actions depending on the project. In our case, it is recommending actions suitable for a Java project. I have selected the “Java with Maven” option as my project is built in Maven.
Step 3 – Generation of Sample pipeline
If you choose an existing option, it will automatically generate a .yaml for the project as shown below.
We will replace the current workflow with the following yml file as shown below:
name: Build and Test Java Spring Boot Application
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up JDK 17
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'temurin'
cache: maven
- name: Build with Maven
run: mvn clean verify
- name: Test Report
uses: actions/upload-artifact@v3
if: success() || failure()
with:
name: SpringBoot Report # Name of the folder
path: target/site/serenity/ # Path to test results
Step 4 – Commit the changes
After the changes, hit the “Commit changes” button.
This will give the option to add a description for the commit. It will enable the user to commit either to the main branch. The user can also commit to any other branch that exists in the project. Click on the “Commit changes” button to set up the workflow file.
Below is the image of the newly created maven.yml file.
Step 5 – Verify that the workflow is running
Next, head over to the “Actions” tab, and you will see your YAML workflow file present under the tab. The yellow sign represents that the job is in the queue.
In Progress – When the job starts building and running, you will see the status change from “Queued” to “in progress”.
Passed – If the build is successful, you will see a green tick mark.
Failed – In case the build has failed, then there will be a red cross mark.
Click on the workflow and the below screen is displayed. It shows the status of the run of the workflow. It also shows the total time taken to run the workflow and the name of the .yml file.
Below shows all the steps of the workflow.
Step 6 – Published artifacts on GitHub
Once the pipeline run, a SpringBoot Report folder will be generated as shown in the below image:
When we click on the folder SpringBoot Report, a zipped file will be downloaded. We can extract it to see all the files contained within it.
The previous tutorial has explained how to install it onto a Windows 10 system and create a Maven orGradle project in Jenkins. Jenkins and GitHub are two powerful tools on their own, but what about using them together? In this tutorial, learn about the benefits of a Jenkins GitHub integration and how to set up the integration on your own.
GitHub is a web-based version control system and collaboration platform for developers. It is the center around which all things involving git revolve. GitHub allows software developers and engineers to create remote, public-facing repositories on the cloud for free. Once you’ve set up a repository on GitHub, you can copy it to your device, add and modify files locally, then “push” your changes back to the repository where your changes display for the public.
What is Jenkins?
Jenkins is a well-known open-source tool that aids in the implementation of Continuous Integration (CI) and Continuous Deployment/Continuous Delivery (CD) processes by automating parts of the software development pipeline such as building, testing, and deployment.
Jenkins Version Used – 2.361.2
In this version of Jenkins, Git Plugin is already installed. If the Git Plugin is not installed, then follow the below steps to add Git Plugin in Jenkins.
Step 1: Open your dashboard
Click on the Manage Jenkins button on your Jenkins dashboard:
Step 2: Select Manage Plugins
Click on the Manage Jenkins. Choose Manage Plugins.
Step 3: Add Git Plugin
On the Plugins Page, go to the Available option.
Select the GIT Plugin
Click on Install without restart.The plugin will take a few moments to finish downloading depending on your internet connection, and will be installed automatically.
You can also select the option Download now and Install after the restart button. In which plugin is installed after the restart
You will be shown a “No updates available” message if you already have the Git plugin installed.
In my case, Git Plugin is already installed, so you can’t see it in the Available Plugin.
Step 4: Verify Git Plugin is installed
Once the plugins have been installed, Go to Manage Jenkinson your Jenkins dashboard. You will see your plugins listed among the rest.
How to Integrate Jenkins With GitHub
Step 1: Create a new project using the Maven project plugin
Give the Name of the project.
Click on the Maven project.
Click on the OK button.
In the General section, enter the project description in the Description box.
Click on create new jobs.
Enter the item name, select the job type, and click OK. We shall create a Maven project as an example.
Step 2: Describe the project in the description section
In the General section, enter the project description in the Description box.
Step 3 – Source Code Management section
You will see a Git option under Source Code Managementif your Git plugin has been installed in Jenkins.
Enter the Git repository URL to pull the code from GitHub. Enter the credentials to log in to GitHub.
I have already set up the credentials to log in to GitHub. For first-time users, click on the Add button and select Jenkins.
Add the username and password used to log in to GitHub. Click on the “Add” button.
Note:- Please make sure that Git is installed on your local machine. To install Git on your local machine, go to this tutorial –How to install Git on Windows 10.
Step 4: Build Management
Go to the Build section of the new job.
In the Root POM textbox, enter pom.xml
In the Goals and Options section, enter “clean test“
Step 5: Select “Publish TestNG Results” from “Post Build Actions”
Scroll down to “Post Build Actions” and click on the “Add Post Build Actions” drop-down list. Select “Publish TestNG Results“. In my case, the project is set up to generate TestNG Report, so I have selected TestNG. But if your project uses any other reporting option, please select that from the dropdown.
Enter TestNG XML Report Pattern as “**target/surefire-reports/testng-results.xml” and click on the “Apply” and “Save” button.
We have created a new project “GitHub_Demo” with the configuration to run TestNG Tests and also to generate TestNG Reports after execution using Jenkins.
Step 6: Execute the tests
Let’s execute it now by clicking on the “Build Now” button.
Right-click on Build Number (here in my case it is #1) and click on Console Output to see the result.
Below is the screenshot of the Console Output. It displays all the information regarding the execution of the tests.
Step 7: View the TestNG Report
Once the execution is completed, we could see a link to view “TestNG Results“.
Click on the TestNG Results. It displays the summary of the tests.
This way, we could integrate GitHub in Jenkins.
Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!
GitLab automatically enables CI/CD pipelines for new projects. It’s just a matter of adding a new configuration file called .gitlab-ci.yml to your code repository with instructions for GitLab on what to run. So simply create the following basic workflow in your main repository directory and commit it:
The Serenity tests run on a headless browser in the pipeline.
What is a headless browser?
A headless browser is like any other browser but without a Head/GUI (Graphical User Interface). A headless browser is used to automate the browser without launching the browser. While the tests are running, we could not see the browser, but we can see the test results coming on the console. The tests can run in the GitLab pipeline if the tests run in the headless mode.
To know, how to create a blank new project in GitLab, please refer tothis tutorial.
Step 2 – Push the project from the local repository to GitLab Repository
To know, how to push the changes in GitLab, please refer to this tutorial.
Step 3 – Create a .gitlab-ci.yml file in the project in GitLab
There are many ways to create a new file in GitLab. One of the ways is to create a file as shown in the below image.
It is a YAML file where you configure specific instructions for GitLab CI/CD. In the .gitlab-ci.yml, we can define:
The scripts you want to run.
Other configuration files and templates you want to include.
Dependencies and caches.
The commands you want to run in sequence and those you want to run in parallel.
The location to deploy your application to.
Whether you want to run the scripts automatically or trigger any of them manually.
Below is a sample example to run the Serenity tests in the GitLab pipeline.
image: markhobson/maven-chrome
stages:
- test
variables:
MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
test:
stage: test
allow_failure: true
# Run the tests
script:
- echo "Executing BDD scenarios with maven"
- mvn clean verify
# Store artifacts
artifacts:
when: always
name: "Serenity Report"
paths:
- target/site/*
expire_in: 24 h
Image – markhobson/maven-chrome is used in this test. It is a docker image for Java automated UI tests.
This command lets the pipeline continue running subsequent jobs, even if the previous job is failed.
allow_failure: true
The below command is used to execute the tests from Maven in the docker image.
mvn clean verify
The below command means that the artifact should be generated every time the pipeline is run, irrespective of the fact if the job is successful or failed.
artifacts:
when: always
The below command provides the name of the artifact. If this is not used, then the artifacts file is named artifacts, which becomes artifacts.zip when downloaded.
artifacts:
name: "Serenity Report"
The below command provides the path of the files that should be present in the artifact.
artifacts:
paths:
- target/site/*
Pipeline configuration begins with jobs. Jobs are the most fundamental element of a .gitlab-ci.yml file.
Jobs are:
Defined with constraints stating under what conditions they should be executed.
Top-level elements with an arbitrary name and must contain at least the script clause.
Not limited in how many can be defined.
Jobs can output an archive of files and directories. This output is known as a job artifact. The expire_in keyword determines how long GitLab keeps the job artifacts. Here, it shows 24 hrs to retain the artifacts.
Step 4 – Run the tests in the GitLab pipeline
Now, when a new change is committed, a pipeline kicks off and it runs all the tests.
Step 5 – Check the status of the pipeline
Once the Status of the pipeline changes to either failed or passed, that means the tests are already executed.
As you can see, the Status is passed, its green colour. This means all the tests present in the test suite are executed and passed. If any test fails in the test suite, the final execution status will be brown. The reason for the brown colour is we have mentioned allow_failure: true.
Below is the execution status report in the GitLab pipeline.
As I have added an artifact also in the .gitalb-ci.yml, which is highlighted in the image. This artifact creates a folder with the name “Serenity_Report”and the reports in this folder come from the path /target/site. This artifact gives us the option to download the reports or browse the report. This report will be available for 24 hours only as mentioned in the gitlab-ci.yml.
Step 6 – Download the report
Once, will click on the download button, it will download “Serenity_Report.zip”. Unzip the folder and it looks like something as shown below:
Example of Index.html
Example of Serenity Summary Report
Congratulations. This tutorial has explained the steps to run Serenity tests in GitLab CI/CD. Happy Learning!!
In this tutorial, we will run the tests in the BDD format in Robot Framework.
What is BDD?
BDD is an Agile software development process in which an application is documented and designed around the behaviour that a user expects to see when interacting with it. BDD helps to avoid bloat, excessive code, unnecessary features, and lack of focus by encouraging developers to focus only on the requested behaviours of an app or program. This methodology combines, augments, and refines test-driven development (TDD) and acceptance testing practices.
The Given-When-Then syntax is a commonly used structure for writing user stories and acceptance criteria in a behaviour-driven development (BDD). It is used to describe the desired behaviour of a system in a clear, concise, and consistent manner.
The structure is broken down into three parts:
Given: This section describes the initial state or context of the system. It sets the scene for the scenario being tested.
When: This section describes the action or event that occurs. It specifies the trigger for the scenario being tested.
Then: This section describes the expected outcome or result of the scenario. It defines the acceptance criteria for the scenario being tested.
Step 1.1 – Open PyCharm and create a new project. Go to File and select New Project from the main menu.
Step 1.2 – Choose the project location. Click the “Browse” button next to the Location field and specify the directory for your project.
Deselect the Create a main.py welcome script checkbox because you will create a new Python file for this tutorial.
Click on the “Create” Button.
Step 1.3 – A new dialog appears asking to open the project using any one of the given options. I have selected New Window as I like to have separate windows for each project.
Below is the image of the new project created in PyCharms.
How to run tests in BDD format in Robot Framework?
Step 2 – Create a new directory in the new project
Right-Click on the project, select New->Directory and provide name as Tests
Below is the image of the new directory.
Right-click on the new directory and select New File and provide the name as BDD_Demo.robot as shown below:
Step 3 – Execute the tests
We are now going to write test cases. The test case details will be as follows −
*** Settings ***
Documentation Tests to login to Login Page
Library SeleniumLibrary
*** Variables ***
${valid_username} Admin
${valid_password} admin123
${invalid_password} 45678
${url} https://opensource-demo.orangehrmlive.com/web/index.php/auth/login
${browser_name} Chrome
${login_error_message} css:.oxd-alert-content--error
${dashboard_title} css:.oxd-topbar-header-breadcrumb-module
*** Test Cases ***
Validate Unsuccessful Login using invalid credentials
[Tags] SMOKE
Given I open the Browser with URL
When I fill the login form ${valid_username} ${invalid_password}
Then I verify the error message is correct
And Close Browser Session
Validate successful Login
[Tags] UAT
Given I open the Browser with URL
When I fill the login form ${valid_username} ${valid_password}
Then I verify Dashboard page opens
And Close Browser Session
*** Keywords ***
Given I open the Browser with URL
Create Webdriver ${browser_name} executable_path=/Vibha_Personal/RobotFramework_Demo/drivers/${browser_name}
Go To ${url}
Maximize Browser Window
Set Selenium Implicit Wait 5
When I fill the login form
[Arguments] ${username} ${password}
Input Text css:input[name=username] ${username}
Input Password css:input[name=password] ${password}
Click Button css:.orangehrm-login-button
Then I verify the error message is correct
Element Text Should Be ${login_error_message} Invalid credentials
Then I verify Dashboard page opens
Element Text Should Be ${dashboard_title} Dashboard
And Close Browser Session
Close Browser
All the below-mentioned keywords are derived from SeleniumLibrary except the last one. The functionality of keywords mentioned above:
1. Open Browser − The keyword opens a new browser instance to the optional URL.
2. Maximize Browser Window – This keyword maximizes the current browser window.
3. Set Selenium Implicit Wait – This keyword sets the implicit wait value used by Selenium.
4. Input Text − This keyword is used to type the given text in the specified textbox identified by the locator name:username.
5. Input Password – This keyword is used to type the given text in the specified password identified by the locator name:password.
The difference compared to Input Text is that this keyword does not log the given password on the INFO level.
6. Click button – This keyword is used to click on the button with location css:.orangehrm-login-button.
7. ${result} – This is a variable that holds the text value of the error message that is located by css:.oxd-alert-content-text
8. Element Text Should Be– This keyword is used to verify that the element locator contains exact the text expected.
To run this script, go to the command line and go to directory tests.
Step 4 – Execute the tests
We need the below command to run the Robot Framework script.
robot .
The output of the above program is
Step 5 – View Report and Log
We have the test case passed. The Robot Framework generates log.html, output.xml, and report.html by default.
Let us now see the report and log details.
Report
Right-click on report.html. Select Open In->Browser->Chrome(any browser of your wish).
The Report generated by the framework is shown below:
Log
Robot Framework has multiple log levels that control what is shown in the automatically generated log file. The default Robot Framework log level is INFO.
Right-click on log.html. Select Open In->Browser->Chrome(any browser of your wish).
That’s it! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
Variables are an essential component of Robot Framework and can be utilized in almost any place in test data. Simply said, if we have values that change frequently and are utilized several times in different test scenarios, adding a variable can help. So, if our value changes in the future, we can just update it in one place, and it will be reflected in all the test cases. Test data and locators are the finest use cases for variables.
In Robot Framework, there are four sorts of variables.
1. Scalar (Identifier: $) – The most common way to use variables in Robot Framework test data is using the scalar variable syntax like ${var}. When this syntax is used, the variable name is replaced with its value as-is.
2. List (Identifier: @) – If a variable value is a list or list-like, a list variable like @{EXAMPLE} is used. In this case, the list is expanded, and individual items are passed in as separate arguments.
3. Dictionary (Identifier: &) – A variable containing a Python dictionary or a dictionary-like object can be used as a dictionary variable like &{EXAMPLE}. In practice, this means that the dictionary is expanded and individual items are passed as named arguments to the keyword.
4. Environment (Identifier: %) – Robot Framework allows using environment variables in the test data using the syntax %{ENV_VAR_NAME}. They are limited to string values.
*** Variables ***
${STRING} cute name #Scalar
${INT_AS_STRING} 1 #Scalar
${INT_AS_INT} ${1} #Scalar
${FLOAT} ${3.14} #Scalar
@{LIST} one two three
&{DICTIONARY} string=name int=${1} list=@{LIST}
${ENVIRONMENT} %{PATH}
Let’s write a simple test using all the above identifiers:
*** Settings ***
Documentation To validate the Login Form
Library SeleniumLibrary
Library Collections
Test Teardown Close Browser Session
*** Variables ***
${valid_username} Admin #Scalar
${valid_password} admin123 #Scalar
${url} https://opensource-demo.orangehrmlive.com/web/index.php/auth/login #Scalar
&{VisibleElements} Forgot your password?=css:.orangehrm-login-forgot-header #Dictionary
*** Test Cases ***
Validate Elements on Login Page
Open the Browser with URL
Verify Element on Login Page
Validate Successful Login
Open the Browser with URL
Fill the login form
Verify Dashboard page opens
Verify items in Dashboard page
Verify Environment variable
*** Keywords ***
Open the Browser with URL
Create Webdriver Chrome executable_path=/Vibha_Personal/RobotFramework_Demo/drivers/chromedriver_linux64
Go To ${url}
Maximize Browser Window
Set Selenium Implicit Wait 5
Verify Element on Login Page
Element Should Be Visible ${VisibleElements}[Forgot your password?]
Fill the login form
Input Text css:input[name=username] ${valid_username}
Input Password css:input[name=password] ${valid_password}
Click Button css:.orangehrm-login-button
Verify Dashboard page opens
Element Text Should Be css:.oxd-topbar-header-breadcrumb-module Dashboard
Verify items in Dashboard page
@{expectedList} = Create List Admin PIM Leave Time Recruitment My Info Performance Dashboard Directory Maintenance Buzz #List
${elements} = Get Webelements css:.oxd-main-menu-item--name
@{actualList} = Create List
FOR ${element} IN @{elements}
LOG ${element.text}
Append To List ${actualList} ${element.text}
END
Lists Should Be Equal ${expectedList} ${actualList}
Close Browser Session
Close Browser
1. ${valid_username} Admin and ${valid_password} admin123 are both scalar variables because we have the $ sign. The variables are storing the username and password values.
2. @{expectedList} is a list variable (identified by the sign @) storing 11 elements in a list (or Array).
3. &{VisibleElements} Forgot your password?=css:.orangehrm-login-forgot-header is a Dictionary variable (identified by &) storing two locators in the form of key=value.
Execute the tests
We need the below command to run the Robot Framework script.
robot Variable_Demo.robot
The output of the above program is
View Report and Log
We have the test case passed. The Robot Framework generates log.html, output.xml, and report.html by default.
Let us now see the report and log details.
Report
Right-click on report.html. Select Open In->Browser->Firefox(any browser of your wish).
Log
Robot Framework has multiple log levels that control what is shown in the automatically generated log file. The default Robot Framework log level is INFO.
Right-click on log.html. Select Open In->Browser->Firefox(any browser of your wish).
The list elements are shown below
That’s it! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
Allure Framework is a flexible lightweight multi-language test report tool that not only shows a very concise representation of what has been tested in a neat web report form but allows everyone participating in the development process to extract maximum useful information from everyday execution of tests.
From the dev/qa perspective, Allure reports shorten common defect lifecycle: test failures can be divided into bugs and broken tests, also logs, steps, fixtures, attachments, timings, history, and integrations with TMS and bug-tracking systems can be configured, so the responsible developers and testers will have all information at hand.
Step 1.1 – Open PyCharm and create a new project. Go to File and select New Project from the main menu.
Step 1.2 – Choose the project location. Click the “Browse” button next to the Location field and specify the directory for your project.
Deselect the Create a main.py welcome script checkbox because you will create a new Python file for this tutorial.
Click on the “Create” Button.
Step 1.3 – A new dialog appears asking to open the project using any one of the given options. I have selected New Window as I like to have separate windows for each project.
Below is the image of the new project created in PyCharms.
Step 2 – Install allure-robotframework plugin
Go to the command prompt and run the below-mentioned command to download the plugin:
pip install allure-robotframework
The below image shows that the plugin is installed successfully.
Step 3 – Add allure-robotframework package to the PyCharms
Go to File->Settings ->Project:RobotFramework_Demo ->Python Interpreter.
Click on the “+” sign and enter allure-r in the search bar. It will show a list of packages. Select the “allure-robotframework” package and click on the “Install Package”.
Once the package is installed, we will see the message that the package is installed successfully.
Once the package is installed, it can be seen under the package list as shown below:
Step 4 – Create 3 new directories in the new project
Right-Click on the project, select New->Directory and provide the name as TestCases, Drivers, and Resources
We have the test case passed. The Robot Framework generates log.html, output.xml, and report.html by default.
These reports are generated in the output/robot folder as I have provided –outputdir ./output/robot in the command line.
Let us now see the report and log details.
Report
Right-click on report.html. Select Open In->Browser->Chrome(any browser of your wish).
As one of the tests purposefully failed to demonstrate how the report would look in the event of a failure, we can see that it is now a Red report. The framework generated the following report:
Log
Robot Framework has multiple log levels that control what is shown in the automatically generated log file. The default Robot Framework log level is INFO.
Right-click on log.html. Select Open In->Browser->Chrome(any browser of your wish).
Step 8 – Generate the Allure Report
To create Allure Report, use the below command
allure serve ./output/allure
This will generate the beautiful Allure Test Report as shown below.
Step 9 – Allure Report Dashboard
The overview page hosts several default widgets representing the basic characteristics of your project and test environment.
Statistics – overall report statistics.
Launches – if this report represents several test launches, statistics per launch will be shown here.
Behaviours – information on results aggregated according to stories and features.
Executors – information on test executors that were used to run the tests.
History Trend – if tests accumulated some historical data, it’s trend will be calculated and shown on the graph.
Environment – information on the test environment.
Categories in Allure Report
The categories tab gives you a way to create custom defect classifications to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).
Suites in Allure Report
On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.
Graphs in Allure Report
Graphs allow you to see different statistics collected from the test data: statuses breakdown or severity and duration diagrams.
Timeline in Allure Report
The timeline tab visualizes retrospective of tests execution, allure adaptors collect precise timings of tests, and here on this tab, they are arranged accordingly to their sequential or parallel timing structure.
Behaviours of Allure Report
This tab groups test results according to Epic, Feature, and Story tags.
Packages in Allure Report
The packages tab represents a tree-like layout of test results, grouped by different packages.
That’s it! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
A headless browser is like any other browser but without a Head/GUI (Graphical User Interface). A headless browser is used to automate the browser without launching the browser. While the tests are running, we could not see the browser, but we can see the test results coming on the console.
Headless browser testing is generally faster when compared to actual UI testing as it doesn’t wait for the whole page to render before performing any action.
The traditional way is to add –headless, and since version 96, Chrome has a new headless mode that allows users to get the full browser functionality (even run extensions). Between versions 96 to 108 it was –headless=chrome, after version 109, it is –headless=new.
The complete program looks like as shown below:
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
public class HeadlessChrome_Demo {
public ChromeDriver driver;
@Test
public void test() {
// Create an object of Chrome Options class
ChromeOptions options = new ChromeOptions();
// pass the argument -–headless and maximize to Chrome Options class.
options.addArguments("--start-maximize");
options.addArguments("--headless=new");
// Create an object of Chrome Driver class and pass the Chrome Options object as
// an argument
driver = new ChromeDriver(options);
System.out.println("Executing Chrome Driver in Headless mode..");
driver.get("https://duckduckgo.com/");
String titlePage = driver.getTitle();
System.out.println("Title of Page :" + titlePage);
Assertions.assertEquals("DuckDuckGo — Privacy, simplified.",titlePage);
// Close the driver
driver.close();
}
}
The output of the above program is
Congratulations!! We are able to run Chrome tests in Selenium in headless mode.
This tutorial walks you through the process of running Robot Framework tests in GitLab pipelines. This is a very important step in achieving CI/CD. Ideally, tests should be run after each change (minor/major) before the latest change ismergedinto the master branch. Let’sassumethat100 changes are merged into the master branch a day and tests are run every time before deployment. In this case, there is no QA manuallystarting100tests a day. Now, what should be done to overcome this problem. Nowaddatest to your GitLab pipeline. Adding a test stage to your pipeline automatically runs the tests when you run the pipeline.
Once the proposed changes are built, then push the commits to a feature branch in a remote repository that’s hosted in GitLab. The push triggers the CI/CD pipeline for your project. Then, GitLab CI/CD runs automated scripts (sequentially or in parallel) to build as well as to test the application. After a successful run of the test scripts, GitLab CI/CD deploys your changes automatically to any environment (DEV/QA/UAT/PROD). But if the test stage is failed in the pipeline, then the deployment is stopped.
To use GitLab CI/CD, we need to keep 2 things in mind:
a) Make sure a runner is available in GitLab to run the jobs. If there is no runner, install GitLab Runner and register a runnerfor your instance, project, or group.
b) Create a .gitlab-ci.yml file at the root of the repository. This file is where CI/CD jobs are defined.
The Selenium tests run on a headless browser in the pipeline.
What is a headless browser?
A headless browser is like any other browser but without a Head/GUI (Graphical User Interface). A headless browser is used to automate the browser without launching the browser. While the tests are running, we could not see the browser, but we can see the test results coming on the console.
Prerequisite
Install Python
Install PIP
Install Robot Framework
Install Robot framework Selenium Library
Install PyCharm IDE
GitLab Account
Implementation Steps:
Step 1 – Create a new Project
Step 1.1 – Open PyCharm and create a new project. Go to File and select New Project from the main menu.
Step 1.2 – Choose the project location. Click the Browse button next to the Location field and specify the directory for your project.
Deselect the Create a main.py welcome script checkbox because you will create a new Python file for this tutorial.
Click on the Create Button.
Step 1.3 – A new dialog will appear asking to Open the project using any one of the given options. I have selected New Window as I like to have separate windows for each project.
Below is the image of the new project created in PyCharms.
Step 2 – Create 2 new directories in the new project
Right-Click on the project, select New->Directory, and provide the name as TestCases and Resources
Step 3 – Create Test Files
This directory contains multiple test case files consisting of test steps.
Right-click on the new directory select New File and provide the name LoginPageTests.robot and ForgetPasswordTests.robot as shown below:
Below is the code for LoginPageTests.robot
*** Settings ***
Documentation Tests to login to Login Page
Library SeleniumLibrary
Test Setup Open the Browser with URL
Test Teardown Close Browser Session
Resource ../Resources/GenericResources.robot
Resource ../Resources/LoginResources.robot
Resource ../Resources/DashboardResources.robot
*** Test Cases ***
Validate Unsuccessful Login using invalid credentials
LoginResources.Fill the login form ${valid_username} ${invalid_password}
LoginResources.Verify the error message is correct
Validate Unsuccessful Login for blank username
LoginResources.Fill the login form ${blank_username} ${valid_password}
LoginResources.Verify the error message is displayed for username
Validate Unsuccessful Login for blank password
LoginResources.Fill the login form ${valid_username} ${blank_password}
LoginResources.Verify the error message is displayed for password
Validate successful Login
LoginResources.Fill the login form ${valid_username} ${valid_password}
DashboardResources.Verify Dashboard page opens
Below is the code for ForgetPasswordTests.robot
*** Settings ***
Documentation Tests to validate Forgot Your Password Page functionality
Library SeleniumLibrary
Test Setup Open the Browser with URL
Test Teardown Close Browser Session
Resource ../Resources/GenericResources.robot
Resource ../Resources/LoginResources.robot
Resource ../Resources/ForgetPasswordResources.robot
*** Test Cases ***
Validate Reset Password Functionality
LoginResources.Go to Forgot Your Password Page
ForgetPasswordResources.Verify Forgot Your Password Page opens
ForgetPasswordResources.Fill the Forgot Password Page
ForgetPasswordResources.Verify the message
Validate Cancel Functionality
LoginResources.Go to Forgot Your Password Page
ForgetPasswordResources.Verify Forgot Your Password Page opens
ForgetPasswordResources.Cancel the Reset Password
ForgetPasswordResources.Verify that Login Page is displayed
Step 4 – Create a Resource file for each page
It maintains the files which contain page elements as well as corresponding keywords.
Right-click on the new directory select New File and provide the name as LoginResources.robot, DashboardResources.robot, GenericResources.robot, and ForgetPasswordResources.robot as shown below:
GenericResources.robot contains the keywords that are common to all the tests, like opening of the browser or closing of the browser.
*** Settings ***
Documentation A resource file with reusable keywords and variables.
Library SeleniumLibrary
*** Variables ***
${valid_username} Admin
${valid_password} admin123
${invalid_username} 1234
${invalid_password} 45678
${blank_username}
${blank_password}
${url} https://opensource-demo.orangehrmlive.com/web/index.php/auth/login
${browser_name} Chrome
*** Keywords ***
Open the Browser with URL
Open Browser ${url} headlesschrome
Maximize Browser Window
Set Selenium Implicit Wait 5
Close Browser Session
Close Browser
Below is the code for LoginResources.robot
*** Settings ***
Documentation All the page objects and keywords of landing page
Library SeleniumLibrary
*** Variables ***
${login_error_message} css:.oxd-alert-content--error
${dashboard_title} css:.oxd-topbar-header-breadcrumb-module
${missing_username_error_message} xpath://*[@class='oxd-form']/div[1]/div/span
${missing_password_error_message} xpath://*[@class='oxd-form']/div[2]/div/span
${forgot_password_link} xpath://div[@class='orangehrm-login-forgot']/p
*** Keywords ***
Fill the login form
[Arguments] ${username} ${password}
Input Text css:input[name=username] ${username}
Input Password css:input[name=password] ${password}
Click Button css:.orangehrm-login-button
Verify the error message is correct
Element Text Should Be ${login_error_message} Invalid credentials
Verify the error message is displayed for username
Element Text Should Be ${missing_username_error_message} Required
Verify the error message is displayed for password
Element Text Should Be ${missing_password_error_message} Required
Go to Forgot Your Password Page
Click Element ${forgot_password_link}
Below is the code for DashboardResources.robot
*** Settings ***
Documentation All the page objects and keywords of Dashboard page
Library SeleniumLibrary
*** Variables ***
${dashboard_title} css:.oxd-topbar-header-breadcrumb-module
*** Keywords ***
Verify Dashboard page opens
Element Text Should Be ${dashboard_title} Dashboard
Below is the code for ForgetPasswordResources.robot
*** Settings ***
Documentation All the page objects and keywords of Forget Password page
Library SeleniumLibrary
*** Variables ***
${forgot_page_title} css:.orangehrm-forgot-password-title
${username} css:.oxd-input--active
${reset_btn} css:.orangehrm-forgot-password-button--reset
${cancel_btn} css:.orangehrm-forgot-password-button--cancel
${reset_message} xpath://div[@class='orangehrm-card-container']/h6
${login_page_title} xpath://*[@class='orangehrm-login-slot']/h5
*** Keywords ***
Verify Forgot Your Password Page opens
Element Text Should Be ${forgot_page_title} Reset Password
Fill the Forgot Password Page
Input Text ${username} abc@gmail.com
Click Button ${reset_btn}
Verify the message
Element Text Should Be ${reset_message} Reset Password link sent successfully
Cancel the Reset Password
Click Button ${cancel_btn}
Verify that Login Page is displayed
Element Text Should Be ${login_page_title} Login
All the below-mentioned keywords are derived from SeleniumLibrary. The functionality of keywords mentioned above:
1. Create Webdriver − The keyword creates an instance of Selenium WebDriver.
2. Go To – This keyword navigates the current browser window to the provided url.
3. Maximize Browser Window – This keyword maximizes the current browser window.
4. Set Selenium Implicit Wait – This keyword sets the implicit wait value used by Selenium.
5. Input Text − This keyword is used to type the given text in the specified textbox identified by the locator name:username.
6. Input Password – This keyword is used to type the given text in the specified password identified by the locator name:password.
The difference compared to Input Text is that this keyword does not log the given password on the INFO level.
7. Click button – This keyword is used to click the button identified by the locator. In this case, it is “Login” button.
8. Element Text Should Be – This keyword is used to verify that the current page contains the exact text identified by the locator. Here, we are checking the exact text “Invalid Credentials”.
To run this script, go to the command line and go to directory tests.
Step 5 – Execute the tests
We need the below command to run the Robot Framework script.
robot .
The output of the above program is
GitLab Section
Step 6 – Create a blank project in GitLab
To know, how to create a blank new project in GitLab, please refer tothis tutorial.
Step 7 – Push the project from the local repository to the Gitlab Repository
To know, how to push the changes in GitLab, please refer to this tutorial.
Step 8 – Create a .gitlab-ci.yml file in the project in GitLab
There are many ways to create a new file in GitLab. One of the ways is to create a file as shown in the below image.
It is a YAML file where you configure specific instructions for GitLab CI/CD. In the .gitlab-ci.yml, we can define:
The scripts you want to run.
Other configuration files and templates you want to include.
Dependencies and caches.
The commands you want to run in sequence and those you want to run in parallel.
The location to deploy your application to.
Whether you want to run the scripts automatically or trigger any of them manually.
Below is the sample pipeline.
stages:
- test
image: ppandiyan/robotframework
run-tests:
stage: test
script:
- robot --outputdir testOutput .
artifacts:
paths:
- testOutput
expire_in: 1 day
Image – ppandiyan/robotframework is used in this test. This docker file contains headlesschrome, headlessfirefox and Python2.7.
This gitlab-ci.yml has only 1 stage – a test that contains the command to run the tests as well as also create an artifact that contains all the surefire reports which can be saved as Test Evidence.
Step 9 – Run the tests in the GitLab pipeline
Now, when a new change is committed, a pipeline kicks off and it runs all the tests.
Step 10 – Check the status of the pipeline
Once the Status of the pipeline changes to either failed or passed that means the tests are already executed.
Let us see the logs of the execution it shows that all the tests are passed here.
As I have added an artifact also in the gitalb-ci.yml, which is highlighted in the image. This artifact creates a folder with the name “testOutput” and the folder contains reports, logs as well as output files. This artifact gives us the option to download the reports or browse the report. This report will be available for 1 day only as mentioned in the gitlab-ci.yml.
Step 11 – Download the report
Once, will click on the download button, it will download “report.zip”. Unzip the folder and it looks like something as shown below:
Example of Report.html
Example of Log.html
Congratulations. This tutorial has explained the steps to run Robot Framework tests in GitLab CI/CD. Happy Learning!!
RequestLibrary is a Robot Framework library aimed to provide HTTP API testing functionalities by wrapping the well-known Python Requests Library.
Implementation Steps:
Step 1 – Create a new project
Step 1.1 – Open PyCharm and create a new project. Go to File and select New Project from the main menu.
Step 1.2 – Choose the project location. Click the “Browse” button next to the Location field and specify the directory for your project.
Deselect the Create a main.py welcome script checkbox because you will create a new Python file for this tutorial.
Click on the “Create” Button.
Step 1.3 – A new dialog appears asking to open the project using any one of the given options. I have selected New Window as I like to have separate windows for each project.
Below is the image of the new project created in PyCharms.
Step 2 – Install RequestLibrary
To install RequestLibrary, you need to use the below command:
pip install robotframework-requests
Step 3 – Add robotframework-requests package to the PyCharms
Go to File->Settings ->Project:RobotFramework_Demo ->Python Interpreter.
Click on the “+” sign and enter pabot in the search bar. It will show a list of packages. Select the “robotframework-requests” package and click on the “Install Package”.
Once the package is installed, we will see the message that the package is installed successfully.
Once the package is installed, it can be seen under the package list as shown below:
Step 4 – Create a new directory in the new project
Right-Click on the project, select New->Directory, and provide the name as API_Tests
Below is the image of the new directory.
Right-click on the new directory and select New File and provide the name as API_Demo.robot as shown below:
Step 5 – Create API tests in Robot Framework
GET Request
The corresponding API Test in Robot Framework.
*** Settings ***
Library RequestsLibrary
Library Collections
*** Variables ***
${base_url} https://reqres.in/api/users
${page_id} 2
*** Test Cases ***
Quick Get Request Test
${response}= GET ${base_url} params=page=${page_id} expected_status=200
log ${response.json()}
Should Be Equal As Strings 6 ${response.json()}[per_page]
Should Be Equal As Strings 12 ${response.json()}[total]
Should Be Equal As Strings 7 ${response.json()}[data][0][id]
The corresponding log of this request is
${response}= We are saving the response of the GET operation in the ${response} variable.
The response is logged in the log by using the below command:
log ${response.json()}
POST Request
The corresponding API Test in Robot Framework.
*** Settings ***
Library RequestsLibrary
Library Collections
*** Variables ***
${base_url} https://reqres.in/api/users
${page_id} 2
${expectedname} test
${expectedjob} team leader
*** Test Cases ***
Quick POST Request Test
&{req_body}= Create Dictionary name=test job=team leader
${response}= POST ${base_url} json=${req_body} expected_status=201
log ${response.json()}
Dictionary Should Contain Key ${response.json()} id
${name}= Get From Dictionary ${response.json()} name
Should Be Equal As Strings ${expectedname} ${name}
${job}= Get From Dictionary ${response.json()} job
Should Be Equal As Strings ${expectedjob} ${job}
The corresponding log of this request is
1.Create Dictionary – This keyword is used from BuiltIn Library. It creates and returns a dictionary based on the given Items. Items are typically given using the key=value syntax.
2. Dictionary should contain key – This keyword is used from Collections library. It is used to verify if the specified key is present in the dictionary.
3. Get From Dictionary – This keyword is used from the Collections library. It is used to return a value from the given dictionary based on the given key. Here, it is used to check the values of the key name and job.
PUT Request
The corresponding API Test in Robot Framework.
*** Settings ***
Library RequestsLibrary
Library Collections
*** Variables ***
${base_url} https://reqres.in/api/users
${updated_expectedname} update_test
${updated_expectedjob} resident
*** Test Cases ***
Quick PUT Request Test
&{req_body}= Create Dictionary name=update_test job=resident
${response}= PUT ${base_url}+/2 json=${req_body} expected_status=200
log ${response.json()}
Dictionary Should Contain Key ${response.json()} name
${name}= Get From Dictionary ${response.json()} name
Should Be Equal As Strings ${updated_expectedname} ${name}
Dictionary Should Contain Key ${response.json()} job
${job}= Get From Dictionary ${response.json()} job
Should Be Equal As Strings ${updated_expectedjob} ${job}
We need the below command to run the Robot Framework script.
robot API_DEMO.robot
The output of the above program is
Step 7 – View Report and Log
We have the test case passed. The Robot Framework generates log.html, output.xml, and report.html by default.
Let us now see the report and log details.
Report
Right-click on report.html. Select Open In->Browser->Chrome(any browser of your wish).
The Report generated by the framework is shown below:
Log
Robot Framework has multiple log levels that control what is shown in the automatically generated log file. The default Robot Framework log level is INFO.
Right-click on log.html. Select Open In->Browser->Chrome(any browser of your wish).
That’s it! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
When running your Jenkins automation jobs, you may need to pass some parameters to your scripts. These parameters can be a URL, a browser name, or even the test user’s credentials. This tutorial will walk you through the process of creating a parameterized Jenkins job for Robot Framework.
It is recommended to go through these tutorials, before creating a parameterized Jenkins job:-
Jenkins supports several parameter types. Below is a list of the most common ones, but keep in mind that different plugins may add new parameter types:
String: any combination of characters and numbers
Choice: a pre-defined set of strings from which a user can pick a value
Credentials: a pre-defined Jenkins credential
File: the full path to a file on the filesystem
Multi-line String: same as String, but allows newline characters
Password: similar to the Credentials type, but allows us to pass a plain text parameter specific to the job or pipeline
Run: an absolute URL to a single run of another job
Implementation Steps
Step 1: Create a new FreeStyle project
Give the Name of the project – RobotFramework_Demo.
Click on the FreeStyle project.
Click on the OK button.
Step 2: Description of the project
In the General section, enter the project description in the Description box.
Check the option – This project is parameterized.
Click on Add Parameter, and we can select any option as shown in the above image.
Step 3: Select the Choice Parameter
Choice Parameter
As I want to run my tests on different browsers, I have selected Choice Parameter.
Give the Name of the Parameter.
Give the Choices of the Parameter – Chrome, Firefox, IE
Give the Description of the Parameter – Select any browser to run the tests (optional).
Step 4: Select a custom workspace
Mention the full path of the project in the Use custom workspace.
Step 5: Source Code Management
In the Source Code Management section, select None.
Step 6: Build Management
Go to the Build section of the new job. Select “Execute Windows batch command”.
Mention the command needed to execute the tests. In this case, I have to execute all the tests, so used the below command:
Step 7: Select “HTML Reports” from “Post Build Actions“
Scroll down to “Post Build Actions” and click on the “Add Post Build Actions” drop-down list.
Select “Publish HTML Reports“.
Enter the HTML directory to archive – Empty, Index page[s] – report.html, and Report title – HTML Report.
Click on the Apply and Save buttons.
If you want to see where the report is saved in Jenkins, go to the Dashboard -> RobotFramework_Demo -> Workspace ->report.html.
We have created a new project “RobotFramework_Demo“.
Step 8: Execute the tests
Let’s execute it now by clicking on the “Build with Parameters” button.
This screen contains the parameters which we have to select. The browser was a choice parameter and selecting a parameter from it and Tags was also a Choice parameter, so mention the tags in it and click on the “Build” button.
Right-click on Build Number (here in my case it is #11).
Click on Console Output to see the result.
Step 9: View the HTML Report
Once the execution is completed, click on go “Back to Project“, and we could see a link to view the “HTML Report“.
Click on the HTML Report. It displays the summary of the tests.
Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!
QA Automation Expert 