GitLab automatically enables CI/CD pipelines for new projects. It’s just a matter of adding a new configuration file called .gitlab-ci.yml to your code repository with instructions for GitLab on what to run. So simply create the following basic workflow in your main repository directory and commit it.
Step 3 – Create a .gitlab-ci.yml file in the project in GitLab
There are many ways to create a new file in GitLab. One of the ways is to create a file as shown in the below image.
It is a YAML file where you configure specific instructions for GitLab CI/CD. In the.gitlab-ci.yml, we can define:
The scripts you want to run.
Other configuration files and templates you want to include.
Dependencies and caches.
The commands you want to run in sequence and those you want to run in parallel.
The location to deploy your application to.
Whether you want to run the scripts automatically or trigger any of them manually.
Below is a sample example to run the Gradle tests in the GitLab pipeline.
image : gradle:7.1.0-jdk11
stages:
- test
variables:
GRADLE_OPTS: "-Dorg.gradle.daemon=false"
test:
stage: test
allow_failure: true
# Run the tests
script:
- echo "Executing BDD scenarios with Gradle"
- gradle clean test
- gradle reports
# Store artifacts
artifacts:
when: always
name: "Gradle Report"
paths:
- lib/target/site/serenity/*
expire_in: 24 h
Image – gradle:7.1.0-jdk11 is used in this test. It is a docker image for Gradle with Java 11 installed in it.
Pipeline configuration begins with jobs. Jobs are the most fundamental element of a .gitlab-ci.yml file.
Jobs are:
Defined with constraints stating under what conditions they should be executed.
Top-level elements with an arbitrary name and must contain at least the script clause.
Not limited in how many can be defined.
Jobs can output an archive of files and directories. This output is known as a job artifact. The expire_in keyword determines how long GitLab keeps the job artifacts. Here, it shows 24 hrs to retain the artifacts.
Step 4 – View GitLab Pipeline
Now, when a new change is committed, a pipeline kicks off and it runs all the tests. To view the pipeline, go to the left panel and click on the Build option. There are a number of sub-options in the Build option, click on the Pipelines.
Step 5 – Run the tests in the GitLab pipeline
The below image shows that the tests are running in the GitLab pipeline.
Step 6 – Check the status of the pipeline
Once the Status of the pipeline changes to either failed or passed, that means the tests are already executed.
As you can see, the Status is passed, its green colour. This means all the tests present in the test suite are executed and passed. If any test fails in the test suite, the final execution status will be brown. The reason for the brown colour is we have mentioned allow_failure: true.
Below shows the execution status report in the GitLab pipeline.
As I have added an artifact also in the .gitalb-ci.yml, which is highlighted in the image. This artifact creates a folder with the name “Serenity_Report”and the reports in this folder come from the path /target/site. This artifact gives us the option to download the reports or browse the report. This report will be available for 24 hours only as mentioned in the gitlab-ci.yml.
Step 7 – Download the report
Once, will click on the download button, it will download “Gradle_Report.zip”. Unzip the folder and it looks like something as shown below:
Example of Index.html
Example of Serenity Summary Report
Congratulations. This tutorial has explained the steps to run Serenity tests in GitLab CI/CD. Happy Learning!!
GitLab automatically enables CI/CD pipelines for new projects. It’s just a matter of adding a new configuration file called .gitlab-ci.yml to your code repository with instructions for GitLab on what to run. So simply create the following basic workflow in your main repository directory and commit it:
The Serenity tests run on a headless browser in the pipeline.
What is a headless browser?
A headless browser is like any other browser but without a Head/GUI (Graphical User Interface). A headless browser is used to automate the browser without launching the browser. While the tests are running, we could not see the browser, but we can see the test results coming on the console. The tests can run in the GitLab pipeline if the tests run in the headless mode.
To know, how to create a blank new project in GitLab, please refer tothis tutorial.
Step 2 – Push the project from the local repository to GitLab Repository
To know, how to push the changes in GitLab, please refer to this tutorial.
Step 3 – Create a .gitlab-ci.yml file in the project in GitLab
There are many ways to create a new file in GitLab. One of the ways is to create a file as shown in the below image.
It is a YAML file where you configure specific instructions for GitLab CI/CD. In the .gitlab-ci.yml, we can define:
The scripts you want to run.
Other configuration files and templates you want to include.
Dependencies and caches.
The commands you want to run in sequence and those you want to run in parallel.
The location to deploy your application to.
Whether you want to run the scripts automatically or trigger any of them manually.
Below is a sample example to run the Serenity tests in the GitLab pipeline.
image: markhobson/maven-chrome
stages:
- test
variables:
MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
test:
stage: test
allow_failure: true
# Run the tests
script:
- echo "Executing BDD scenarios with maven"
- mvn clean verify
# Store artifacts
artifacts:
when: always
name: "Serenity Report"
paths:
- target/site/*
expire_in: 24 h
Image – markhobson/maven-chrome is used in this test. It is a docker image for Java automated UI tests.
This command lets the pipeline continue running subsequent jobs, even if the previous job is failed.
allow_failure: true
The below command is used to execute the tests from Maven in the docker image.
mvn clean verify
The below command means that the artifact should be generated every time the pipeline is run, irrespective of the fact if the job is successful or failed.
artifacts:
when: always
The below command provides the name of the artifact. If this is not used, then the artifacts file is named artifacts, which becomes artifacts.zip when downloaded.
artifacts:
name: "Serenity Report"
The below command provides the path of the files that should be present in the artifact.
artifacts:
paths:
- target/site/*
Pipeline configuration begins with jobs. Jobs are the most fundamental element of a .gitlab-ci.yml file.
Jobs are:
Defined with constraints stating under what conditions they should be executed.
Top-level elements with an arbitrary name and must contain at least the script clause.
Not limited in how many can be defined.
Jobs can output an archive of files and directories. This output is known as a job artifact. The expire_in keyword determines how long GitLab keeps the job artifacts. Here, it shows 24 hrs to retain the artifacts.
Step 4 – Run the tests in the GitLab pipeline
Now, when a new change is committed, a pipeline kicks off and it runs all the tests.
Step 5 – Check the status of the pipeline
Once the Status of the pipeline changes to either failed or passed, that means the tests are already executed.
As you can see, the Status is passed, its green colour. This means all the tests present in the test suite are executed and passed. If any test fails in the test suite, the final execution status will be brown. The reason for the brown colour is we have mentioned allow_failure: true.
Below is the execution status report in the GitLab pipeline.
As I have added an artifact also in the .gitalb-ci.yml, which is highlighted in the image. This artifact creates a folder with the name “Serenity_Report”and the reports in this folder come from the path /target/site. This artifact gives us the option to download the reports or browse the report. This report will be available for 24 hours only as mentioned in the gitlab-ci.yml.
Step 6 – Download the report
Once, will click on the download button, it will download “Serenity_Report.zip”. Unzip the folder and it looks like something as shown below:
Example of Index.html
Example of Serenity Summary Report
Congratulations. This tutorial has explained the steps to run Serenity tests in GitLab CI/CD. Happy Learning!!
A headless browser is like any other browser but without a Head/GUI (Graphical User Interface). A headless browser is used to automate the browser without launching the browser. While the tests are running, we could not see the browser, but we can see the test results coming on the console.
Headless browser testing is generally faster when compared to actual UI testing as it doesn’t wait for the whole page to render before performing any action.
The traditional way is to add –headless, and since version 96, Chrome has a new headless mode that allows users to get the full browser functionality (even run extensions). Between versions 96 to 108 it was –headless=chrome, after version 109, it is –headless=new.
The complete program looks like as shown below:
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
public class HeadlessChrome_Demo {
public ChromeDriver driver;
@Test
public void test() {
// Create an object of Chrome Options class
ChromeOptions options = new ChromeOptions();
// pass the argument -–headless and maximize to Chrome Options class.
options.addArguments("--start-maximize");
options.addArguments("--headless=new");
// Create an object of Chrome Driver class and pass the Chrome Options object as
// an argument
driver = new ChromeDriver(options);
System.out.println("Executing Chrome Driver in Headless mode..");
driver.get("https://duckduckgo.com/");
String titlePage = driver.getTitle();
System.out.println("Title of Page :" + titlePage);
Assertions.assertEquals("DuckDuckGo — Privacy, simplified.",titlePage);
// Close the driver
driver.close();
}
}
The output of the above program is
Congratulations!! We are able to run Chrome tests in Selenium in headless mode.
This tutorial walks you through the process of running Robot Framework tests in GitLab pipelines. This is a very important step in achieving CI/CD. Ideally, tests should be run after each change (minor/major) before the latest change ismergedinto the master branch. Let’sassumethat100 changes are merged into the master branch a day and tests are run every time before deployment. In this case, there is no QA manuallystarting100tests a day. Now, what should be done to overcome this problem. Nowaddatest to your GitLab pipeline. Adding a test stage to your pipeline automatically runs the tests when you run the pipeline.
Once the proposed changes are built, then push the commits to a feature branch in a remote repository that’s hosted in GitLab. The push triggers the CI/CD pipeline for your project. Then, GitLab CI/CD runs automated scripts (sequentially or in parallel) to build as well as to test the application. After a successful run of the test scripts, GitLab CI/CD deploys your changes automatically to any environment (DEV/QA/UAT/PROD). But if the test stage is failed in the pipeline, then the deployment is stopped.
To use GitLab CI/CD, we need to keep 2 things in mind:
a) Make sure a runner is available in GitLab to run the jobs. If there is no runner, install GitLab Runner and register a runnerfor your instance, project, or group.
b) Create a .gitlab-ci.yml file at the root of the repository. This file is where CI/CD jobs are defined.
The Selenium tests run on a headless browser in the pipeline.
What is a headless browser?
A headless browser is like any other browser but without a Head/GUI (Graphical User Interface). A headless browser is used to automate the browser without launching the browser. While the tests are running, we could not see the browser, but we can see the test results coming on the console.
Prerequisite
Install Python
Install PIP
Install Robot Framework
Install Robot framework Selenium Library
Install PyCharm IDE
GitLab Account
Implementation Steps:
Step 1 – Create a new Project
Step 1.1 – Open PyCharm and create a new project. Go to File and select New Project from the main menu.
Step 1.2 – Choose the project location. Click the Browse button next to the Location field and specify the directory for your project.
Deselect the Create a main.py welcome script checkbox because you will create a new Python file for this tutorial.
Click on the Create Button.
Step 1.3 – A new dialog will appear asking to Open the project using any one of the given options. I have selected New Window as I like to have separate windows for each project.
Below is the image of the new project created in PyCharms.
Step 2 – Create 2 new directories in the new project
Right-Click on the project, select New->Directory, and provide the name as TestCases and Resources
Step 3 – Create Test Files
This directory contains multiple test case files consisting of test steps.
Right-click on the new directory select New File and provide the name LoginPageTests.robot and ForgetPasswordTests.robot as shown below:
Below is the code for LoginPageTests.robot
*** Settings ***
Documentation Tests to login to Login Page
Library SeleniumLibrary
Test Setup Open the Browser with URL
Test Teardown Close Browser Session
Resource ../Resources/GenericResources.robot
Resource ../Resources/LoginResources.robot
Resource ../Resources/DashboardResources.robot
*** Test Cases ***
Validate Unsuccessful Login using invalid credentials
LoginResources.Fill the login form ${valid_username} ${invalid_password}
LoginResources.Verify the error message is correct
Validate Unsuccessful Login for blank username
LoginResources.Fill the login form ${blank_username} ${valid_password}
LoginResources.Verify the error message is displayed for username
Validate Unsuccessful Login for blank password
LoginResources.Fill the login form ${valid_username} ${blank_password}
LoginResources.Verify the error message is displayed for password
Validate successful Login
LoginResources.Fill the login form ${valid_username} ${valid_password}
DashboardResources.Verify Dashboard page opens
Below is the code for ForgetPasswordTests.robot
*** Settings ***
Documentation Tests to validate Forgot Your Password Page functionality
Library SeleniumLibrary
Test Setup Open the Browser with URL
Test Teardown Close Browser Session
Resource ../Resources/GenericResources.robot
Resource ../Resources/LoginResources.robot
Resource ../Resources/ForgetPasswordResources.robot
*** Test Cases ***
Validate Reset Password Functionality
LoginResources.Go to Forgot Your Password Page
ForgetPasswordResources.Verify Forgot Your Password Page opens
ForgetPasswordResources.Fill the Forgot Password Page
ForgetPasswordResources.Verify the message
Validate Cancel Functionality
LoginResources.Go to Forgot Your Password Page
ForgetPasswordResources.Verify Forgot Your Password Page opens
ForgetPasswordResources.Cancel the Reset Password
ForgetPasswordResources.Verify that Login Page is displayed
Step 4 – Create a Resource file for each page
It maintains the files which contain page elements as well as corresponding keywords.
Right-click on the new directory select New File and provide the name as LoginResources.robot, DashboardResources.robot, GenericResources.robot, and ForgetPasswordResources.robot as shown below:
GenericResources.robot contains the keywords that are common to all the tests, like opening of the browser or closing of the browser.
*** Settings ***
Documentation A resource file with reusable keywords and variables.
Library SeleniumLibrary
*** Variables ***
${valid_username} Admin
${valid_password} admin123
${invalid_username} 1234
${invalid_password} 45678
${blank_username}
${blank_password}
${url} https://opensource-demo.orangehrmlive.com/web/index.php/auth/login
${browser_name} Chrome
*** Keywords ***
Open the Browser with URL
Open Browser ${url} headlesschrome
Maximize Browser Window
Set Selenium Implicit Wait 5
Close Browser Session
Close Browser
Below is the code for LoginResources.robot
*** Settings ***
Documentation All the page objects and keywords of landing page
Library SeleniumLibrary
*** Variables ***
${login_error_message} css:.oxd-alert-content--error
${dashboard_title} css:.oxd-topbar-header-breadcrumb-module
${missing_username_error_message} xpath://*[@class='oxd-form']/div[1]/div/span
${missing_password_error_message} xpath://*[@class='oxd-form']/div[2]/div/span
${forgot_password_link} xpath://div[@class='orangehrm-login-forgot']/p
*** Keywords ***
Fill the login form
[Arguments] ${username} ${password}
Input Text css:input[name=username] ${username}
Input Password css:input[name=password] ${password}
Click Button css:.orangehrm-login-button
Verify the error message is correct
Element Text Should Be ${login_error_message} Invalid credentials
Verify the error message is displayed for username
Element Text Should Be ${missing_username_error_message} Required
Verify the error message is displayed for password
Element Text Should Be ${missing_password_error_message} Required
Go to Forgot Your Password Page
Click Element ${forgot_password_link}
Below is the code for DashboardResources.robot
*** Settings ***
Documentation All the page objects and keywords of Dashboard page
Library SeleniumLibrary
*** Variables ***
${dashboard_title} css:.oxd-topbar-header-breadcrumb-module
*** Keywords ***
Verify Dashboard page opens
Element Text Should Be ${dashboard_title} Dashboard
Below is the code for ForgetPasswordResources.robot
*** Settings ***
Documentation All the page objects and keywords of Forget Password page
Library SeleniumLibrary
*** Variables ***
${forgot_page_title} css:.orangehrm-forgot-password-title
${username} css:.oxd-input--active
${reset_btn} css:.orangehrm-forgot-password-button--reset
${cancel_btn} css:.orangehrm-forgot-password-button--cancel
${reset_message} xpath://div[@class='orangehrm-card-container']/h6
${login_page_title} xpath://*[@class='orangehrm-login-slot']/h5
*** Keywords ***
Verify Forgot Your Password Page opens
Element Text Should Be ${forgot_page_title} Reset Password
Fill the Forgot Password Page
Input Text ${username} abc@gmail.com
Click Button ${reset_btn}
Verify the message
Element Text Should Be ${reset_message} Reset Password link sent successfully
Cancel the Reset Password
Click Button ${cancel_btn}
Verify that Login Page is displayed
Element Text Should Be ${login_page_title} Login
All the below-mentioned keywords are derived from SeleniumLibrary. The functionality of keywords mentioned above:
1. Create Webdriver − The keyword creates an instance of Selenium WebDriver.
2. Go To – This keyword navigates the current browser window to the provided url.
3. Maximize Browser Window – This keyword maximizes the current browser window.
4. Set Selenium Implicit Wait – This keyword sets the implicit wait value used by Selenium.
5. Input Text − This keyword is used to type the given text in the specified textbox identified by the locator name:username.
6. Input Password – This keyword is used to type the given text in the specified password identified by the locator name:password.
The difference compared to Input Text is that this keyword does not log the given password on the INFO level.
7. Click button – This keyword is used to click the button identified by the locator. In this case, it is “Login” button.
8. Element Text Should Be – This keyword is used to verify that the current page contains the exact text identified by the locator. Here, we are checking the exact text “Invalid Credentials”.
To run this script, go to the command line and go to directory tests.
Step 5 – Execute the tests
We need the below command to run the Robot Framework script.
robot .
The output of the above program is
GitLab Section
Step 6 – Create a blank project in GitLab
To know, how to create a blank new project in GitLab, please refer tothis tutorial.
Step 7 – Push the project from the local repository to the Gitlab Repository
To know, how to push the changes in GitLab, please refer to this tutorial.
Step 8 – Create a .gitlab-ci.yml file in the project in GitLab
There are many ways to create a new file in GitLab. One of the ways is to create a file as shown in the below image.
It is a YAML file where you configure specific instructions for GitLab CI/CD. In the .gitlab-ci.yml, we can define:
The scripts you want to run.
Other configuration files and templates you want to include.
Dependencies and caches.
The commands you want to run in sequence and those you want to run in parallel.
The location to deploy your application to.
Whether you want to run the scripts automatically or trigger any of them manually.
Below is the sample pipeline.
stages:
- test
image: ppandiyan/robotframework
run-tests:
stage: test
script:
- robot --outputdir testOutput .
artifacts:
paths:
- testOutput
expire_in: 1 day
Image – ppandiyan/robotframework is used in this test. This docker file contains headlesschrome, headlessfirefox and Python2.7.
This gitlab-ci.yml has only 1 stage – a test that contains the command to run the tests as well as also create an artifact that contains all the surefire reports which can be saved as Test Evidence.
Step 9 – Run the tests in the GitLab pipeline
Now, when a new change is committed, a pipeline kicks off and it runs all the tests.
Step 10 – Check the status of the pipeline
Once the Status of the pipeline changes to either failed or passed that means the tests are already executed.
Let us see the logs of the execution it shows that all the tests are passed here.
As I have added an artifact also in the gitalb-ci.yml, which is highlighted in the image. This artifact creates a folder with the name “testOutput” and the folder contains reports, logs as well as output files. This artifact gives us the option to download the reports or browse the report. This report will be available for 1 day only as mentioned in the gitlab-ci.yml.
Step 11 – Download the report
Once, will click on the download button, it will download “report.zip”. Unzip the folder and it looks like something as shown below:
Example of Report.html
Example of Log.html
Congratulations. This tutorial has explained the steps to run Robot Framework tests in GitLab CI/CD. Happy Learning!!
Step 1.1 – Open PyCharm and create a new project. Go to File and select “New Project” from the main menu.
Step 1.2 – Choose the project location. Click the “Browse” button next to the Location field and specify the directory for your project.
Deselect the Create a main.py welcome script checkbox because you will create a new Python file for this tutorial.
Click on the “Create” Button.
Step 1.3 – A new dialog will appear asking to Open the project using any one of the given options. I have selected New Window as I like to have separate windows for each project.
Below is the image of the new project created in PyCharms.
How to set variable values from Runtime command?
Step 2 – Create a new directory in the new project
Right-Click on the project, select New->Directory and provide name as PageObject
Below is the image of the new directory.
Right-click on the new directory and select New File and provide the name as LoginPage.robot and ForgetPasswordPage.robot as shown below:
Step 3 – Download ChromeBinaries from the below location
The tests are going to use the Chrome browser, so we need to download the ChromeBinaries to open a blank browser in Chrome.
The chromedriver and geckodriver are placed in a folder name drivers in the RobotFramework_Demo project. I have renamed chromedriver to chrome and geckodriver to firefox.
*** Settings ***
Documentation Tests to login to Login Page
Library SeleniumLibrary
*** Variables ***
${valid_username} Admin
${valid_password} admin123
${invalid_username} 1234
${invalid_password} 45678
${blank_username}
${blank_password}
${url} https://opensource-demo.orangehrmlive.com/web/index.php/auth/login
${browser_name} Chrome
${login_error_message} css:.oxd-alert-content--error
${dashboard_title} css:.oxd-topbar-header-breadcrumb-module
${missing_username_error_message} xpath://*[@class='oxd-form']/div[1]/div/span
${missing_password_error_message} xpath://*[@class='oxd-form']/div[2]/div/span
*** Test Cases ***
Validate Unsuccessful Login using invalid credentials
[Tags] SMOKE
Open the Browser with URL
Fill the login form ${valid_username} ${invalid_password}
Verify the error message is correct
Close Browser Session
Validate Unsuccessful Login for blank username
[Tags] REGRESSION
Open the Browser with URL
Fill the login form ${blank_username} ${valid_password}
Verify the error message is displayed for username
Close Browser Session
Validate Unsuccessful Login for blank password
[Tags] SMOKE REGRESSION
Open the Browser with URL
Fill the login form ${valid_username} ${blank_password}
Verify the error message is displayed for password
Close Browser Session
Validate successful Login
[Tags] UAT
Open the Browser with URL
Fill the login form ${valid_username} ${valid_password}
Verify Dashboard page opens
Close Browser Session
*** Keywords ***
Open the Browser with URL
Create Webdriver ${browser_name} executable_path=/Vibha_Personal/RobotFramework_Demo/drivers/${browser_name}
Go To ${url}
Maximize Browser Window
Set Selenium Implicit Wait 5
Fill the login form
[Arguments] ${username} ${password}
Input Text css:input[name=username] ${username}
Input Password css:input[name=password] ${password}
Click Button css:.orangehrm-login-button
Verify the error message is correct
Element Text Should Be ${login_error_message} Invalid credentials
Verify Dashboard page opens
Element Text Should Be ${dashboard_title} Dashboard
Verify the error message is displayed for username
Element Text Should Be ${missing_username_error_message} Required
Verify the error message is displayed for password
Element Text Should Be ${missing_password_error_message} Required
Close Browser Session
Close Browser
ForgetPasswordPage.robot
*** Settings ***
Documentation Tests to validate Forgot Your Password Page functionality
Library SeleniumLibrary
*** Variables ***
${forgot_password_link} xpath://div[@class='orangehrm-login-forgot']/p
${forgot_page_title} css:.orangehrm-forgot-password-title
${username} css:.oxd-input--active
${reset_btn} css:.orangehrm-forgot-password-button--reset
${cancel_btn} css:.orangehrm-forgot-password-button--cancel
${reset_message} xpath://div[@class='orangehrm-card-container']/h6
${login_page_title} xpath://*[@class='orangehrm-login-slot']/h5
*** Test Cases ***
Validate Reset Password Functionality
Open the Browser with URL
Go to Forgot Your Password Page
Verify Forgot Your Password Page opens
Fill the Forgot Password Page
Verify the message
Close Browser Session
Validate Cancel Functionality
Open the Browser with URL
Go to Forgot Your Password Page
Verify Forgot Your Password Page opens
Cancel the Reset Password
Verify that Login Page is displayed
Close Browser Session
*** Keywords ***
Open the Browser with URL
Create Webdriver ${browser_name} executable_path=/Vibha_Personal/RobotFramework_Demo/drivers/${browser_name}
Go To ${url}
Maximize Browser Window
Set Selenium Implicit Wait 5
Go to Forgot Your Password Page
Click Element ${forgot_password_link}
Verify Forgot Your Password Page opens
Element Text Should Be ${forgot_page_title} Reset Password
Fill the Forgot Password Page
Input Text ${username} abc@gmail.com
Click Button ${reset_btn}
Verify the message
Element Text Should Be ${reset_message} Reset Password link sent successfully
Cancel the Reset Password
Click Button ${cancel_btn}
Verify that Login Page is displayed
Element Text Should Be ${login_page_title} Login
Close Browser Session
Close Browser
Step 5 – Execute various Commands
5.1. Execute all tests in a Directory
When we want to execute all the tests present in a directory, we can use the below command:
robot .
LoginPage contains 4 tests and ForgetPasswordPage contains 2 tests. The output of the above program is
5.2. Execute a test file
When we want to execute only 1 test file present in the directory, we can use the below command:
robot LoginPage.robot
LoginPage contains 4 tests. The output of the above program executes 4 tests as shown below:
5.3. Execute a test by test name
When we want to execute a test by its name, we can use the below command:
robot -t "Validate successful Login" .
The output of the above program is
5.4. Execute a test by Regular Expression
When we don’t know the complete name of the test case, but a term or functionality, we can use a regular expression to find the test. Here, we have a test with the name “Validate Cancel Functionality” in “ForgetPasswordPage.robot”. So, the regular expression will find this test and will execute.
robot -t "*Cancel*" .
The output of the above program is
That’s it! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
In this tutorial, we will rerun the failed tests in Robot Framework.
We occasionally experience rendering problems, which cause our Test Cases to fail. And we must run our complete Test Suite once more. This will be a major pain if you are working on a complicated project with many suite files or many Test Cases.
Rerunning only failed test cases after completion and providing a single report can alleviate this major pain in Robot Framework.
Robot Framework provides two attributes “–rerunfailed” and “–merge” merge will rerun the failure and merge the different results for a single report.
Step 1.1 – Open PyCharm and create a new project. Go to File and select “New Project” from the main menu.
Step 2 – Choose the project location. Click the “Browse” button next to the Location field and specify the directory for your project.
Deselect the Create a main.py welcome script checkbox because you will create a new Python file for this tutorial.
Click on the “Create” Button.
Step 1.3 – A new dialog will appear asking to Open the project using any one of the given options. I have selected New Window as I like to have separate windows for each project.
Below is the image of the new project created in PyCharms.
Step 2 – Create a new directory in the new project
Right-Click on the project, select New->Directory and provide name as Tests
Below is the image of the new directory.
Right-click on the new directory and select New File and provide the name as RerunFailedTests.robot as shown below:
Step 3 – Download ChromeBinaries from the below location
The tests are going to use the Chrome browser, so we need to download the ChromeBinaries to open a blank browser in Chrome.
The chromedriver is placed in a folder name drivers in the RobotFramework_Demo project.
Step 4 – Create a simple Selenium Test
*** Settings ***
Documentation Tests to login to Login Page
Library SeleniumLibrary
*** Variables ***
${valid_username} Admin
${valid_password} admin123
${invalid_username} 1234
${invalid_password} 45678
${blank_username}
${blank_password}
${url} https://opensource-demo.orangehrmlive.com/web/index.php/auth/login
${browser_name} Chrome
${login_error_message} css:.oxd-alert-content--error
${dashboard_title} css:.oxd-topbar-header-breadcrumb-module
${missing_username_error_message} xpath://*[@class='oxd-form']/div[1]/div/span
${missing_password_error_message} xpath://*[@class='oxd-form']/div[2]/div/span
*** Test Cases ***
Validate Unsuccessful Login using invalid credentials
[Tags] SMOKE
Open the Browser with URL
Fill the login form ${valid_username} ${invalid_password}
Verify the error message is correct
Close Browser Session
Validate Unsuccessful Login for blank username
[Tags] REGRESSION
Open the Browser with URL
Fill the login form ${blank_username} ${valid_password}
Verify the error message is displayed for username
Close Browser Session
Validate Unsuccessful Login for blank password
[Tags] SMOKE REGRESSION
Open the Browser with URL
Fill the login form ${valid_username} ${blank_password}
Verify the error message is displayed for password
Close Browser Session
Validate successful Login
[Tags] UAT
Open the Browser with URL
Fill the login form ${valid_username} ${valid_password}
Verify Dashboard page opens
Close Browser Session
*** Keywords ***
Open the Browser with URL
Create Webdriver ${browser_name} executable_path=/Vibha_Personal/RobotFramework_Demo/drivers/${browser_name}
Go To ${url}
Maximize Browser Window
Set Selenium Implicit Wait 5
Fill the login form
[Arguments] ${username} ${password}
Input Text css:input[name=username] ${username}
Input Password css:input[name=password] ${password}
Click Button css:.orangehrm-login-button
Verify the error message is correct
Element Text Should Be ${login_error_message} Invalid credentials
Verify Dashboard page opens
Element Text Should Be ${dashboard_title} Dashboard
Verify the error message is displayed for username
Element Text Should Be ${missing_username_error_message} Required
Verify the error message is displayed for password
Element Text Should Be ${missing_password_error_message} Required!
Close Browser Session
Close Browser
Step 5 – Execute the tests
We need the below command to run the Robot Framework script. This is the normal execution of your test suite and save your output in output1.xml
robot --output output1.xml RerunFailedTests.robot
The output of the above program is
Let us check the output1.xml file.
We can see that Test 3 failed here.
Below is the report generated by the Robot Framework
Step 6 – Rerun the failed tests
Now, let us rerun the failed tests using the below command. I intentionally failed my test in the last run. Inthis run, I will fix the test. I will change the expected result from Required! to Required.This checks the failure results in the output1.xml file and reruns the failures once again and saves the new result in the output2.xml file
In this tutorial, we will create a project in PyCharms and create TAGS in the tests in Robot Framework using Selenium WebDriver.
The use of tags in Robot Framework is a basic yet effective mechanism for classifying test cases. Tags are free words that can be used for at least the following things:
Tags appear in test reports, logs, and, of course, test data, providing metadata to test instances.
Case-by-case statistics (total, passed, failed are automatically collected based on tags).
You can use tags to include or exclude test cases from being run.
You can use tags to indicate which test cases should be skipped.
Step 1.1 – Open PyCharm and create a new project. Go to File and select “New Project” from the main menu.
Step 1.2 – Choose the project location. Click the “Browse” button next to the Location field and specify the directory for your project.
Deselect the Create a main.py welcome script checkbox because you will create a new Python file for this tutorial.
Click on the “Create” Button.
Step 1.3 – A new dialog will appear asking to Open the project using any one of the given options. I have selected New Window as I like to have separate windows for each project.
Below is the image of the new project created in PyCharms.
How to create a tag in the test in Robot Framework?
Step 1 – Create a new directory in the new project
Right-Click on the project, select New->Directory and provide name as Tests
Below is the image of the new directory.
Right-click on the new directory and select New File and provide the name as Test_Demo.robot as shown below:
Step 2 – Download ChromeBinaries from the below location
The tests are going to use the Chrome browser, so we need to download the ChromeBinaries to open a blank browser in Chrome.
The chromedriver is placed in a folder name drivers in the RobotFramework_Demo project.
Step 3 – Create a simple Selenium Test
*** Settings ***
Documentation Tests to login to Login Page
Library SeleniumLibrary
*** Variables ***
${valid_username} Admin
${valid_password} admin123
${invalid_username} 1234
${invalid_password} 45678
${blank_username}
${blank_password}
${url} https://opensource-demo.orangehrmlive.com/web/index.php/auth/login
${browser_name} Chrome
${login_error_message} css:.oxd-alert-content--error
${dashboard_title} css:.oxd-topbar-header-breadcrumb-module
${missing_username_error_message} xpath://*[@class='oxd-form']/div[1]/div/span
${missing_password_error_message} xpath://*[@class='oxd-form']/div[2]/div/span
*** Test Cases ***
Validate Unsuccessful Login using invalid credentials
[Tags] SMOKE
Open the Browser with URL
Fill the login form ${valid_username} ${invalid_password}
Verify the error message is correct
Close Browser Session
Validate Unsuccessful Login for blank username
[Tags] REGRESSION
Open the Browser with URL
Fill the login form ${blank_username} ${valid_password}
Verify the error message is displayed for username
Close Browser Session
Validate Unsuccessful Login for blank password
[Tags] SMOKE REGRESSION
Open the Browser with URL
Fill the login form ${valid_username} ${blank_password}
Verify the error message is displayed for password
Close Browser Session
Validate successful Login
[Tags] UAT
Open the Browser with URL
Fill the login form ${valid_username} ${valid_password}
Verify Dashboard page opens
Close Browser Session
*** Keywords ***
Open the Browser with URL
Create Webdriver ${browser_name} executable_path=/Vibha_Personal/RobotFramework_Demo/drivers/${browser_ame}
Go To ${url}
Maximize Browser Window
Set Selenium Implicit Wait 5
Fill the login form
[Arguments] ${username} ${password}
Input Text css:input[name=username] ${username}
Input Password css:input[name=password] ${password}
Click Button css:.orangehrm-login-button
Verify the error message is correct
Element Text Should Be ${login_error_message} Invalid credentials
Verify Dashboard page opens
Element Text Should Be ${dashboard_title} Dashboard
Verify the error message is displayed for username
Element Text Should Be ${missing_username_error_message} Required
Verify the error message is displayed for password
Element Text Should Be ${missing_password_error_message} Required
Close Browser Session
Close Browser
Scenarios
In the above robot test file, we have a total of 4 tests.
Test 1 -> SMOKE
Test 2 -> REGRESSION
Test 3 -> SMOKE, REGRESSION
Test 4 -> UAT
1. To execute the tests tagged with UAT, we have only 1 test, so only 1 test – Test 4 will execute.
robot --include UAT .
The output of the above program is
2. To execute the tests tagged with SMOKE, we have 2 tests, so 2 tests – Test 1 and Test 3 will execute.
robot --include SMOKE .
The output of the above program is
3. To execute the tests tagged with SMOKE OR REGRESSION, we have 3 tests, so 3 tests – Test 1, Test 2, and Test 3 will execute.
robot --include SMOKEORREGRESSION .
The output of the above program is
4. To execute the tests tagged with SMOKE AND REGRESSION, we have only 1 test – Test 3 will execute.
robot --include SMOKEANDREGRESSION .
The output of the above program is
5. To execute the tests tagged with not UAT, we have a total of 4 tests and 1 test is tagged with UAT. So now 3 tests should execute – Test 1, Test 2, and Test 3.
robot --exclude UAT LoginPage.robot
The output of the above program is
That’s it! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
Step 1 – Open PyCharm and create a new project. Go to File and select “New Project” from the main menu.
Step 2 – Choose the project location. Click the “Browse” button next to the Location field and specify the directory for your project.
Deselect the Create a main.py welcome script checkbox because you will create a new Python file for this tutorial.
Python best practice is to create a virtualenv for each project. In most cases, PyCharm creates a new virtual environment automatically, and you don’t need to configure anything. But, in this case, we need to create a Base Interpreter that has the robot framework already installed. This is explained in this tutorial – How to install and setup Robot Framework for Python. To preview and modify the venv options. Expand the Python Interpreter: New Virtualenv Environment node and select a tool used to create a new virtual environment. Let’s choose Virtualenv tool, and specify the location of the environment and the base Python interpreter used for the new virtual environment.
Click on the “Create” Button.
Step 3 – A new dialog will appear asking to Open the project using any one of the given options. I have selected New Window as I like to have separate windows for each project.
Below is the image of the new project created in PyCharms.
How to create a Selenium test in Robot Framework?
Step 1 – Create a new directory in the new project
Right-Click on the project, select New->Directory and provide name as Tests
Below is the image of the new directory.
Right-click on the new directory and select New File and provide the name as Test_Demo.robot as shown below:
Step 2 – Download ChromeBinaries from the below location
The tests are going to use the Chrome browser, so we need to download the ChromeBinaries to open a blank browser in Chrome.
*** Settings ***
Documentation To validate the Login Form
Library SeleniumLibrary
*** Test Cases ***
Validate Unsuccessful Login
Open the Browser with URL
Fill the login form
Verify error message is correct
*** Keywords ***
Open the Browser with URL
Create Webdriver Chrome executable_path=/Vibha_Personal/RobotFramework_Demo/drivers/chromedriver_linux64
Go To https://opensource-demo.orangehrmlive.com/web/index.php/auth/login
Maximize Browser Window
Set Selenium Implicit Wait 5
Fill the login form
Input Text css:input[name=username] Admin
Input Password css:input[name=password] Admin
Click Button css:.orangehrm-login-button
Verify error message is correct
${result}= Get Text CSS:.oxd-alert-content-text
Should Be Equal As Strings ${result} Invalid credentials
The chromedriver is placed in a folder name drivers in RobotFramework_Demo project.
I will explain the terms Settings, Test Cases, Keywords in the next tutorial.
To run this script, go to the command line and go to directory tests.
We need the below command to run the Robot Framework script.
robot Test_Demo.robot
The output of the above program is
We have the test case passed. The Robot Framework generates log.html, output.xml, and report.html by default.
Let us now see the report and log details.
Report
Right-click on report.html. Select Open In->Browser->Chrome(any browser of your wish).
The Report generated by the framework is shown below:
Log
Robot Framework has multiple log levels that control what is shown in the automatically generated log file. The default Robot Framework log level is INFO.
Right-click on log.html. Select Open In->Browser->Chrome(any browser of your wish).
That’s it! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
Robot Framework is a generic open-source automation framework. It can be used for test automation and robotic process automation (RPA). RPA is extensively used for Web Application Automation, API Automation, RPA, and Database Testing.
Robot Framework has an easy syntax, utilizing human-readable keywords. Its capabilities can be extended by libraries implemented with Python, Java, or many other programming languages.
Jenkins is a well-known open-source tool that aids in the implementation of Continuous Integration (CI) and Continuous Deployment/Continuous Delivery (CD) processes by automating parts of the software development pipeline such as building, testing, and deployment.
What is Selenium?
Selenium is an open-source automation tool that has been widely used for testing web applications. It is basically a suite of software or a set of JAR files to automate web browsers across various browsers. Selenium is used by many companies, but a few to mention are Netflix, Google, HubSpot, Fitbit, and more. Selenium supports multiple languages like Python, Pearl, Ruby, .Net, Java, C#, PHP.
Selenium supports parallel test execution. That means we can open multiple browsers at the same time and execute the test cases. This decrease the execution time significantly. Selenium supports Cross browser test execution.
QA Automation Expert 