In the previous tutorial, I explained how to mention Manual Test Cases in Serenity Report. In this tutorial, I will explain how to attach test evidence to manual tests. Before this tutorial, I suggest you to refer the tutorial which explain How to generate Serenity Report.
It is always advisable to attach screenshots or other files to our manual test reports as additional evidence, specially for failed scenarios. The @manual-test-evidence tag allows you to do just this. You can either include a link to an external site, as shown here:
Mentioning the path of evidence in the test is not a very good way to attach test evidence to the manual tests. An alternative approach and favorable one is to place the image in the src/test/resources/assets folder and include a relative link to this file (starting with “assets/“):
@manual
@manual-result:failed
@manual-last-tested:sprint-1
@manual-test-evidence:assets/DB1.PNG
Scenario: Verify different credentials are provided to Admin, Dev and QA to access Master Database
Given User is connected to Master Database
Then Different credentials are provided to Admin, Business, Dev and QA to access Master Database
Test Evidence is only displayed if the @manual-last-tested annotation is defined in serenity.properties.
serenity.project.name = Serenity and Cucumber Report Demo
current.target.version = sprint-1
Execute the test suite by using the below command
mvn clean verify
As we the the Serenity Reports (index.html and serenity-summary.html) are generated under target/site/serenity.
Below is the sample index.html report which has test evidence attached to the manual test.
You can see that there is a new tag with name – Test Evidence. This is the screenshot I have placed under assets folder.
Click on the link and a new page with the screesnhot placed under assets folder opens.
Congratulations. You are able to attach the test evidence to manual tests in Serenity Report. Hope you enjoyed this tutorial. Cheers!!
Cucumber is primarily and traditionally used for automating executable specifications. But with Serenity BDD, you can add special tags to indicate that a scenario represents a manual test case.
You can flag any Cucumber scenario as manual simply by using the @manual tag. In the below example, I have tagged a scenario as “@manual”. The last scenario is tagged as “manual”. By default, @manual scenarios are marked as pending in the Serenity reports.
Feature: Login to HRM
@ValidCredentials
Scenario: Login with valid credentials
Given User is on Home page
When User enters username as "Admin"
And User enters password as "admin123"
Then User should be able to login successfully
@InValidCredentials
Scenario Outline: Login with invalid credentials
Given User is on Home page
When User enters username as '<username>'
And User enters password as '<password>'
Then User should be able to see error message '<errorMessage>'
Examples:
|username |password |errorMessage |
|admin |admin |Invalid credentials |
| |admin123 |Username cannot be empty |
|Admin | |Password cannot be empty |
| | |Username can be empty |
@ForgetPassword
Scenario: Verify Forget Password Functionality
Given User is on Home page
When User clicks on Forgot your password link
Then User should be able to see new page which contains Reset Password button
@manual
Scenario: Verify credentials present in Master Database not older than 30 days
Given User is connected to Master Database
Then Username "Admin" and password "admin123" are present in Master Database not older than 30 days
Execute the test suite using below command
mvn clean verify
The scenario marked with @manual tag will now appear as a Manual test case in the Serenity report (Index.html). To know how to create Serenity Report, click here.
We can indicate a different result by adding the @manual-result tag as shown here:
A passing test: @manual-result:passed A failing test: @manual-result:failed A compromised test: @manual-result:compromised
If we want to record the result of a manual test, we should include both the @manual and the @manual-result tags.
@manual
@manual-result:passed
Scenario: Verify credentials present in Master Database not older than 30 days
Given User is connected to Master Database
Then Username "Admin" and password "admin123" are present in Master Database not older than 30 days
@manual
@manual-result:failed
Scenario: Verify different credentials are provided to Admin, Dev and QA to access Master Database
Given User is connected to Master Database
Then Different credentials are provided to Admin, Business, Dev and QA to access Master Database
This image shows that there are 2 manual tests. I have marked one manual test as passed and another one as failed which is clearly shown in this image.
How to update Manul Test Results
In the below example, we are considering that the team is working on Sprint-1. We have executed the manual tests and marked the status in the feature file as shown below.
@manual
@manual-result:passed
@manual-last-tested:sprint-1
Scenario: Verify credentials present in Master Database not older than 30 days
Given User is connected to Master Database
Then Username "Admin" and password "admin123" are present in Master Database not older than 30 days
@manual
@manual-result:failed
@manual-last-tested:sprint-1
Scenario: Verify different credentials are provided to Admin, Dev and QA to access Master Database
Given User is connected to Master Database
Then Different credentials are provided to Admin, Business, Dev and QA to access Master Database
In the Serenity properties , the team also records the current version (or sprint number):
serenity.project.name = Serenity and Cucumber Report Demo
current.target.version = sprint-1
Now, execute the feature file. This is how the report look like.
Now, we are in next sprint. Update the value of current.target.version in serenity.properties file.
serenity.project.name = Serenity and Cucumber Report Demo
current.target.version = sprint-2
Now, when the manual scenario is processed, it will be marked as pending, with a note indicating that a new manual test is required:
Both the maual tests which were marked as pass and fail are now pending tests as shown in the image.
We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
In the previous tutorial, I explained the Generation of Serenity Report (index.html) using Cucumber6 and JUnit. Index.html report that acts both as a test report and living documentation for the product. It has various views like Overall Test Status, Requirement View, Capabilities View, and Features View.
Sometimes it is useful to be able to send a short summary of the test outcomes via email. Serenity allows us to generate a single-page, self-contained HTML summary report, containing an overview of the test results, and a configurable breakdown of the status of different areas of the application.
Pre-Requisite
Java 11 installed
Maven installed
Eclipse or IntelliJ installed
This framework consists of:
Java 11
Maven – 3.8.1
Serenity – 2.6.0
Serenity Maven – 2.6.0
Serenity Cucumber6 – 2.6.0
JUnit – 4.13.2
Maven Surefire Plugin – 3.0.0-M5
Maven Failsafe Plugin – 3.0.0-M5
Maven Comiler Plugin – 3.8.1
Implementation Steps
Update Properties section in Maven pom.xml
Add repositories and pluginRepository to Maven pom.xml
Add Serenity, Serenity Cucumber and JUnitdependencies to POM.xml
Update Build Section of pom.xml
Create source folder – src/test/resources and features folder within src/test/resources to create test scenarios in Feature file
Create theStep Definition class or Glue Code
Create a Serenity-Cucumber Runner class
Create serenity.conf file under src/test/resources
Createserenity.properties file in the root of the project
Run the tests through commandline which generates Serenity Report
To know about Step 1 to 3, please refer here. These steps are the same for Index.html report and emailable report.
Now, add the below-mentioned plugin. These reports are configured in the Serenity Maven plugin, where you need to do two things. First, you need to add a dependency for the serenity-emailer module in the plugin configuration. Then, you need to tell Serenity to generate the email report when it performs the aggregation task.
I have provided the location of Firefoxdriver through the command line. I believe this is the best way to run the test. We can hard-code the path in the test code or in serenity.conf file. In that case, you don’t need to provide the location of Firefoxdriver through command line. You can use the below command.
mvn verify
The output of the above program is
This image shows that two different types of reports are generated by Serenity – Full Report (index.html) and Single Page HTML Summary ( serenity-summary.html ).
This emailable report is called serenity-summary.html. This is generated under site/serenity/ serenity-summary.html
You can see a sample of such a report here:
As you can see in the above execution status, out of six tests, one test failed. The same information is displayed in the report.
This report provides a summary of the test execution.
The Functional Coverage section lets us highlight key areas of your application. By default, this section will list test results for each Feature. But we can configure the report to group results by other tags as well.
We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
Below is the sample project which uses Selenium and JUnit4 which is used to generate an Allure Report.
We have 2 pages. Below is the code for Login Page which contains all the web elements and methods related to that web elements.
Note:- This is a sample code. There could be the probability that XPath would have changed. So, the tests won’t run as expected and please keep this in mind.
public class LoginPage {
WebDriver driver;
By userName = By.name("txtUsername");
By password = By.name("txtPassword");
By titleText = By.id("logInPanelHeading");
By login = By.id("btnLogin");
By errorMessage = By.id("spanMessage");
public LoginPage(WebDriver driver) {
this.driver = driver;
}
// Set user name in textbox
public void setUserName(String strUserName) {
driver.findElement(userName).sendKeys(strUserName);
}
// Set password in password textbox
public void setPassword(String strPassword) {
driver.findElement(password).sendKeys(strPassword);
}
// Click on login button
public void clickLogin() {
driver.findElement(login).click();
}
@Step("Verify title of Login Page")
public void verifyPageTitle() {
String loginPageTitle = driver.findElement(titleText).getText();
assertTrue(loginPageTitle.contains("LOGIN Panel"));
}
/* Failed Test */
@Step("Verify error message when invalid credentail is provided")
public void verifyErrorMessage() {
String invalidCredentialErrorMessage = driver.findElement(errorMessage).getText();
assertTrue(invalidCredentialErrorMessage.contains("Incorrect Credentials"));
}
@Step("Enter username and password")
public void login(String strUserName, String strPasword) {
// Fill user name
this.setUserName(strUserName);
// Fill password
this.setPassword(strPasword);
// Click Login button
this.clickLogin();
}
}
assertTrue() is imported from the below JUnit package for assertion.
public class DashboardPage {
WebDriver driver;
By dashboardPageTitle = By.id("welcome");
By assignLeaveOption = By.cssSelector(
"#dashboard-quick-launch-panel-menu_holder > table > tbody > tr > td:nth-child(1) > div > a > span");
By leaveListOption = By.cssSelector(
"#dashboard-quick-launch-panel-menu_holder > table > tbody > tr > td:nth-child(2) > div > a > span");
By timesheetsOption = By.cssSelector(
"#dashboard-quick-launch-panel-menu_holder > table > tbody > tr > td:nth-child(3) > div > a > span");
By applyLeaveOption = By.cssSelector(
"#dashboard-quick-launch-panel-menu_holder > table > tbody > tr > td:nth-child(4) > div > a > span");
public DashboardPage(WebDriver driver) {
this.driver = driver;
}
@Step("Verify title of Dashboard page")
public void verifyDashboardPageTitle() {
String DashboardPageTitle = driver.findElement(dashboardPageTitle).getText();
assertTrue(DashboardPageTitle.contains("Welcome"));
}
@Step("Verify Assign Leave Quick Launch Options on Dashboard page")
public void verifyAssignLeaveOption() {
String QuickLaunchOptions = driver.findElement(assignLeaveOption).getText();
assertTrue(QuickLaunchOptions.contains("Assign Leave"));
}
@Step("Verify Leave List Quick Launch Options on Dashboard page")
public void verifyLeaveListOption() {
String LeaveListQuickLaunchOption = driver.findElement(leaveListOption).getText();
assertTrue(LeaveListQuickLaunchOption.contains("Leave List"));
}
@Step("Verify Assign Leave Quick Launch Options on Dashboard page")
public void verifytimesheetsOption() {
String timesheetsOptionQuickLaunchOption = driver.findElement(timesheetsOption).getText();
assertTrue(timesheetsOptionQuickLaunchOption.contains("Timesheets"));
}
@Step("Verify Leave List Quick Launch Options on Dashboard page")
public void verifyApplyLeaveOption() {
String applyLeaveQuickLaunchOptions = driver.findElement(applyLeaveOption).getText();
assertTrue(applyLeaveQuickLaunchOptions.contains("Apply Leave"));
}
}
Test Classes related to various Pages
BaseTest.java
public class BaseTest {
public static WebDriver driver;
LoginPage objLogin;
DashboardPage objDashboardPage;
@Step("Start the application")
@BeforeEach
public void setup() {
System.setProperty("webdriver.gecko.driver",
"C:\\Users\\Vibha\\Software\\geckodriver-v0.26.0-win64\\geckodriver.exe");
driver = new FirefoxDriver();
driver.manage().window().maximize();
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
driver.get("https://opensource-demo.orangehrmlive.com/");
}
@Step("Stop the application")
@AfterEach
public void close() {
driver.close();
}
}
@BeforeEach is used to signal that the annotated method should be executed before each @Test, @RepeatedTest, @ParameterizedTest, @TestFactory, and @TestTemplate method in the current test class. It is imported from:-
import org.junit.jupiter.api.BeforeEach;
@AfterEach is used to signal that the annotated method should be executed after each@Test, @RepeatedTest, @ParameterizedTest, @TestFactory, and @TestTemplatemethod in the current test class. It is imported from:-
import org.junit.jupiter.api.AfterEach;
LoginTests.java
@Epic("Web Application Regression Testing using JUnit5")
@Feature("Login Page Tests")
public class LoginTests extends BaseTest {
LoginPage objLogin;
DashboardPage objDashboardPage;
@Severity(SeverityLevel.NORMAL)
@Test
@Description("Test Description : Verify the title of Login Page")
@Story("Title of Login Page")
public void verifyLoginPage() {
// Create Login Page object
objLogin = new LoginPage(driver);
// Verify login page text
objLogin.verifyPageTitle();
}
@Severity(SeverityLevel.BLOCKER)
@Test
@Description("Test Description : Login Test with invalid credentials")
@Story("Unsuccessful Login to Application")
public void invalidCredentialTest() {
// Create Login Page object
objLogin = new LoginPage(driver);
objLogin.login("test", "test123");
// Verify login page text
objLogin.verifyErrorMessage();
}
}
DashboardTests.java
package com.example.Junit5AllureReportDemo.tests;
import org.junit.jupiter.api.Test;
import com.example.Junit5AllureReportDemo.pages.DashboardPage;
import com.example.Junit5AllureReportDemo.pages.LoginPage;
import io.qameta.allure.Description;
import io.qameta.allure.Epic;
import io.qameta.allure.Feature;
import io.qameta.allure.Severity;
import io.qameta.allure.SeverityLevel;
import io.qameta.allure.Story;
@Epic("Web Application Regression Testing using JUnit5")
@Feature("Dashboard Page Tests")
public class DashboardTests extends BaseTest {
LoginPage objLogin;
DashboardPage objDashboardPage;
@Severity(SeverityLevel.BLOCKER)
@Test
@Description("Test Description : Verify title of Dashboard page")
@Story("Title of Dashboard Page")
public void dashboardTitleTest() {
objLogin = new LoginPage(driver);
// login to application
objLogin.login("Admin", "admin123");
// go the dashboard page
objDashboardPage = new DashboardPage(driver);
objDashboardPage.verifyDashboardPageTitle();
}
@Severity(SeverityLevel.BLOCKER)
@Test
@Description("Test Description : Verify Assign Leave Option in Quick Link Menu")
@Story("Validation of Assign Leave Option")
public void assignLeaveOptionTest() {
objLogin = new LoginPage(driver);
// login to application
objLogin.login("Admin", "admin123");
// go the dashboard page
objDashboardPage = new DashboardPage(driver);
objDashboardPage.verifyAssignLeaveOption();
}
@Severity(SeverityLevel.BLOCKER)
@Test
@Description("Test Description : Verify Apply Leave Option in Quick Link Menu")
@Story("Validation of Apply Leave Option")
public void applyLeaveOptionTest() {
objLogin = new LoginPage(driver);
// login to application
objLogin.login("Admin", "admin123");
// go the dashboard page
objDashboardPage = new DashboardPage(driver);
objDashboardPage.verifyApplyLeaveOption();
}
@Severity(SeverityLevel.BLOCKER)
@Test
@Description("Test Description : Verify Leave List Option in Quick Link Menu")
@Story("Validation of Leave List Option")
public void leaveListOptionTest() {
objLogin = new LoginPage(driver);
// login to application
objLogin.login("Admin", "admin123");
// go the dashboard page
objDashboardPage = new DashboardPage(driver);
objDashboardPage.verifyLeaveListOption();
}
@Severity(SeverityLevel.BLOCKER)
@Test
@Description("Test Description : Verify Timesheets Option in Quick Link Menu")
@Story("Validation of Timesheets Option")
public void timesheetsOptionTest() {
objLogin = new LoginPage(driver);
// login to application
objLogin.login("Admin", "admin123");
// go the dashboard page
objDashboardPage = new DashboardPage(driver);
objDashboardPage.verifyTimesheetsOption();
}
}
Step 5 – Run the Test and Generate Allure Report
To run the tests, use the below command
mvn clean test
In the below image, we can see that one test failed and six passed out of seven tests.
This will create the allure-results folder with all the test reports. These files will be used to generate Allure Report.
To create Allure Report, use the below command
allure serve
This will generate the beautiful Allure Test Report as shown below.
Allure Report Dashboard
Categories in Allure Report
The categories tab gives you a way to create custom defect classifications to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).
Suites in Allure Report
On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.
Graphs in Allure Report
Graphs allow you to see different statistics collected from the test data: status breakdown or severity and duration diagrams.
Timeline in Allure Report
Timeline tab visualizes retrospective of tests execution, allure adaptors collect precise timings of tests, and here on this tab they are arranged accordingly to their sequential or parallel timing structure.
Behaviors of Allure Report
This tab groups test results according to Epic, Feature, and Story tags.
Packages in Allure Report
Packages tab represents a tree-like layout of test results, grouped by different packages.
We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
Below is the sample project which uses Selenium and TestNG which is used to generate an Allure Report.
We have 2 pages. Below is the code for Login Page which contains all the web elements and methods related to that web elements.
LoginPage.java
public class LoginPage {
WebDriver driver;
By userName = By.name("txtUsername");
By password = By.name("txtPassword");
By titleText = By.id("logInPanelHeading");
By login = By.id("btnLogin");
By errorMessage = By.id("spanMessage");
public LoginPage(WebDriver driver) {
this.driver = driver;
}
// Set user name in textbox
public void setUserName(String strUserName) {
driver.findElement(userName).sendKeys(strUserName);
}
// Set password in password textbox
public void setPassword(String strPassword) {
driver.findElement(password).sendKeys(strPassword);
}
// Click on login button
public void clickLogin() {
driver.findElement(login).click();
}
@Step("Verify title of Login Page")
public void verifyPageTitle() {
String loginPageTitle = driver.findElement(titleText).getText();
Assert.assertTrue(loginPageTitle.contains("LOGIN Panel"));
}
/* Failed Test */
@Step("Verify error message when invalid credentail is provided")
public void verifyErrorMessage() {
String invalidCredentialErrorMessage = driver.findElement(errorMessage).getText();
Assert.assertTrue(invalidCredentialErrorMessage.contains("Incorrect Credentials"));
}
@Step("Enter username and password")
public void login(String strUserName, String strPasword) {
// Fill user name
this.setUserName(strUserName);
// Fill password
this.setPassword(strPasword);
// Click Login button
this.clickLogin();
}
}
Dashboard.java
public class DashboardPage {
WebDriver driver;
By dashboardPageTitle = By.id("welcome");
By options = By.cssSelector(
"#dashboard-quick-launch-panel-menu_holder > table > tbody > tr > td:nth-child(1) > div > a > span");
public DashboardPage(WebDriver driver) {
this.driver = driver;
}
@Step("Verify title of Dashboard page")
public void verifyDashboardPageTitle() {
String DashboardPageTitle = driver.findElement(dashboardPageTitle).getText();
Assert.assertTrue(DashboardPageTitle.contains("Welcome"));
}
@Step("Verify Quick Launch Options on Dashboard page")
public void verifyQuickLaunchOptions() {
String QuickLaunchOptions = driver.findElement(options).getText();
Assert.assertTrue(QuickLaunchOptions.contains("Assign Leave"));
}
}
Below are the Test classes for Login Page and Dashboard Page. Here, we have BaseTest Class also which contains the common methods needed by other test pages.
BaseTest.java
public class BaseTest {
public static WebDriver driver;
LoginPage objLogin;
DashboardPage objDashboardPage;
@Step("Start the application")
@BeforeMethod
public void setup() {
System.setProperty("webdriver.gecko.driver",
"C:\\Users\\Vibha\\Software\\geckodriver-v0.26.0-win64\\geckodriver.exe");
driver = new FirefoxDriver();
driver.manage().window().maximize();
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
driver.get("https://opensource-demo.orangehrmlive.com/");
}
@Step("Stop the application")
@AfterMethod
public void close() {
driver.close();
}
}
LoginTests.java
@Epic("Web Application Regression Testing")
@Feature("Login Page Tests")
@Listeners(TestExecutionListener.class)
public class LoginTests extends BaseTest {
LoginPage objLogin;
DashboardPage objDashboardPage;
@Severity(SeverityLevel.NORMAL)
@Test(priority = 0, description = "Verify Login Page")
@Description("Test Description : Verify the title of Login Page")
@Story("Title of Login Page")
public void verifyLoginPage() {
// Create Login Page object
objLogin = new LoginPage(driver);
// Verify login page text
objLogin.verifyPageTitle();
}
/* Failed Test */
@Severity(SeverityLevel.BLOCKER)
@Test(priority = 1, description = "Login with invalid username and password")
@Description("Test Description : Login Test with invalid credentials")
@Story("Unsuccessful Login to Application")
public void invalidCredentialTest() {
// Create Login Page object
objLogin = new LoginPage(driver);
objLogin.login("test", "test123");
// Verify login page text
objLogin.verifyErrorMessage();
}
}
We can order tests by severity by using @Severityannotation. Click here to know more about other Allure annotations.
DashboardTests.java
@Epic("Web Application Regression Testing")
@Feature("Dashboard Page Tests")
@Listeners(TestExecutionListener.class)
public class DashboardTests extends BaseTest {
LoginPage objLogin;
DashboardPage objDashboardPage;
@Severity(SeverityLevel.BLOCKER)
@Test(priority = 0, description = "Verify Dashboard Page")
@Description("Test Description : After successful login to application opens Dashboard page")
@Story("Successful login of application opens Dashboard Page")
public void DasboardTest() {
objLogin = new LoginPage(driver);
// login to application
objLogin.login("Admin", "admin123");
// go the dashboard page
objDashboardPage = new DashboardPage(driver);
// Verify dashboard page
objDashboardPage.verifyQuickLaunchOptions();
}
}
We can group tests with @Epic, @Feature, and @Stories annotations. Click here to know more about other Allure annotations.
TestExecutionListener.class
We can add attachments to our reports by using @Attachmentannotation. It can return String, byte [], etc. I need to add @Listeners({ TestExecutionListener.class }) declaration at the top of the test classes. Click here to know more about other Allure annotations.
public class TestExecutionListener implements ITestListener {
@Attachment(value = "Screenshot of {0}", type = "image/png")
public byte[] saveScreenshot(String name, WebDriver driver) {
return (byte[]) ((TakesScreenshot) driver).getScreenshotAs(OutputType.BYTES);
}
@Override
public void onTestFailure(ITestResult result) {
saveScreenshot(result.getName(), BaseTest.driver);
}
}
Step 5 – Create testng.xml for the project
TestNG.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name = "Allure Reports">
<test name = "Login Page Tests">
<classes>
<class name = "com.example.TestNGAllureReportDemo.tests.LoginTests"/>
</classes>
</test>
<test name =" Dashboard Tests">
<classes>
<class name = "com.example.TestNGAllureReportDemo.tests.DashboardTests"/>
</classes>
</test>
</suite>
Step 6 – Run the Test and Generate Allure Report
To run the tests, use the below command
mvn clean test
In the below image, we can see that one test failed and two passed out of three tests.
To create an Allure Report, use the below command
allure serve
This will generate the beautiful Allure Test Report as shown below.
Allure Report Dashboard
The overview page hosts several default widgets representing the basic characteristics of your project and test environment.
Statistics – overall report statistics.
Launches – if this report represents several test launches, statistics per launch will be shown here.
Behaviors – information on results aggregated according to stories and features.
Executors – information on test executors that were used to run the tests.
History Trend – if tests accumulated some historical data, it’s a trend that will be calculated and shown on the graph.
Environment – information on the test environment.
Categories in Allure Report
The categories tab gives you a way to create custom defects classifications to apply for test results. There are two categories of defects – Product Defects (failed tests) and Test Defects (broken tests).
Suites in Allure Report
On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.
Graphs in Allure Report
Graphs allow you to see different statistics collected from the test data: status breakdown or severity and duration diagrams.
Timeline in Allure Report
The timeline tab visualizes retrospective of tests execution, allure adaptors collect precise timings of tests, and here on this tab, they are arranged accordingly to their sequential or parallel timing structure.
Behaviors of Allure Report
This tab groups test results according to Epic, Feature, and Story tags.
Packages in Allure Report
The packages tab represents a tree-like layout of test results, grouped by different packages.
If you click on the (highlighted tab), it will show the test execution report in the below format.
Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!
Another important pre-requisite is addition of below instruction in cucumber.properties file
cucumber.publish.enabled=true
Cucumber.properties file should be present in src/test/resources.
Now, execute the Cucumber tests by using command — mvn test in command line.
Below image shows the Console message generated
If you follow the link, you’ll see your report rendered in glorious colour. Below is the image of such Cucumber Report. This report provide the information about
No Of Test Scenarios Executed
Passed vs Failed percentage
When the tests are executed like 8 minutes ago or 15 hours ago
Time taken to execute the Test Suite
Window Version
Java Version
Cucumber-jvm Version
Name of the feature file executed
Test Scenario step execution – passed, failed, pending
The report is self destructed in 24 hours. It is mentioned as a warning at the top of the report.
To save the report for future use, click on the link – Keep your future reports forever. It will open a dialog box as shown below. You can create a Report Collection and the reports present in Report Collection are not auto-deleted.
Login to GitHub. A new page as shown below will appear. Create a collection by providing the name in the Name box and click the “Create new collection” button.
This page contains a token as shown in the image in the below screen. Mention this environment variable with token in cucumber.properties file which is present within src/test/resources.
When the tests will be executed, now the report will be saved under Cucumber Gradle Reports Collection.
Below is the image of the report which is saved under Cucumber Gradle Reports collection in GitHub.
If you have not added cucumber.properties to your project, then will get a screen as shown below.
Delete Report
The report can be deleted by clicking on Delete button on the report. Once you have clicked on Delete Report button, a dialog box as shown below is displayed. Once you click on Delete button present in the dialog box, the report is deleted permanently.
That’s it! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
QA Automation Expert 