ChainTest Report with TestNG

vibssingh

Run ChainTest Report on Docker

Last Updated On

HOME

ChainTest Reports can be run on docker (dynamic report).

What is ChainLP?

ChainLP (Chain-Long-Playing like LP Record) is a Java (Spring) server which packs the Angular frontend and is distributed as a Docker image. ChainLP is the framework component providing historical analytics.

The recommended way to run ChainLP is with docker-compose. Currently, the supported databases are listed below but most RDBMS database should work.

  • H2
  • MySQL
  • PostgreSQL

For each database, there is a separate docker-compose.yml available at chainlp/docker. H2 provides the most straight-forward way to test, but it is NOT recommended for production use.

Docker image is available from https://hub.docker.com/r/anshooarora/chaintest.

We need to have Desktop Docker up and running. To know how to install the Desktop Docker, please refer to this tutorial – How to install Desktop Docker on Windows 11

Go the location where these docker images are placed and run the below command. In this example, we are running h2 docker image.

docker-compose -f docker-compose-h2.yml up

The output of the above execution is

Go to the http:localhost:80 and we will see an empty ChainTest Report.

Run the project contains the ChainTest report configuration. Refer this tutorial to get the ChainTest Report configuration – ChainTest Report with Cucumber and TestNG.

Add the following properties to chaintest.properties file in src/test/resources

# chaintest configuration
chaintest.project.name= ChaninTest Report with Cucumber and TestNG

# generators:
## chainlp
chaintest.generator.chainlp.enabled=true
chaintest.generator.chainlp.class-name=com.aventstack.chaintest.generator.ChainLPGenerator
chaintest.generator.chainlp.host.url=http://localhost/
chaintest.generator.chainlp.client.request-timeout-s=30
chaintest.generator.chainlp.client.expect-continue=false
chaintest.generator.chainlp.client.max-retries=3

Run the test either through command line or through Test Runner class.

Run the tests through command line

To run the tests, use the below command

mvn clean test

Updation of ChainTest Report

ChainTest Report is updated with the current test run as shown below:

Click on the project and it will show the complete test execution report.

Click on cucumber-jvm option. It will show the detailed execution report. This report includes a pie chart for the feature, Scenario and Steps. Here, we have executed 1 feature file and 1 scenario. The Feature file has failed. So, the Feature pie chart shows 100% failed. There are 5 scenarios. Out of 5, 4 tests are passed. 1 test failed. Therefore, the Scenario pie chart shows 80% passed. This report also shows error message for the failed test.

We see that one of the tests has failed. The screenshot of the failed test is also attached in the report.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

vibssingh

ChainTest Report Tutorials

Last Updated on

HOME

ChainTest Report is a comprehensive reporting framework supporting multiple generators – static, email & realtime, historical analytics with ChainLP.

Chapter 1 Replacement of Extent Report: ChainTest Report with Selenium and TestNG
Chapter 2 How to add logs and screenshot to ChainTest Report with TestNG – NEW
Chapter 3 ChainTest Report with Cucumber and TestNG – NEW
Chapter 4 ChainTest Report with Selenium and JUnit5 – NEW
Chapter 5 Run ChainTest Report on Docker – NEW
vibssingh

How to add logs and screenshot to ChainTest Report with TestNG

HOME

In the previous tutorial, will explain ChainTest Report and its implementation with Selenium with TestNG. In this tutorial, we will discuss the steps to add the screenshot for the failed tests in the ChainTest Report.

The chaintest-core client is the framework component that supports plugins to store embeds for each report. For example, with SimpleGenerator, the client saves all embeds relative to the report file in the resources folder.

Starting chaintest-testng plugin 1.0.5+, screenshots can be attached to tests while the test is in-flight or completing in the @BeforeMethod and @AfterMethod hooks.

@Test
public void testMethod(final Method method) {
  
//log
  ChainTestListener.log("log example");

  // embed
  ChainTestListener.embed(bytes, "image/png");
}

The complete implementation of ChainTest Report can be found here – Replacement of Extent Report: ChainTest Report with Selenium and TestNG

I have updated the BaseTests and added the implementation of adding the screenshots.

package com.example;

import com.aventstack.chaintest.plugins.ChainTestListener;
import com.aventstack.chaintest.service.ChainPluginService;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.edge.EdgeDriver;
import org.openqa.selenium.edge.EdgeOptions;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.testng.ITestResult;
import org.testng.annotations.*;

import java.time.Duration;

@Listeners(ChainTestListener.class)
public class BaseTests {

    public static WebDriver driver;
    public final static int TIMEOUT = 10;

    @BeforeTest
    @Parameters("browserName")
    public void setup(String browserName) throws Exception {

        System.out.println("Browser : " + browserName);

        switch (browserName.toLowerCase().trim()) {

            case "chrome":
                ChromeOptions options = new ChromeOptions();
                options.addArguments("--remote-allow-origins=*");
                options.addArguments("--no-sandbox");
                options.addArguments("--disable-dev-shm-usage");
                // options.addArguments("--headless");
                driver = new ChromeDriver(options);
                break;

            case "firefox":
                FirefoxOptions firefoxOptions = new FirefoxOptions();
                driver = new FirefoxDriver(firefoxOptions);
                break;
            case ("edge"):
                EdgeOptions edgeOptions = new EdgeOptions();
                driver = new EdgeDriver(edgeOptions);
                break;

            default:
                System.out.println("Incorrect browser is supplied...." + browserName);
                throw new IllegalArgumentException("WRONG BROWSER : " + browserName);
        }

        driver.manage().window().maximize();
        driver.get("https://the-internet.herokuapp.com/login");
        driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(TIMEOUT));
    }

    @AfterMethod
    public void attachScreenshot(ITestResult result){
        if(!result.isSuccess()){
            ChainTestListener.embed(takeScreenshot(), "image/png");
        }
    }

    public byte[] takeScreenshot(){
        return ((TakesScreenshot)(driver)).getScreenshotAs(OutputType.BYTES);
    }

   @AfterTest
   public void tearDown() {
       driver.quit();
   }

}

How to add logs to the Report?

log is imported from package – import com.aventstack.chaintest.plugins.ChainTestListener;

ChainTestListener.log("Page Title :" + actualLoginPageTitle);

I have updated the LoginPageTests to add the logs also in the report.

package com.example;

import com.aventstack.chaintest.plugins.ChainTestListener;
import org.testng.Assert;
import org.testng.annotations.Test;

public class LoginPageTests extends BaseTests {

    String actualLoginPageTitle;
    String actualErrorMessage;
    String actualSecurePageTitle;

    @Test(priority = 0)
    public void verifyPageTitle() {

        actualLoginPageTitle = driver.getTitle();
        ChainTestListener.log("Page Title :" + actualLoginPageTitle);

        // Verify Page Title
        Assert.assertEquals(actualLoginPageTitle, "The Internet !!");

    }

    @Test(priority = 1)
    public void invalidCredentials() {

        LoginPage loginPage = new LoginPage(driver);
        loginPage.login("tomsmith", "happy!");
        actualErrorMessage = loginPage.getErrorMessage();
        ChainTestListener.log("Error Message :" + actualErrorMessage);

        // Verify Error Message
        Assert.assertTrue(actualErrorMessage.contains("Your password is invalid!"));

    }

    @Test(priority = 2)
    public void validLogin() {

        LoginPage loginPage = new LoginPage(driver);
        loginPage.login("tomsmith", "SuperSecretPassword!");

        SecurePage securePage = new SecurePage(driver);
        actualSecurePageTitle = securePage.getSecurePageTitle();
        ChainTestListener.log("Actual Dashboard Page Title :" + actualSecurePageTitle);

        // Verify Home Page
        Assert.assertTrue(actualSecurePageTitle.contains("You logged into a secure area!"));

    }

}

Execute the tests either using testng.xml or through command line

mvn clean test

The output of the above program is

Open the index.html report and we can see that the failed test has the screenshot attached to it.

Double click on the screenshot attached in the report and we can see the screenshot in the full screen.

Below highlighted ones are the logs mentioned in the LoginPageTests class.

Note:- The addition of screenshots for the failed test as well as the logs are only implemented in index.html report, and not in email.html.

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

vibssingh

Replacement of Extent Report: ChainTest Report with Selenium and TestNG

HOME

ExtentReport, one of the most popular and extensively used test reports, has been officially retired due to changes in automated testing techniques. The same developer, Anshoo Arora, fills the void with ChainTest, a potent reporting system. With its enhanced capabilities, real-time data, and seamless integration options, ChainTest is anticipated to transform how we see test results.

Table of Contents

  1. What is ChainTest Report?
  2. Key Features of ChainTest Report
  3. Prerequisite
  4. Dependency List
  5. Structure of Project
  6. Implementation
    1. Add ChainTest TestNG plugin dependency in Maven Project
    2. Create Pages and Test Code for the pages
    3. Add chaintest.properties
    4. Add ChainTestListener
    5. Create a testng.xml at the root directory
    6. Run the tests from testng.xml
    7. View ChainTest Reports
    8. Run the tests through command line

What is ChainTest Report?

ChainTest is a complete reporting system, it supports a variety of sources, including email, static data, and real-time, historical analytics with ChainLP. The next-generation reporting framework ChainTest was developed for a number of test frameworks, such as Cucumber, JUnit, and TestNG.

Key Features of ChainTest Report

Combination of Report – ChainTest generates a combination of static report, email and dynamic report.

Real Time Analysis – ChainTest’s incorporation of real-time statistics allows teams to monitor test executions in real time. This feature is very helpful for extensive testing projects with tight deadlines.

Simple SetUp – Setting up ChainTest is simple and quick with an easily accessible properties file. The setup procedure is streamlined by the ease with which users can specify project names, server URLs, and report options.

Support multiple test framework – This is compatible with a number of test frameworks, such as Cucumber, JUnit, and TestNG. Integration with PyTest is also expected to be available soon.

Docker-Enabled Setup – Docker simplifies the process of setting up ChainTest. The Chain LP server can be spun up in a matter of minutes by users by pulling the Docker image, which streamlines deployment and minimizes system requirements.

Monitoring of Historical Data – The historical data tracking tool, which enables teams to examine performance trends across several builds, is one notable feature. This makes it easier to spot trends and enhance test-taking techniques.

Prerequisite

  1. Java 17 is installed
  2. Maven is installed
  3. Eclipse or IntelliJ is installed

Dependency List:

  1. Selenium – 4.28.1
  2. Java 17
  3. TestNG – 7.10.2
  4. Maven – 3.9.6
  5. ChainTest TestNG – 1.0.7

Structure of Project

Implementation

Add ChainTest TestNG plugin dependency in Maven Project.

<dependency>
  <groupId>com.aventstack</groupId>
  <artifactId>chaintest-testng</artifactId>
  <version>${chaintest.testng.version}</version>
</dependency>

The complete pom.xml looks like as shown below:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.example</groupId>
  <artifactId>ChainTest_TestNG_Demo</artifactId>
  <version>1.0-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>ChainTest_TestNG_Demo</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <selenium.version>4.28.1</selenium.version>
    <testng.version>7.10.2</testng.version>
    <chaintest.testng.version>1.0.7</chaintest.testng.version>
    <maven.compiler.plugin.version>3.13.0</maven.compiler.plugin.version>
    <maven.compiler.source.version>17</maven.compiler.source.version>
    <maven.compiler.target.version>17</maven.compiler.target.version>
    <maven.surefire.plugin.version>3.2.5</maven.surefire.plugin.version>
    <aspectj.version>1.9.20.1</aspectj.version>
  </properties>

  <dependencies>

    <!-- Selenium -->
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>${selenium.version}</version>
    </dependency>

    <!-- TestNG -->
    <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>${testng.version}</version>
      <scope>test</scope>
    </dependency>

    <!-- Chain Test Report -->
    <dependency>
      <groupId>com.aventstack</groupId>
      <artifactId>chaintest-testng</artifactId>
      <version>${chaintest.testng.version}</version>
    </dependency>
    
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>${maven.compiler.plugin.version}</version>
        <configuration>
          <source>${maven.compiler.source.version}</source>
          <target>${maven.compiler.target.version}</target>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>${maven.surefire.plugin.version}</version>
        <configuration>
          <suiteXmlFiles>
            <suiteXmlFile>testng.xml</suiteXmlFile>
          </suiteXmlFiles>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

Create Pages and Test Code for the pages

BasePage

package com.example;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.support.PageFactory;

public class BasePage {
    public WebDriver driver;

    public BasePage(WebDriver driver) {
        this.driver = driver;
        PageFactory.initElements(driver,this);
    }

}

LoginPage

package com.example;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;

public class LoginPage extends BasePage {

    public LoginPage(WebDriver driver) {
        super(driver);

    }

    @FindBy(name = "username")
    public WebElement userName;

    @FindBy(name = "password")
    public WebElement password;

    @FindBy(xpath = "//*[@type='submit']")
    public WebElement loginBtn;

    @FindBy(xpath = "//*[@id='flash']")
    public WebElement errorMessage;

    public void login(String strUserName, String strPassword) {
        userName.sendKeys(strUserName);
        password.sendKeys(strPassword);
        loginBtn.click();
    }

    public String getErrorMessage() {
        return errorMessage.getText();
    }

}

SecurePage

package com.example;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;

public class SecurePage extends BasePage {

    public SecurePage(WebDriver driver) {
        super(driver);

    }

    @FindBy(xpath = "//*[@id='flash']")
    public WebElement securePageTitle;

    public String getSecurePageTitle() {
        return securePageTitle.getText();
    }

}

BaseTests

package com.example;

import com.aventstack.chaintest.plugins.ChainTestListener;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.edge.EdgeDriver;
import org.openqa.selenium.edge.EdgeOptions;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.testng.ITestResult;
import org.testng.annotations.*;

import java.time.Duration;

@Listeners(ChainTestListener.class)
public class BaseTests {

    public static WebDriver driver;
    public final static int TIMEOUT = 10;

    @BeforeTest
    @Parameters("browserName")
    public void setup(String browserName) throws Exception {

        System.out.println("Browser : " + browserName);
       
        switch (browserName.toLowerCase().trim()) {

            case "chrome":
                ChromeOptions options = new ChromeOptions();
                options.addArguments("--remote-allow-origins=*");
                options.addArguments("--no-sandbox");
                options.addArguments("--disable-dev-shm-usage");
                driver = new ChromeDriver(options);
                break;

            case "firefox":
                FirefoxOptions firefoxOptions = new FirefoxOptions();
                driver = new FirefoxDriver(firefoxOptions);
                break;
            case ("edge"):
                EdgeOptions edgeOptions = new EdgeOptions();
                driver = new EdgeDriver(edgeOptions);
                break;

            default:
                System.out.println("Incorrect browser is supplied...." + browserName);
                throw new IllegalArgumentException("Wrong Browser provided: " + browserName);
        }

        driver.manage().window().maximize();
        driver.get("https://the-internet.herokuapp.com/login");
        driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(TIMEOUT));
    }

   @AfterTest
   public void tearDown() {
       driver.quit();
   }

}

LoginPageTests

package com.example;

import org.testng.Assert;
import org.testng.annotations.Test;

public class LoginPageTests extends BaseTests {

    String actualLoginPageTitle;
    String actualErrorMessage;
    String actualDashboardPageTitle;

    @Test(priority = 0)
    public void verifyPageTitle() {

        actualLoginPageTitle = driver.getTitle();

        // Verify Page Title - Failed Test
       Assert.assertEquals(actualLoginPageTitle, "The Internet !!");
    }

    @Test(priority = 1)
    public void invalidCredentials() {

        LoginPage loginPage = new LoginPage(driver);
         loginPage.login("tomsmith", "happy!");
        actualErrorMessage = loginPage.getErrorMessage();

        // Verify Error Message
        Assert.assertTrue(actualErrorMessage.contains("Your password is invalid!"));
    }

    @Test(priority = 2)
    public void validLogin() {

        LoginPage loginPage = new LoginPage(driver);
        loginPage.login("tomsmith", "SuperSecretPassword!");

        SecurePage securePage = new SecurePage(driver);
        actualSecurePageTitle = securePage.getSecurePageTitle();

        // Verify Home Page
        Assert.assertTrue(actualSecurePageTitle.contains("You logged into a secure area!"));
    }
}

Add chaintest.properties 

Add chaintest.properties to the classpath, example: src/test/resources/chaintest.properties.

# chaintest configuration
chaintest.project.name= ChaninTest Report with Selenium and TestNG

# generators:
## chainlp
chaintest.generator.chainlp.enabled=true
chaintest.generator.chainlp.class-name=com.aventstack.chaintest.generator.ChainLPGenerator
chaintest.generator.chainlp.host.url=http://localhost/
chaintest.generator.chainlp.client.request-timeout-s=30
chaintest.generator.chainlp.client.expect-continue=false
chaintest.generator.chainlp.client.max-retries=3

## simple
chaintest.generator.simple.enabled=true
chaintest.generator.simple.document-title=chaintest
chaintest.generator.simple.class-name=com.aventstack.chaintest.generator.ChainTestSimpleGenerator
chaintest.generator.simple.output-file=target/chaintest/Index.html
chaintest.generator.simple.offline=false
chaintest.generator.simple.dark-theme=true
chaintest.generator.simple.datetime-format=yyyy-MM-dd hh:mm:ss a
chaintest.generator.simple.js=
chaintest.generator.simple.css=

## email
chaintest.generator.email.enabled=true
chaintest.generator.email.class-name=com.aventstack.chaintest.generator.ChainTestEmailGenerator
chaintest.generator.email.output-file=target/chaintest/Email.html
chaintest.generator.email.datetime-format=yyyy-MM-dd hh:mm:ss a

Add ChainTestListener 

Add ChainTestListener in the list of @Listeners of the test class. Here, I have added this to BaseTests class.

@Listeners(ChainTestListener.class)
public class BaseTests {
}

Create a testng.xml at the root directory.

Create a testng.xml at the root of the project. This xml contains parameter detail like allow passing parameters to test methods ie browser name.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="ChainTestReport with TestNG">

    <test name="Chrome Test">
        <parameter name="browserName" value="chrome" />
        <classes>
            <class name="com.example.LoginPageTests"/>
        </classes>
    </test> <!-- Test -->

    <test name="Firefox Test">
        <parameter name="browserName" value="firefox" />
        <classes>
            <class name="com.example.LoginPageTests"/>
        </classes>
    </test> <!-- Test -->

    <test name="Edge Test">
        <parameter name="browserName" value="edge" />
        <classes>
            <class name="com.example.LoginPageTests"/>
        </classes>
    </test> <!-- Test -->

</suite> <!-- Suite -->

Run the tests from testng.xml.

Right click on testng.xml and select Run ‘…\testng.xml’

The execution status looks like as shown below. There are 3 tests and 1 test will fail. These 3 tests are executed in 3 different browsers – Chrome, Firefox and Edge. So, total we are executing 9 tests and out of them 6 passed and 3 failed.

View ChainTest Reports

ChainTest Reports are generated in target folder – Index.html and email.html.

Index.html

Right-click and open with Web Browser.

The index.html has a summary section that displays the summary of the execution. The summary includes the overview of the pass/fail using a pictogram, start time, end time, and pass/fail details of tests as shown in the image below.

Email.html

This report contains the high level summary of the test execution.

Run the tests through command line

To run the tests, use the below command

mvn clean test

We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!