How to run JUnit5 tests in order

HOME

The general practices say that automated tests should be able to run independently and with no specific order, as well as the result of the test should not depend on the results of previous tests. But there are situations where a specific order of test execution can be justified, especially in integration or end-to-end tests. The test methods don’t follow a specific order by default to execute the tests. The test cases need not necessarily execute in the order in which they have been written.

There are different ways or modes to set the order of execution for the test cases.  This article shows how to control the JUnit 5 test execution order via the following MethodOrderer classes:

  • DisplayName – sorts test methods alphanumerically based on their display names
  • MethodName – sorts test methods alphanumerically based on their names and formal parameter lists
  • Alphanumeric – sorts test methods alphanumerically based on their names and formal parameter lists. This is deprecated from JUnit Version 5.7 onwards
  • OrderAnnotation – sorts test methods numerically based on values specified via the @Order annotation
  • Random – orders test methods pseudo-randomly and support the configuration of a custom seed
  • Custom Order – A custom ordering sequence can be implemented by the interface MethodOrderer and providing it as the argument to @TestMethodOrder.

Let us create JUnit5 Tests and execute them.

public class OrderRandomDemo {

    @Test
    void test_Add() {

        System.out.println("test_Add()");
        assertEquals(10, 3 + 7);
    }

    @Test
    void test_Subtract() {

        System.out.println("test_Subtract()");
        assertEquals(10, 14 - 4);
    }

    @Test
    void test_Multiply() {

        System.out.println("test_Multiply()");
        assertEquals(10, 5 * 2);
    }

    @Test
    void test_Divide() {

        System.out.println("test_Divide()");
        assertEquals(10, 30 / 3);
    }

    @Test
    void test_IsEven() {

        System.out.println("test_IsEven()");
        assertEquals(0, 10%2);
    }

}

The output of the above program

1. DisplayName

It sorts test methods alphanumerically based on their display names. Test Method can be anything annotated with @Test, @RepeatedTest, @ParameterizedTest, @TestFactory, or @TestTemplate.

@TestMethodOrder(MethodOrderer.DisplayName.class)

@TestMethodOrder is a type-level annotation that is used to configure a MethodOrderer for the test methods of the annotated test class or test interface.

MethodOrderer defines the API for ordering the test methods in a given test class.

Test Method – It is any method annotated with @Test, @RepeatedTest, @ParameterizedTest, @TestFactory, or @TestTemplate.

DisplayName.class – MethodOrderer that sorts methods alphanumerically based on their names using String.compareTo(String).
If two methods have the same name, String representations of their formal parameter lists will be used as a fallback for comparing the methods.

An example of sorting the tests based on their display names.

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import static org.junit.jupiter.api.Assertions.assertEquals;

@TestMethodOrder(MethodOrderer.DisplayName.class)
public class DisplayNameOrderedTests {

    @DisplayName("C")
    @Test
    void test_Add() {

        System.out.println("test_Add()");
        assertEquals(10, 3 + 7);
    }


    @DisplayName("E")
    @Test
    void test_Multiply() {

        System.out.println("test_Multiply()");
        assertEquals(10, 5 * 2);
    }

    @DisplayName("A")
    @Test
    void test_Divide() {

        System.out.println("test_Divide()");
        assertEquals(10, 30 / 3);
    }


    @DisplayName("D")
    @Test
    void test_Subtract() {

        System.out.println("test_Subtract()");
        assertEquals(10, 18 - 8);
    }

    @DisplayName("B")
    @Test
    void test_IsEven() {

        System.out.println("test_IsEven()");
        assertEquals(0, 18%2);
    }
}

We can see that the test methods are sorted alphanumerically based on their display name starting from A to E. The output of the above program

2. MethodName

This annotation sorts methods alphanumerically based on their names using String.compareTo(String).
If two methods have the same name, String representations of their formal parameter lists will be used as a fallback for comparing the methods.

@TestMethodOrder(MethodOrderer.MethodName.class)

Let us see an example of MethodName.

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import static org.junit.jupiter.api.Assertions.assertEquals;

@TestMethodOrder(MethodOrderer.MethodName.class)
class MethodNameOrderedTests {

    @Test
    void testE() {

        System.out.println("testE");
        assertEquals(10, 3 + 7);
    }

    @Test
    void testA() {

        System.out.println("testA");
        assertEquals(10, 14 - 4);
    }

    @Test
    void testC() {
        System.out.println("testC");
        assertEquals(10, 5 * 2);
    }

    @Test
    void testB() {
        System.out.println("testB");
        assertEquals(10, 30 / 3);
    }

    @Test
    void testD() {
        System.out.println("testD");
        assertEquals(10, 10 + 0);
    }

}

The output of the above program

3. OrderAnnotation

This sorts test method numerically based on values specified via the @Order annotation.
Any methods that are assigned the same order value will be sorted arbitrarily adjacent to each other.
When any method is not annotated with @Order, it will be assigned the default order value, which will effectively cause them to appear at the end of the sorted list.

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)

Let us see an example of OrderAnnotation.

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import static org.junit.jupiter.api.Assertions.assertEquals;

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class OrderAnnotationDemo {

    @Test
    @Order(3)
    void test_Add() {

        System.out.println("test_Add()");
        assertEquals(10, 3 + 7);
    }

    @Test
    @Order(4)
    void test_IsOdd() {
        System.out.println("test_IsOdd()");
        assertEquals(1, 11%2);
    }

    @Test
    void test_Subtract() {
        System.out.println("test_Subtract()");
        assertEquals(10, 14 - 4);
    }

    @Test
    @Order(1)
    void test_Multiply() {
        System.out.println("test_Multiply()");
        assertEquals(10, 5 * 2);
    }

    @Test
    @Order(4)
    void test_Divide() {
        System.out.println("test_Divide()");
        assertEquals(10, 30 / 3);
    }

    @Test
    @Order(2)
    void test_IsEven() {
        System.out.println("test_IsEven()");
        assertEquals(0, 10%2);
    }
}

Here, test_Subtract() is not assigned any order value, so it is displayed as the last one in the last.

4. Random

These sorts of test methods are pseudo-randomnly.

@TestMethodOrder(MethodOrderer.Random.class)

Let us create a program to show the random order of tests in JUnit5.

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import static org.junit.jupiter.api.Assertions.assertEquals;

@TestMethodOrder(MethodOrderer.Random.class)
public class OrderRandomDemo {

    @Test
    void test_Add() {

        System.out.println("test_Add()");
        assertEquals(10, 3 + 7);
    }

    @Test
    void test_Subtract() {

        System.out.println("test_Subtract()");
        assertEquals(10, 14 - 4);
    }

    @Test
    void test_Multiply() {

        System.out.println("test_Multiply()");
        assertEquals(10, 5 * 2);
    }

    @Test
    void test_Divide() {

        System.out.println("test_Divide()");
        assertEquals(10, 30 / 3);
    }

    @Test
    void test_IsEven() {

        System.out.println("test_IsEven()");
        assertEquals(0, 10%2);
    }
}

The output of the above program

5. Custom Order

We can define our own custom ordering sequence by implementing the interface MethodOrderer and providing it as the argument to @TestMethodOrder.

Here, the tests are arranged in descending method order.

import org.junit.jupiter.api.MethodDescriptor;
import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.MethodOrdererContext;

public class DescendingMethodOrder implements MethodOrderer {

    @Override
    public void orderMethods(MethodOrdererContext context) {
        context.getMethodDescriptors().sort((MethodDescriptor m1, MethodDescriptor m2) ->
                m2.getMethod().getName().compareTo(m1.getMethod().getName()));
    }
    
}

Now, test the above custom order.

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import static org.junit.jupiter.api.Assertions.assertEquals;

@TestMethodOrder(DescendingMethodOrder.class)
public class CustomOrderTests {

    @Test
    void test_Add() {
        System.out.println("test_Add()");
        assertEquals(10 , 4 + 6);
    }

    @Test
    void test_Subtract() {
        System.out.println("test_Subtract()");
        assertEquals(10 , 17 - 7);
    }

    @Test
    void test_Multiply() {
        System.out.println("test_Multiply()");
        assertEquals(10 , 5 * 2);
    }

    @Test
    void test_Divide() {
        System.out.println("test_Divide()");
        assertEquals(10 , 20/2);
    }

    @Test
    void test_IsEven() {
        System.out.println("test_IsEven()");
        assertEquals(0 , 20%2);
    }
}

Notice the test output. The tests are executed in descending order. The result of the above program is

Test Classes Ordering

  1. ClassName: sorts test classes alphanumerically based on their fully qualified class names.
  2. DisplayName: sorts test classes alphanumerically based on their display names (see display name generation precedence rules).
  3. OrderAnnotation: sorts test classes numerically based on values specified via the @Order annotation.
  4. Random: orders test classes pseudo-randomly and support the configuration of a custom seed.

The configured ClassOrderer will be applied to all top-level test classes (including static nested test classes) and @Nested test classes.

1. ClassName

The Test Classes are sorted alphanumerically based on their fully qualified class names.

package JUnit5;

import org.junit.jupiter.api.ClassOrderer;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestClassOrder;

import static org.junit.jupiter.api.Assertions.assertEquals;

@TestClassOrder(ClassOrderer.ClassName.class)
public class ClassNameOrderTests {

    @Nested
    class Addition {

        @Test
        void test_Add() {
            System.out.println("test_Add()");
            assertEquals(10, 3 + 7);
        }
    }

    @Nested
    class IsEven {

        @Test
        void test_IsEven() {
            System.out.println("test_IsEven()");
            assertEquals(0, 10 % 2);
        }
    }

    @Nested
    class Subtraction {

        @Test
        void test_Subtract() {
            System.out.println("test_Subtract()");
            assertEquals(9, 14 - 5);
        }
    }

    @Nested
    class Multiply {

        @Test
        void test_Multiply() {
            System.out.println("test_Multiply()");
            assertEquals(10, 5 * 2);
        }
    }
}

The result of the above program is

2. DisplayName

It sorts test classes alphanumerically based on their display names.

package JUnit5;

import org.junit.jupiter.api.*;

import static org.junit.jupiter.api.Assertions.assertEquals;

@TestClassOrder(ClassOrderer.DisplayName.class)
public class ClassDisplayNameTests {

    @Nested
    @DisplayName("B")
    class AppFlowTests {

        @Test
        void test_Add() {
            System.out.println("test_Add()");
            assertEquals(10, 6 + 4);
        }
    }

    @Nested
    @DisplayName("C")
    class TearDownTests {

        @Test
        void test_Subtract() {
            System.out.println("test_Subtract()");
            assertEquals(10, 15 - 5);

        }
    }

    @Nested
    @DisplayName("A")
    class SetupTests {

        @Test
        void test_Multiply() {
            System.out.println("test_Multiply()");
            assertEquals(10, 5 * 2);
        }
    }
}

The result of the above program is

3. OrderAnnotation in Class

The test classes are sorted numerically based on values specified via the @Order annotation.

import org.junit.jupiter.api.*;

import static org.junit.jupiter.api.Assertions.assertEquals;

@TestClassOrder(ClassOrderer.ClassName.class) //sorts test classes alphanumerically based on their fully qualified class names.
public class ClassOrderedTests {

    @Nested
    @Order(2)
    class AppFlowTests {

        @Test
        void test_Add() {
            System.out.println("test_Add()");
            assertEquals(10, 3 + 7);
        }
    }

    @Nested
    @Order(3)
    class TearDownTests {

        @Test
        void test_Subtract() {
            System.out.println("test_Subtract()");
            assertEquals(9, 14 - 5);
        }
    }

    @Nested
    @Order(1)
    class SetupTests {

        @Test
        void test_Multiply() {
            System.out.println("test_Multiply()");
            assertEquals(10, 5 * 2);
        }
    }
}

The result of the above program is

4. Random

The test classes are sorted pseudo-randomly and support the configuration of a custom seed.

import org.junit.jupiter.api.*;
import static org.junit.jupiter.api.Assertions.assertEquals;

@TestClassOrder(ClassOrderer.Random.class)
public class ClassRandomTests {

    @Nested
    class Addition {

        @Test
        void test_Add() {
            System.out.println("test_Add()");
            assertEquals(10, 3 + 7);
        }
    }

    @Nested
    class IsEven {

        @Test
        void test_IsEven() {
            System.out.println("test_IsEven()");
            assertEquals(0, 10 % 2);
        }
    }

    @Nested
    class Subtraction {

        @Test
        void test_Subtract() {
            System.out.println("test_Subtract()");
            assertEquals(9, 14 - 5);
        }
    }

    @Nested
    class Multiply {

        @Test
        void test_Multiply() {
            System.out.println("test_Multiply()");
            assertEquals(10, 5 * 2);
        }
    }
}

The result of the above program is

Congratulation!! We have gone through different types of ordering in JUnit5. Happy Learning!!

How to disable tests in JUnit5 – @Disabled

HOME

JUnit5 also provides the support to exclude/disable the tests. The @Disabled annotation in JUnit5 is used to exclude the test methods from the test suite. This annotation can be applied over a test class as well as over individual test methods. This annotation accepts only one optional parameter where we can mention the reason to skip the tests. @Disabled annotation can be used without providing any reason but its always good to provide a reason why this particular test case has been disabled, or issue tracker id for better understanding.

1.Disable Test Methods

In the below example, I have annotated 2 test methods out of 5 test methods as @Disabled with a parameter which specify the reason for disabling the test that means these 2 test methods should not be executed.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.jupiter.api.*;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import static org.junit.jupiter.api.Assertions.*;

class DisabledTestsDemo {

    WebDriver driver;

    @BeforeEach
    public void setUp() {
        
        WebDriverManager.chromedriver().setup();
        ChromeOptions chromeOptions = new ChromeOptions();
        driver = new ChromeDriver(chromeOptions);
        driver.manage().window().fullscreen();
        driver.get("http://automationpractice.com/index.php");

    }

    @Disabled("This test is not applicable for Sprint 14")
    @Test
    void verifyPopularLink() {

        boolean displayed = driver.findElement(By.xpath("//*[@id='home-page-tabs']/li[1]/a")).isDisplayed();
        assertTrue(displayed);

    }

    @Test
    void verifyContactNumber() {

        String contactDetail = driver.findElement(By.xpath("//span[@class='shop-phone']/strong")).getText();
        assertEquals("0123-456-789", contactDetail);

    }

    @Disabled("This test is blocked till bug 1290 is fixed")
    @Test
    void verifyWomenLink() {

        boolean enabled = driver.findElement(By.xpath("//*[@id='block_top_menu']/ul/li[1]/a")).isEnabled();
        assertTrue(enabled);
    }

    @Test
    void contactUSPageTest() {

        driver.findElement(By.xpath("//*[@id='contact-link']/a")).click();
        String contactPage = driver.getTitle();
        assertEquals("Contact us - My Store", contactPage);
    }

    @Test
    void signInPageTest() {

        driver.findElement(By.className("login")).click();
        String signInPage = driver.getTitle();
        assertEquals("Login - My Store", signInPage);
    }

    @AfterEach
    public void tearDown() {
        driver.close();
    }
}

The output of the above test shows, the 2 tests that are annotated with @Disabled are not executed.

2. Disable Test Class

When we annotate a class as @Disabled, then all the test methods present within that test class will not be executed.

In the below example, there are 2 test classes – Demo and DisabledTestsDemo. Demo class contains 1 test method whereas DisabledTestsDemo contains 5 test methods. I have annotated DisabledTestsDemo class as @Disabled which means all 5 tests present within it will not be executed.

In the below example, there is a base class that contains the initialization of webDriver as well as closing of the same.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;

public class Base {

    static WebDriver driver=null;

    @BeforeEach
    public void setUp() {

        WebDriverManager.chromedriver().setup();
        ChromeOptions chromeOptions = new ChromeOptions();
        driver = new ChromeDriver(chromeOptions);
        driver.manage().window().fullscreen();
        driver.get("http://automationpractice.com/index.php");

    }

    @AfterEach
    public void tearDown() {
        driver.close();
    }
}

Class 1 – Demo

import org.junit.jupiter.api.*;

public class Demo extends Base{

    @Test
    public void homePage() {
        Assertions.assertEquals("My Store",driver.getTitle());

    }
}

Class 2 – DisabledTestsDemo

import org.junit.jupiter.api.*;
import org.openqa.selenium.By;
import static org.junit.jupiter.api.Assertions.*;

@Disabled
class DisabledTestsDemo extends Base {

    @Test
    void verifyPopularLink() {

        boolean displayed = driver.findElement(By.xpath("//*[@id='home-page-tabs']/li[1]/a")).isDisplayed();
        assertTrue(displayed);

    }

    @Test
    void verifyContactNumber() {

        String contactDetail = driver.findElement(By.xpath("//span[@class='shop-phone']/strong")).getText();
        assertEquals("0123-456-789", contactDetail);

    }

    @Test
    void verifyWomenLink() {

        boolean enabled = driver.findElement(By.xpath("//*[@id='block_top_menu']/ul/li[1]/a")).isEnabled();
        assertTrue(enabled);
    }

    @Test
    void contactUSPageTest() {

        driver.findElement(By.xpath("//*[@id='contact-link']/a")).click();
        String contactPage = driver.getTitle();
        assertEquals("Contact us - My Store", contactPage);
    }

    @Test
    void signInPageTest() {

        driver.findElement(By.className("login")).click();
        String signInPage = driver.getTitle();
        assertEquals("Login - My Store", signInPage);
    }
}

The output of the above test shows, all the test methods present in the class that is annotated with @Disabled are not executed.

Congratulations. We are able to understand the usage of @Disabled annotation in JUnit5. Happy Learning!!

How to Retry Test in JUnit5 – @RepeatedTest

HOME

In JUnit5, JUnit Jupiter provides the ability to repeat a test a specified number of times by annotating a method with @RepeatedTest. We can specify the repetition frequency as a parameter to the @RepeatedTest annotation.

When do we use the Repeated Test?

Imagine, we are testing an online shopping website. In the process of placing an order, we need to pay for the product. But the payment gateway is a third-party service, and we cannot control the connectivity between our website and the payment gateway. We know that clicking on the ‘Payment’ link sometime shows “Exception: Page cannot be displayed”. This is an intermittent issue. So, we don’t want to fail the test if this payment link does not work. We can configure to click this payment link multiple times, before marking the test case failed. Here comes the Repeated Test in the picture.

A few points to keep in mind for @RepeatedTest are as follows:

  1. The methods annotated with @RepeatedTest cannot be static, otherwise, the test cannot be found.
  2. The methods annotated with @RepeatedTest cannot be private, otherwise, the test cannot be found.
  3. The return type of the method annotated with @RepeatedTest must be void only, otherwise, the test cannot be found.

The below example will run the test 5 times.

 @DisplayName("Addition")
 @RepeatedTest(3)
 void repeatTest(){
    int a = 4;
    int b = 6;
    assertEquals(10, a+b,"Incorrect sum of numbers");
 }

The output of the above test:

Each invocation of a repeated test behaves like the execution of a regular @Test method with full support for the same lifecycle callbacks and extensions.

It means that @BeforeEach and @AfterEach annotated lifecycle methods will be invoked for each invocation of the test.

@BeforeEach  annotation is used to signal that the annotated method should be executed before each invocation of the @Test@RepeatedTest@ParameterizedTest, or @TestFactory method in the current class. This is the replacement of the @Before Method in JUnit4.

TestInfo is used to inject information about the current test or container into @Test, @RepeatedTest, @ParameterizedTest, @TestFactory, @BeforeEach, @AfterEach, @BeforeAll, and @AfterAll methods.
If a method parameter is of type TestInfo, JUnit will supply an instance of TestInfo corresponding to the current test or container as the value for the parameter.

RepetitionInfo is used to inject information about the current repetition of a repeated test into the @RepeatedTest, @BeforeEach, and @AfterEach methods.
If a method parameter is of type RepetitionInfo, JUnit will supply an instance of RepetitionInfo corresponding to the current repeated test as the value for the parameter.

In the below example, @BeforeEach will get executed before each repetition of each repeated test. By having the TestInfo and RepetitionInfo injected into the method, we see that it’s possible to obtain information about the currently executing repeated test.

import org.junit.jupiter.api.*;
import static org.junit.jupiter.api.Assertions.*;

public class RepeatCycleDemo {

    @BeforeEach
    void init(TestInfo testInfo, RepetitionInfo repetitionInfo) {
      System.out.println("Before Each init() method called");
      int currentRepetition = repetitionInfo.getCurrentRepetition();
      int totalRepetitions = repetitionInfo.getTotalRepetitions();
      String methodName = testInfo.getTestMethod().get().getName();
      System.out.println(String.format("About to execute repetition %d of %d for %s", currentRepetition, totalRepetitions, methodName));
     }

    @RepeatedTest(3)
    void repeatedTestWithRepetitionInfo(RepetitionInfo repetitionInfo) {
      int a = 4;
      int b = 6;
      assertEquals(10, a+b, repetitionInfo.getTotalRepetitions());
    }

    @AfterEach
    public void cleanUpEach(){
      System.out.println("=================After Each cleanUpEach() method called =================");
    }
}

The output of the above test:

Custom Display Name

In addition to specifying the number of repetitions, a custom display name can be configured for each repetition via the name attribute of the @RepeatedTest annotation.

The display name can be a pattern composed of a combination of static text and dynamic placeholders. The following placeholders are currently supported.

  1. {displayName}: display name of the @RepeatedTest method
  2. {currentRepetition}: the current repetition count
  3. {totalRepetitions}: the total number of repetitions
    @BeforeEach
    void init(TestInfo testInfo, RepetitionInfo repetitionInfo) {
        System.out.println("Before Each init() method called");
        int currentRepetition = repetitionInfo.getCurrentRepetition();
        int totalRepetitions = repetitionInfo.getTotalRepetitions();
        String methodName = testInfo.getTestMethod().get().getName();
        System.out.println(String.format("About to execute repetition %d of %d for %s", //
                currentRepetition, totalRepetitions, methodName));
    }

   //Custom Display  
    @RepeatedTest(value = 2, name = "{displayName} {currentRepetition}/{totalRepetitions}")
    @DisplayName("Repeat JUnit5 Test")
    void customDisplayName(TestInfo testInfo) {
        assertEquals("Repeat JUnit5 Test 1/2", testInfo.getDisplayName());
    }

The output of the above test:

The default display name for a given repetition is generated based on the following pattern: “repetition {currentRepetition} of {totalRepetitions}”. Thus, the display names for individual repetitions of the previous repeatedTest() example would be repetition 1 of 10, repetition 2 of 10.

We can use one of two predefined formats for displaying the name – LONG_DISPLAY_NAME and SHORT_DISPLAY_NAME. The latter is the default format if none is specified.

  1. RepeatedTest.LONG_DISPLAY_NAME – {displayName} :: repetition {currentRepetition} of {totalRepetitions}
  2. RepeatedTest.SHORT_DISPLAY_NAME – repetition {currentRepetition} of {totalRepetitions}

Below is an example of SHORT_DISPLAY_NAME

    @RepeatedTest(value = 3, name = RepeatedTest.SHORT_DISPLAY_NAME)
    @DisplayName("Multiplication")
    void customDisplayNameWithShortPattern() {
        assertEquals(8, 6*5);
 }

In this case, the name is displayed as Multiplication repetition 1 of 3, repetition 2 of 3, and soon.

The output of the above test:

Below is an example of LONG_DISPLAY_NAME

  @RepeatedTest(value = 3, name = RepeatedTest.LONG_DISPLAY_NAME)
    @DisplayName("Addition")
    void customDisplayNameWithLongPattern(TestInfo testInfo) {
        System.out.println("Display Name :"+ testInfo.getDisplayName());
        System.out.println("Test Class Name :"+ testInfo.getTestClass());
        System.out.println("Test Method :"+ testInfo.getTestMethod());
        assertEquals(8, 3+7);
    }

The output of the above test:

As we can see in the example, we have used @DisplayName(“Addition”), but as it is name = RepeatedTest.LONG_DISPLAY_NAME, so the name of the tests are now Addition :: repetition 1 of 3, Addition :: repetition 2 of 3 and soon.

Congratulations. We are able to execute the tests multiple times by using the @RepeatedTest annotation. Happy Learning!!

How to run JUnit5 tests through Command Line

HOME

The previous tutorial explains to configure Junit in IntelliJ and run the tests as JUnit Tests. This tutorial shows the steps to run the tests through command line. We can ask, why we need to run the tests through command line?? There are many reasons, one of the reason is to achieve CI/CD. To run the tests in pipeline, they need to be run through command line. Another reason is that we don’t need to open the IDE to run the tests. Third reason is that many reports are only generated (Serenity, Cucumber), if the tests run through command line.

Below is a JUnit5 test.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.jupiter.api.*;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;

public class Demo {

    WebDriver driver;

    @BeforeEach
    public void setUp() {
        WebDriverManager.chromedriver().setup();
        ChromeOptions chromeOptions = new ChromeOptions();
        driver = new ChromeDriver(chromeOptions);
        driver.manage().window().fullscreen();
    }

    @Test
    public void Junit5Test() {
        driver.get("http://automationpractice.com/index.php");
        System.out.println("Title of Page :" + driver.getTitle());
        System.out.println("Page URL : " + driver.getCurrentUrl());
        Assertions.assertEquals("My Store",driver.getTitle());

    }

    @AfterEach
    public void tearDown() {
        driver.close();
    }
}

Let us see what happens when we try to run the JUnit tests through Command Line. This command is used to run the tests present in Demo class.

mvn clean test -Dtest=Demo

The output generated by the test is shown below

This shows that surefire-plugin is need to be add to the project to run t he tests successfully through command line.

Add surefire-plugin to the project

Go back to the Apache Maven Project and copy the code.

 <dependencies>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>5.8.2</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.8.2</version>
            <scope>test</scope>
        </dependency>
 
</dependencies>       
    
<build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M5</version>
                <dependencies>
                    <dependency>
                        <groupId>org.junit.jupiter</groupId>
                        <artifactId>junit-jupiter-engine</artifactId>
                        <version>5.8.2</version>
                    </dependency>
                </dependencies>
            </plugin>
        </plugins>
    </build>

Again run the tests through command line using the command mentioned above. Now, will see that the tests are executed successfully.

Congratulations. We can see that JUnit5 tests are executed through command line. Happy Learning!!

AssertJ – Fluent Assertions in Java

HOME

This tutorial describes the usage of the AssertJ – Fluent Assertions framework for writing tests in Java.

Introduction to AssertJ

The AssertJ project provides fluent assertion statements for test code written in Java. These assert statements are typically used with Java JUnit tests. 

AssertJ is composed of several modules:

A core module to provide assertions for JDK types (String, Iterable, Stream, Path, File, Map…​)
1. A Guava module to provide assertions for Guava types (Multimap, Optional…​)
2. A Joda Time module to provide assertions for Joda Time types (DateTime, LocalDateTime)
3. A Neo4J module to provide assertions for Neo4J types (Path, Node, Relationship…​)
4. A DB module to provide assertions for relational database types (Table, Row, Column…​)
5. A Swing module provides a simple and intuitive API for functional testing of Swing user interfaces

What is AssertJ Core?

AssertJ is a Java library that provides a rich set of assertions and truly helpful error messages, improves test code readability, and is designed to be super easy to use within any IDE.

AssertJ Core major versions depend on different Java versions:

  • AssertJ Core 3.x requires Java 8 or higher
  • AssertJ Core 2.x requires Java 7 or higher

AssertJ Core 3.x includes all AssertJ Core 2.x features and adds Java 8 specific ones (like exception assertions with lambdas).

Maven

<dependency>
    <groupId>org.assertj</groupId>
    <artifactId>assertj-core</artifactId>
    <version>3.22.0</version>
    <scope>test</scope>
</dependency>

Gradle

testImplementation 'org.assertj:assertj-core:3.22.0'

The Assertions class is the only class you need to start using AssertJ, it provides all the methods you need.

import static org.assertj.core.api.Assertions.*;
  1. Simple Assertion

Verify that age is greater or equal to 50. This assertion will fail

int age = 20;
assertThat(age).isGreaterThanOrEqualTo(30);

There is another way to perform the same test. Don’t import the static package.

import org.assertj.core.api.Assertions;
int age = 20;

// Verify that age is greater or equal to 50
Assertions.assertThat(age).isGreaterThanOrEqualTo(30);

This assertion will pass.

int age = 50;

// Verify that age is greater or equal to 50
Assertions.assertThat(age).isGreaterThanOrEqualTo(30);

2. Array Assertions

For an Iterable or an Array there are multiple ways of asserting that their content exist. One of the most common assertions would be to check if an Iterable or Array contains a given element:

int age = 30;
List<Integer> ages = Arrays.asList(20, 25, 33, 45);

// Verify that ages list contains age(30) or not
Assertions.assertThat(ages).contains(age);

Verify if a list is empty or not

	List<String> names = Arrays.asList("Here", "Keep", "Ugly", "door", "time");

   @Test
	public void assertJAssertionsExample8() {
		Assertions.assertThat(names).isEmpty();

	}

Verify if a List starts with a given character. For example “Ugly”:

	List<String> names = Arrays.asList("Here", "Keep", "Ugly", "door", "time");

	@Test
	public void assertJAssertionsExample8() {

		// Verify that ages list contains age(30) or not
		Assertions.assertThat(names).startsWith("Ugly");

	}

Assert the size of list

List<Integer> ages = Arrays.asList(20, 25, 33, 45);

// Verify that list ages contains 5 elements
Assertions.assertThat(ages).hasSize(5);

Chaining of assertions

AssertJ allows you to be concise by chaining multiple assertions.

int age = 30;
List<Integer> ages = Arrays.asList(20, 25, 33, 45);

// Verify that the list of age contains 20, and size of list is 4 and match the
// values of all elements
Assertions.assertThat(ages).contains(20).hasSize(4).allMatch(a -> a >= 10 && a <= 30);

3. Assertion description

It is often valuable to describe the assertion performed, especially for boolean assertions where the default error message just complains that it got false instead of true (or vice versa).

You can set such a description with as(String description, Object…​ args) but remember to do it before calling the assertion otherwise it is simply ignored as a failing assertion breaks the chained calls.

Example of a failing assertion with a description:

String name = "Happy Days are here";
Assertions.assertThat(name).as("check name").startsWith("Life");

The error message starts with the given description in [check name]

4. Assertions for Date

AssertJ provides special assertions for the Java date class.

LocalDateTime date1 = LocalDate.of(1992, 2, 14).atStartOfDay();
LocalDateTime date2 = LocalDate.of(1998, 1, 1).atStartOfDay();
Assertions.assertThat(date1).isEqualTo(date2);

LocalDateTime isAfter

LocalDateTime date1 = LocalDate.of(1992, 2, 14).atStartOfDay();
LocalDateTime date2 = LocalDate.of(1998, 1, 1).atStartOfDay();
Assertions.assertThat(date1).isAfter(date2);

LocalDateTime isBefore

LocalDateTime date1 = LocalDate.of(2025, 2, 14).atStartOfDay();
Assertions.assertThat(date1).isBefore(LocalDateTime.now());

5. Soft Assertions

Soft assertions AssertJ collects all assertion errors instead of stopping at the first one. Since soft assertions don’t fail at the first error, you need to tell AssertJ when to report the captured assertion errors, we are using assertAll().

SoftAssertions softly = new SoftAssertions();

softly.assertThat("George Martin").as("great authors").isEqualTo("JK Rowling");
softly.assertThat(42).as("comparison").isGreaterThan(120);
softly.assertThat("50").isEqualTo("23");

// Don't forget to call assertAll() otherwise no assertion errors are reported!
softly.assertAll();

6. Object Assertions

Objects can be compared in various ways either to determine equality of two objects or to examine the fields of an object.

public class AssertJEmployee {

	String name;
	int age;
	float salary;

	public AssertJEmployee(String name, int age, float salary) {
		super();
		this.name = name;
		this.age = age;
		this.salary = salary;
	}

	public String getName() {
		return name;
	}

	public void setName(String name) {
		this.name = name;
	}

	public int getAge() {
		return age;
	}

	public void setAge(int age) {
		this.age = age;
	}

	public float getSalary() {
		return salary;
	}

	public void setSalary(float salary) {
		this.salary = salary;
	}

}
@Test
public void test() {

	AssertJEmployee emp1 = new AssertJEmployee("Tim", 24, 24000);
	AssertJEmployee emp2 = new AssertJEmployee("Tim", 20, 24000);

	Assertions.assertThat(emp1).usingRecursiveComparison().isEqualTo(emp2);
}

In the below example, we have used isEqualTo() method that compares object references. We can see that both objects are same, but has different reference. So, the assertion fails here.

@Test
public void test() {

	AssertJEmployee emp1 = new AssertJEmployee("Tim", 24, 24000);
	AssertJEmployee emp2 = new AssertJEmployee("Tim", 24, 24000);

	Assertions.assertThat(emp1).isEqualTo(emp2);
}

Congratulation! We are able to understand the use of AssertJ. Happy Learning!!

JUnit5 Assertions Example

HOME

JUnit5 contains the assertions available as in JUnit 4 Assertions as well there are a few additional new asserts too. In this post, let’s discuss each new assertion in JUnit5 works in detail with examples.

1. assertIterableEquals

The assertIterableEquals() asserts that the expected and the actual iterables are deeply equal. In order to be equal, both iterable must return equal elements in the same order and it isn’t required that the two iterables are of the same type in order to be equal.

Example 1 – In this example, the number of elements as well as the sequence of elements is in the same order in both Iterables. It is not mandatory to have Iterables of the same type, so we can see as one of the Iterable is ArrayList whereas another Iteratble is of type LinkedList.

@Test
void iterableEqualsPositive() {
     Iterable<String> iterat1 = new ArrayList<>(asList("Java", "Junit", "Test"));
     Iterable<String> iterat2 = new LinkedList<>(asList("Java", "Junit", "Test"));

     assertIterableEquals(iterat1, iterat2);
  }

The assertion passes as the sequence and number of elements in both Iterables are the same.

Example 2 – In the below example, the ordering of elements in the Iterables is different.

@Test
void iterableEqualsNegative() {
     Iterable<String> iterat1 = new ArrayList<>(asList("Java", "Junit", "Test"));
     Iterable<String> iterat2 = new ArrayList<>(asList("Java","Test","Junit" ));

     assertIterableEquals(iterat1, iterat2);
 }

Here, we can see that the sequence of elements in Iterable are different, so the assertion fails.

Example 3 – In the below example, the number of elements is different in the Iterables. Iterable 1 has 3 elements whereas Iterable 2 has 4 elements.

@Test
void iterableEqualsNegative1() {
     Iterable<String> iterat1 = new ArrayList<>(asList("Java", "Junit", "Test"));
     Iterable<String> iterat2 = new LinkedList<>(asList("Java", "Junit", "Test","Junit5"));

     assertIterableEquals(iterat1, iterat2);
 }

As both Iterables do not have same number of elements, so the Assertion has failed.

Note:- There are no assertions like assertNotIterableEquals() or assertIterableNotEquals().

2. assertLinesMatch

This Assertion asserts that the expected list of Strings matches the actual list of String.
This method differs from other assertions that effectively only check String.equals(Object), in that it uses the following staged matching algorithm:
For each pair of expected and actual lines do
a) check if expected.equals(actual) – if yes, continue with next pair
b) otherwise treat expected as a regular expression and check via String.matches(String) – if yes, continue with the next pair
c) otherwise check if an expected line is a fast-forward marker, if yes apply to fast-forward actual lines accordingly (see below) and goto 1.

Example 1 – In the below example, expected has a regular expression that matches with the elements of actual.

@Test
void linesMatchPositive() {
    List<String> expected = asList("Java", "\\d+", ".*");
    List<String> actual = asList("Java", "11", "JUnit");

    assertLinesMatch(expected, actual);
}

As the regular expression of elements matches, the assertion passes.

Example 2 – In the below example, the elements in the lists are different.

@Test
void linesMatchNegative1() {
   List<String> expected = asList("Java", "\\d+", ".*");
   List<String> actual = asList("Test","Java", "11");

   assertLinesMatch(expected, actual);
}

The assertion fails as the elements are different in actual and expected.

Example 3 – In the below example, the number of elements is different in expected and actual lists.

@Test
void linesMatchNegative2() {
    List<String> expected = asList("Java", "\\d+", ".*");
    List<String> actual = asList("Java", "11");

    assertLinesMatch(expected, actual);
}

The assertion fails as the number of elements in expected is 3 whereas 2 elements are available in actual list.

3. assertThrows

The new assertThrows() assertion allows us a clear and a simple way to assert if an executable throws the specified exception type.

Example 1 – In the below example, the length of string arr is null, so it throws a NullPointerException

@Test
void exceptionTestingPositive() {

    String arr = null;
    Exception exception = assertThrows(NullPointerException .class, () -> arr.length() );
    assertEquals(null, exception.getMessage());
 }

Result

Example 2 – In the below example, the exception thrown by String arr is NullPointerException. But, we are asserting it with ArithmeticException.

@Test
 void exceptionTestingNegative() {

     String arr = null;
     Exception exception = assertThrows(ArithmeticException .class, () -> arr.length() );
     assertEquals("Arithmetic Exception", exception.getMessage());
 }

Result

4. assertTimeout

When we want to assert that execution of the supplied executable completes before the given timeout, we can use assertTimeout().

Example 1 – In the below example, assertTimeout() is 2 sec, which means the assertion should be completed within 2 secs. We are waiting for 1 sec and then perform the assertion.

 @Test
     void assertTimeoutPositive() {
       int a = 4;
       int b= 5;
        assertTimeout(
                ofSeconds(2),
                () -> {
                    // code that requires less then 2 seconds to execute
                    Thread.sleep(1000);
                }
        );
        assertEquals(9, (a + b));
    }

As the assertion is within the specified time of assertTimeout(), the timeout assertion passes and the test passes.

Example 2 – In the below example, assertTimeout() is 2 sec whereas are waiting for 5 sec and then performing the assertion.

  @Test
   void assertTimeoutNegative() {
        int a = 4;
        int b= 5;
        assertTimeout(
                ofSeconds(2),
                () -> {
                    // code that requires less then 2 seconds to execute
                    Thread.sleep(5000);
                }
        );
        assertEquals(9, (a + b));
    }

As the assertion is outside the specified time of assertTimeout(), so the test fails. The assertion fails with an error message similar to: “execution exceeded timeout of 2000 ms by 3010 ms”.

Example 3 – In the below example, the assertion is mentioned just after

The executable will be executed in the same thread as that of the calling code. Consequently, execution of the executable will not be preemptively aborted if the timeout is exceeded.

@Test
 void assertTimeoutNegative1() {
    int a = 4;
    int b= 5;
    assertTimeout(
        ofSeconds(2),
        () -> {
                // code that requires less then 2 seconds to execute
                 Thread.sleep(5000);
                 assertEquals(10, (a + b));
              }
        );
    }

This shows that the assertion assertEquals() is still executed after the timeout also.

5. assertTimeoutPreemptively()

This assertion works just like assertTimeout(). When we want to assert that the execution of the supplied executable completes before the given timeout, we can use assertTimeoutPreemptively(). The only difference is that here the executable will be executed in a different thread than that of the calling code, whereas in assertTimeout() the executable will be executed in the same thread as that of the calling code. Furthermore, execution of the executable will be preemptively aborted if the timeout is exceeded here as contrary to assertTimeout() where the executable will not be preemptively aborted.

@Test
    void assertPreemptiveTimeoutNegative() {
        int a = 4;
        int b= 5;
        assertTimeoutPreemptively(
                ofSeconds(2),
                () -> {
                    // code that requires less then 2 seconds to execute
                    Thread.sleep(5000);
                    assertEquals(9, (a + b));
                }
        );
    }

Result

There is another very famous assertion called assertAll() [Group Assertions]. This is discussed in another tutorial.

In this post, We saw that JUnit Jupiter comes with many of the assertion methods that JUnit 4 has and adds a few that lend themselves well to being used with Java 8 lambdas. All JUnit Jupiter assertions are static methods in the org.junit.jupiter.api.Assertions class.

How to configure Junit in Intellij

HOME

In this tutorial we will discuss to create a JUnit  project using IntelliJ. We will be at first creating a simple Java Project and will add JUnit5 as well as create a Maven Project, then we will add a basic Class and a JUnit Test for it.

Java Project

Step 1 – Create a new Java Project.

To create a new Java project in Intellij, please refer to this tutorial.

Step 2 – Right click on the project and select Open Module Settings.

Step 3 – Go to the “Libraries” group, click the little plus (look up), and choose “From Maven…” option.

Step 4 – Search for “junit” — something like “junit:junit-4.13“. Click the “OK” button.

Step 5 – A new dialog will appear to confirm that “junit:junit:4.13.2” will be added to the module. Click the “OK” button.

Step 6 – This screens shows that junit:junit:4.13.2 is added to the Libraries. It contains the highlighted classes – junit-4.13.2.jar and hamcrest-core-1.3.jar. Click the “OK” button.

Step 7 – This image shows that the Junit is added to the External Libraries.

Step 8 – Create a Java Class – JUnit4Test under src and create a JUnit test to verify that it is installed properly.

import org.junit.Assert;
import org.junit.Test;

public class JUnit4Test {

    @Test
    public void Test() {

        String str1 = "Happy";
        String str2 = new String("Happy");
        Assert.assertEquals("String1 and String 2 are equal",str1, str2);

    }
}

Step 9 – There are many ways to run the test. One of the way is to Right-Click and select Run JUnit4Test

The successful execution of the test shows that the JUnit is configured properly.

Maven Project

Add Junit dependency to the POM.xml and build the project.

<dependencies>

    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.13.2</version>
        <scope>test</scope>
    </dependency>
    
</dependencies>

Now we need to apply the changes in the build script. Press Ctrl+Shift+O or click Load Maven Changes in the notification that appears in the top-right corner of the editor.

Create a Java Class – JUnit4Test under src/test/java and create a JUnit test to verify that it is installed properly.

import org.junit.Test;

import static org.junit.Assert.assertArrayEquals;

public class JUnitMavenTest {

    @Test
    public void Test() {

        String[] expected = {"happy","days","summer","spring"};
        String[] actual = {"happy","days","summer","spring"};

        assertArrayEquals("Expected and Actual Arrays are not equal",expected,actual);

    }
}

Similarly, to add JUnit5 we can add below mentioned dependencies to the POM.xml.

<dependency>
     <groupId>org.junit.jupiter</groupId>
     <artifactId>junit-jupiter-engine</artifactId>
     <version>5.8.2</version>
     <scope>test</scope>
</dependency>

<dependency>
     <groupId>org.junit.jupiter</groupId>
     <artifactId>junit-jupiter-api</artifactId>
     <version>5.8.2</version>
     <scope>test</scope>
</dependency>

Congratulations. We are able to add JUnit to Java or Maven project. Happy Learning!!

Grouped Assertions in JUnit 5 – assertAll()

HOME

What is JUnit5?

JUnit 5 is composed of several different modules from three different sub-projects.

JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage

The JUnit Platform serves as a foundation for launching testing frameworks on the JVM. It also defines the Test Engine  API for developing a testing framework that runs on the platform.

JUnit Jupiter is the combination of the new programming model and extension model for writing tests and extensions in JUnit 5. The Jupiter sub-project provides a Test Engine for running Jupiter based tests on the platform.

JUnit Vintage provides a Test Engine for running JUnit 3 and JUnit 4 based tests on the platform. It requires JUnit 4.12 or later to be present on the class path or module path.

To use JUnit5, add the Junit5 maven dependency to the POM.xml

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-engine</artifactId>
    <version>5.8.2</version>
    <scope>test</scope>
</dependency>

<dependency>
	<groupId>org.junit.jupiter</groupId>
	<artifactId>junit-jupiter-api</artifactId>
    <version>5.8.2</version>
	<scope>test</scope>
</dependency>

JUnit Jupiter comes with many of the assertion methods that JUnit 4 already has and adds a few more that lend themselves well to being used with Java 8 lambdas. All JUnit Jupiter assertions are static methods. They come from class:

org.junit.jupiter.api.Assertions

JUnit 5 supports an additional Assertion feature called Grouped assertions. In JUnit4, when you have to assert let’s say – 3 different conditions, then you need to write 3 different assertions and they will be executed sequentially. Imagine, 2nd assertion fails, then the program stops there and will not go to the 3rd assertion. To overcome this problem, JUnit5 has a new assertion called Grouped assertion that execute all the conditions present within it using assertAll(), irrespective of the fact if any assertion fails or not and generate a consolidated report with all the failures. To be honest, I love this feature and use very frequently.

Grouped Assertions With Heading As Parameter

Example 1– Below is an example of positive assertAll(). In the below example, all assertions are of same type assertEquals() within assertAll() assertion. It consists of the heading parameter with the value “GroupedAssertionsWithSameAssertionType”.

@Test
void allPositive1() {
   assertAll(
     "GroupedAssertionsWithSameAssertionType",
      () -> assertEquals(8, 5+3, "8 is not sum of 5 and 3"),
      () -> assertEquals("java", "JAVA".toLowerCase()),
      () -> assertEquals(16,4*4,"16 is not product of 4 and 4")
   );
}

Result

As all 3 assertions pass, so the final result passes.

Example2 – In the below example, the assertions are of different types assertEquals(), assertNotNull and assertNotEquals() within assertAll() assertion. It consists of the heading parameter with the value “GroupedAssertionsWithDifferentAssertionType”.

@Test
void allPositive2() {

    String str ="Spring";
    assertAll(
        "GroupedAssertionsWithDifferentAssertionType",
         () -> assertEquals(8, 5+3, "8 is not sum of 5 and 3"),
         () -> assertNotNull(str, () -> "The string should be null"),
         () -> assertEquals("java", "JAVA".toLowerCase()),
         () -> assertNotEquals(20,5*3,"20 is product of 5 and 3")
    );
 }

Result

As all 3 assertions pass, so the final result passes.

Example 3 – In the below example, out of 4 assertions, 3 assertions are failing, so the output will have the detail about all 3 assertion errors.

 @Test
 void allNegative() {

    String str ="Spring";
    Iterable<String> iterat1 = new ArrayList<>(asList("Java", "Junit4", "Test"));
    Iterable<String> iterat2 = new ArrayList<>(asList("Java", "Junit5", "Test"));
    // In a grouped assertion all assertions are executed, and all failures will be reported together
    assertAll(
        "Negative-GroupedAssertionsWithDifferentAssertionType",
        () -> assertIterableEquals(iterat1, iterat2),
        () -> assertNull(str, () -> "The string should be null"),
        () -> assertEquals("java", "JAVA"),
        () -> assertNotEquals(20,5*3,"20 is product of 5 and 3")
     );
 }

Result

As one of the assert in the group fails, instead of AssertionFailureError it results in MultipleFailuresError thereby displaying the heading of the grouped assertion passed as the input parameter i.e. Negative-GroupedAssertionsWithDifferentAssertionType in this example. This image shows all the 3 assertion failures.

Assertion 1 fails as we were expecting JUnit4 , but response has JUnit5
Assertion 2 fails as string was not NULL.
Assertion 3 fails as java is not equal to JAVA (case sensitivity).

Grouped Assertions Without Heading As Parameter

The assertAll () can be implemented without using the heading parameter. The below example is same as the above one, we are just skipping the heading part.

@Test
void allNegative() {

      String str ="Spring";
      Iterable<String> iterat1 = new ArrayList<>(asList("Java", "Junit4", "Test"));
      Iterable<String> iterat2 = new ArrayList<>(asList("Java", "Junit5", "Test"));
      // In a grouped assertion all assertions are executed, and all failures will be reported together
      assertAll(
          () -> assertIterableEquals(iterat1, iterat2),
          () -> assertNull(str, () -> "The string should be null"),
          () -> assertEquals("java", "JAVA"),
          () -> assertNotEquals(20,5*3,"20 is product of 5 and 3")
      );
  }

Result

The result displays without heading.

Nested Or Dependent Grouped Assertions

When one assertAll() includes one or more assertAll() then these are referred to as a Nested or Dependent grouped assertions.

Example 1 – In this case, first the assertAll() validates if sum is correct or not. The sum is correct here, so the control moves to the nested assertAll() to verify all the assertions present within it.

@Test
void allDependentPositive() {

   String str ="Spring";

   // Within a code block, if an assertion fails the subsequent code in the same block will be skipped.
    assertAll(
         "DependentPositiveAssertions",
         () -> {
                assertEquals(8, 5 + 3, "8 is not sum of 5 and 3");

                // Executed only if the previous assertion is valid.
                assertAll("sub-heading",
                   () -> assertNotNull(str, () -> "The string should be null"),
                   () -> assertEquals("java", "JAVA".toLowerCase()),
                   () -> assertEquals(20, 5 * 4, "20 is product of 5 and 4")
                ); // end of inner AssertAll()
           }

      );  // end of outer AssertAll()
 }

Result

All the assertions within nested assertAll() are passes. So the final result passes.

Example 2 – In the below example, outer AssertAll() fails, so all the assertions within nested/dependent assertAll() are not executed.

 @Test
 void allDependentNegative() {

    String str ="Spring";

    // Within a code block, if an assertion fails the subsequent code in the same block will be skipped.
    assertAll(
       "DependentPositiveAssertions",
        () -> {
                assertEquals(8, 5 + 4, "8 is not sum of 5 and 3");

                // Executed only if the previous assertion is valid.
                assertAll("sub-heading",
                   () -> assertNull(str, () -> "The string should be null"),
                   () -> assertNotEquals("java", "JAVA".toLowerCase()),
                   () -> assertNotEquals(20, 5 * 4, "20 is product of 5 and 4")
               ); // end of inner AssertAll()
           }

        ); // end of outer AssertAll()
    }

Result

Example 3 – In the below example, outer AssertAll() passes, so all the assertions within nested/dependent assertAll() are executed.

@Test
void allDependentNegative1() {

    String str ="Spring";

   // Within a code block, if an assertion passes the subsequent code in the same block will be executed.
    assertAll(
      "DependentNegativeAssertions",
      () -> {
             assertEquals(8, 5 + 3, "8 is not sum of 5 and 3");

             // Executed only if the previous assertion is valid.
             assertAll("sub-heading",
               () -> assertNull(str, () -> "The string should be null"),
               () -> assertNotEquals("java", "JAVA".toLowerCase()),
               () -> assertNotEquals(20, 5 * 4, "20 is product of 5 and 4")
             ); // end of inner AssertAll()
         }

      ); // end of outer AssertAll()
    }

Result

The nested assertions have failed. So they can be seen in the execution status.

In short,

  1. When first assertAll() method passes, then all the subsequent assertions within that block will be executed and these assertions can further pass or fails.
  2. When the first assertAll() assertion fails, then the execution of subsequent assertions are skipped.

That’s it! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!

How to run Rest API Tests in GitLab CI/CD

HOME

This tutorial explains the process to run the Rest API Tests in GitLab pipeline. This is a significant step towards achieving CI/CD. Ideally, the tests need to run after any change (minor/major) before merging the change to master branch. Suppose there are 100 changes in a day, and any QA won’t want to start the tests manually 100 times in a day. So, now adding tests to GitLab pipeline comes to the picture. We can add a test stage to the pipeline and the tests will run automatically when the pipeline run, or we can schedule the tests to run automatically every hour or day using GitLab pipeline.

Pre-Requisite:

  1. Rest Assured – 4.3.3
  2. Java 11
  3. Maven / Gradle
  4. TestNG /JUnit
  5. GitLab account

To use GitLab CI/CD, we need to keep 2 things in mind:-

a) Make sure a runner is available in GitLab to run the jobs. If there is no runner, install GitLab Runner and register a runner for your instance, project, or group.

b) Create a .gitlab-ci.yml file at the root of the repository. This file is where you define your CI/CD jobs.

Step 1 – Create a new Maven Project

Step 2 – Add the below mentioned pom.xml that shows all the dependencies need to add to the project

<dependencies>
      <!-- https://mvnrepository.com/artifact/org.testng/testng -->
      <dependency>
         <groupId>org.testng</groupId>
         <artifactId>testng</artifactId>
         <version>7.4.0</version>
         <scope>test</scope>
      </dependency>
      <!-- https://mvnrepository.com/artifact/io.rest-assured/rest-assured -->
      <dependency>
         <groupId>io.rest-assured</groupId>
         <artifactId>rest-assured</artifactId>
         <version>4.3.3</version>
         <scope>test</scope>
      </dependency>
      <!-- https://mvnrepository.com/artifact/org.json/json -->
      <dependency>
         <groupId>org.json</groupId>
         <artifactId>json</artifactId>
         <version>20210307</version>
      </dependency>
   </dependencies>
   <build>
      <plugins>
         <!-- Compiler plug-in -->
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.5.1</version>
            <configuration>
               <source>11</source>
               <!--For JAVA 8 use 1.8-->
               <target>11</target>
               <!--For JAVA 8 use 1.8-->
            </configuration>
         </plugin>
         <!-- Added Surefire Plugin configuration to execute tests -->
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            <configuration>
               <suiteXmlFiles>
                  <suiteXmlFile>testng.xml</suiteXmlFile>
               </suiteXmlFiles>
            </configuration>
         </plugin>
      </plugins>
   </build>
</project>

It is needed to add maven-surefire plugin to run the TestNG tests through command line. To know more about this, please refer to this tutorial.

Step 3 – Create the Test Code to test the Rest API

Here, 2 tests are created. One of the tests get all the employee data (GET) whereas another test creates an employee (POST).

import static io.restassured.RestAssured.given;
import static org.hamcrest.Matchers.equalTo;

import org.json.JSONObject;
import org.testng.annotations.Test;

import io.restassured.http.ContentType;

public class RestAPIDemo {

	@Test(description = "To get the details of employee with id 2", priority = 0)
	public void verifyUser() {

		// Given
		given()
				// When
				.when().get("http://dummy.restapiexample.com/api/v1/employee/2")
				// Then
				.then().statusCode(200).statusLine("HTTP/1.1 200 OK")
				// To verify booking id at index 3
				.body("data.employee_name", equalTo("Garrett Winters"))
				.body("message", equalTo("Successfully! Record has been fetched."));
	}

	@Test(description = "To create a new employee", priority = 1)
	public void createUser() {

		JSONObject data = new JSONObject();

		// Map<String, String> map = new HashMap<String, String>();

		data.put("employee_name", "APITest");
		data.put("employee_salary", "99999");
		data.put("employee_age", "30");

		// GIVEN
		given().baseUri("http://dummy.restapiexample.com/api").contentType(ContentType.JSON).body(data.toString())

				// WHEN
				.when().post("/v1/create")

				// THEN
				.then().statusCode(200).body("data.employee_name", equalTo("APITest"))
				.body("message", equalTo("Successfully! Record has been added."));

	}
}

Step 4 – Create testng.xml to run the tests through TestNG

Now, let’s create a testng.xml to run the TestNG tests. If JUnit is used instead of TestNG, then this step is not needed.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test name="Test">
    <classes>
      <class name="com.example.RestAssured_TestNG_Demo.RestAPIDemo"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

Step 5 – Run the tests through command line

Now, let us execute the tests through command line. Go to the place where pom.xml of the project is placed and use the below command to run the tests. This step makes sure that all the test are running as expected.

GitLab Section

Step 6 – Create a blank project in GitLab

Refer to this tutorial to create a new blank project – How to create a new project in GitLab

Step 7 – Push the project from the local repository to GitLab Repository

Refer to this tutorial to push the changes – How to push new local GIT Repository to GitLab

Step 8 – Create .gitlab-ci.yml file in the project in GitLab

It is a YAML file where you configure specific instructions for GitLab CI/CD. In the .gitlab-ci.yml, we can define:

  • The scripts you want to run.
  • Other configuration files and templates you want to include.
  • Dependencies and caches.
  • The commands you want to run in sequence and those you want to run in parallel.
  • The location to deploy your application.
  • Whether you want to run the scripts automatically or trigger any of them manually.
image: adoptopenjdk/maven-openjdk11

stages:
  - test

variables:
  MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"

test:
  stage: test
  allow_failure: true

# Run the tests
  script:
    - mvn $MAVEN_OPTS clean package
    - mvn compile test

# Store artifacts
  artifacts:
    when: always
    name: "report"
    paths:
    - target/surefire-reports/*
    expire_in: 1 h

Step 9 – Run the tests in the GitLab pipeline

Now, when a new change is committed, a pipeline kicks off and it runs all the tests.

Step 10 – Check the status of the pipeline

Once the Status of the pipeline changes to either failed or passed.. that means the tests are already executed. Here, the pipeline is passed with brown color means that the execution of the test is completed with some failures.

I have added an artifact in the gitalb-ci.yml with the name “report”. This artifact creates a folder with the name “report” and the reports in this folder come from the path /target/surefire-reports. This artifact gives us an option to download the reports or browse the report. This report will be available for 1 hour only as mentioned in the gitlab-ci.yml.

Step 11 – Download the report

Click on the Download button and the report zip file is downloaded. Unzip the folder, and it contains all different types of surefire-reports.

Example of Emailable-Report.html

Example of Index.html

Congratulations. This tutorial has explain the steps to run Selenium tests in GitLab CI/CD. Happy Learning!!

JUnit Tutorials

HOME

JUnit is an open source Unit Testing Framework for JAVA.
JUnit is a simple framework to write repeatable tests. It is an instance of the xUnit architecture for unit testing frameworks.

JUnit4

Chapter 1 How to configure Junit in Intellij
Chapter 2 How to run JUnit5 tests through Command Line
Chapter 3 JUnit4 Assertions
Chapter 4 Integration of Cucumber with Selenium and JUnit
Chapter 5 Integration of Serenity with Cucumber6 and JUnit5
Chapter 6 Integration of Serenity with JUnit4
Chapter 7 Rest API Test in Cucumber BDD

JUnit5

Chapter 1 JUnit5 Assertions Example
Chapter 2 Grouped Assertions in JUnit 5 – assertAll()
Chapter 3 How to Retry Test in JUnit5 – @RepeatedTest
Chapter 4 How to disable tests in JUnit5 – @Disabled
Chapter 5 How to run JUnit5 tests in order
Chapter 6 Integration of Serenity with JUnit5
Chapter 7 Integration of Serenity with Cucumber6 and JUnit5