Difference between JUnit4 and JUnit5

HOME

In this article, we’ll see an overview of the differences between the two versions of the library.

1. Architecture

JUnit 4 has everything bundled into a single jar file whereas JUnit 5 is composed of 3 sub-projects i.e. JUnit Platform, JUnit Jupiter, and JUnit Vintage.

JUnit4

<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.13.2</version>
    <scope>test</scope>
</dependency>

JUnit Platform: It defines the TestEngine API for developing new testing frameworks that run on the platform.
JUnit Jupiter: It has all new JUnit annotations and TestEngine implementation to run tests written with these annotations.
JUnit Vintage: To support running JUnit 3 and JUnit 4 written tests on the JUnit 5 platform.

JUnit5

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-engine</artifactId>
    <version>5.9.0-M1</version>
    <scope>test</scope>
</dependency>
 
<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-api</artifactId>
    <version>5.9.0-M1</version>
    <scope>test</scope>
</dependency>

<dependency>
    <groupId>org.junit.vintage</groupId>
    <artifactId>junit-vintage-engine</artifactId>
    <version>5.9.0-M1</version>
    <scope>test</scope>
</dependency>

2. JDK Version

JUnit 4 requires Java 5 (or higher) whereas JUnit 5 requires Java 8 (or higher).

3. Imports

JUnit 5 uses the org.JUnit package for its annotations and classes whereas JUnit 5 uses the new org.JUnit.jupiter package for its annotations and classes. For example, org.JUnit.Test becomes org.JUnit.jupiter.api.Test.
@Before annotation of JUnit4 is renamed to @BeforeEach in JUnit5
@After annotation of JUnit4 is renamed to @AfterEach in JUnit5
@BeforeClass annotation of JUnit4 is renamed to @BeforeAll in JUnit5
@AfterClass annotation of JUnit4 is renamed to @AfterAll in JUnit5

4. Assertions

JUnit 5 assertions are now in org.JUnit.jupiter.api.Assertions whereas JUnit4 assertions are in org.JUnit.Assert. Most of the common assertions, like assertEquals() and assertNotNull() look the same as before, but there are a few key differences:

  • The error message is now the last argument, for example, assertEquals(“my message”, 1, 2) would be assertEquals(1, 2, “my message”)
  • Most assertions now accept a lambda that constructs the error message, which is only called when the assertion fails. Below is an example of the same.
    @Test
    void nullNegative() {
        String str = "Summer";

        Assertions.assertNull(str, () -> "The string should be null");
    }

The output of the above program is

  • assertTimeout() and assertTimeoutPreemptively() have replaced the @Timeout annotation (note that there is a @Timeout annotation in JUnit 5, but it works differently than JUnit 4).
  • There are several new assertions in JUnit5- assertAll(), assertIterableEquals(), assertLinesMatch(), assertThrows() and assertDoesNotThrow(). To know more about assertions in JUnit5, please refer to this tutorial – JUnit5 Assertions Example

5. Assumptions

In Junit 4, org.junit.Assume contains methods for stating assumptions about the conditions in which a test is meaningful. It has the following five methods:

  • assumeFalse()
  • assumeNoException()
  • assumeNotNull()
  • assumeThat()
  • assumeTrue()

JUnit5 has the following three methods:

  • assumeFalse()
  • assumingThat​()
  • assumeTrue()

Below is an example of assumeThat() annotation in JUnit5.

    @Test
    void assumingThatTest() {
        System.setProperty("ENV", "UAT");
        assumingThat(
                "UAT".equals(System.getProperty("ENV")),
                () -> {
                    // Since the condition is true, this assertion will get executed
                    System.out.println("Assuming that executable executed");
                    assertEquals((num1+num2),num4,"The product of "+ num1 +" and "+ num2 +" is not equal to "+num4);
                });
        
        System.out.println("Loop outside");
        assertEquals((num5-num2),num6,"The difference of "+ num5 +" and "+num2+" is not equal to " + num6);
    }

The output of the above program is

6. Conditional Test Execution

In JUnit4, @Ignore is used to skip the execution of a test whereas @Disabled or one of the other built-in execution conditions is used to skip the execution of the test in JUnit5. To know more about skipping the tests in JUnit5, please refer to this tutorial – How to disable tests in JUnit5 – @Disabled.

Below is an example of @Disabled in JUnit5.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.jupiter.api.*;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import static org.junit.jupiter.api.Assertions.*;
 
class DisabledTestsDemo {
 
    WebDriver driver;
 
    @BeforeEach
    public void setUp() {
         
        WebDriverManager.chromedriver().setup();
        ChromeOptions chromeOptions = new ChromeOptions();
        driver = new ChromeDriver(chromeOptions);
        driver.manage().window().fullscreen();
        driver.get("http://automationpractice.com/index.php");
 
    }
 
    @Disabled("This test is not applicable for Sprint 14")
    @Test
    void verifyPopularLink() {
 
        boolean displayed = driver.findElement(By.xpath("//*[@id='home-page-tabs']/li[1]/a")).isDisplayed();
        assertTrue(displayed);
    }
 
    @Test
    void verifyContactNumber() {
 
        String contactDetail = driver.findElement(By.xpath("//span[@class='shop-phone']/strong")).getText();
        assertEquals("0123-456-789", contactDetail);
    }
 
    @Disabled("This test is blocked till bug 1290 is fixed")
    @Test
    void verifyWomenLink() {
 
        boolean enabled = driver.findElement(By.xpath("//*[@id='block_top_menu']/ul/li[1]/a")).isEnabled();
        assertTrue(enabled);
    }
 
    @AfterEach
    public void tearDown() {
        driver.close();
    }
}

The output of the above program is

JUnit 5 provides the ExecutionCondition extension API to enable or disable a test or container (test class) conditionally. This is like using @Disabled on a test but it can define custom conditions. There are multiple built-in conditions, such as:

  • @EnabledOnOs and @DisabledOnOs: Enables a test only on specified operating systems.
  • @EnabledOnJre and @DisabledOnJre: Specifies the test should be enabled or disabled for specific versions of Java.
  • @EnabledIfSystemProperty: Enables a test based on the value of a JVM system property.
  • @EnabledIf: Uses scripted logic to enable a test if scripted conditions are met.

7. Extending JUnit

@RunWith no longer exists; superseded by @ExtendWith in JUnit5.

In JUnit 4, customizing the framework generally meant using a @RunWith annotation to specify a custom runner. Using multiple runners was problematic, and usually required chaining or using a @Rule. This has been simplified and improved in JUnit 5 using extensions.

import net.serenitybdd.core.Serenity;
import net.serenitybdd.junit5.SerenityJUnit5Extension;
import net.thucydides.core.annotations.Managed;
import net.thucydides.core.annotations.Steps;
import net.thucydides.core.annotations.Title;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.openqa.selenium.WebDriver;
import static org.assertj.core.api.Assertions.assertThat;
 
 @ExtendWith(SerenityJUnit5Extension.class)
 class ApplicationLoginJUnit5Tests {
 
        @Managed
        WebDriver driver;
 
        @Steps
        NavigateAction navigateAction;
 
        @Steps
        StepLoginPage loginPage;
 
        @Test
        @Title("Login to application with valid credentials navigates to DashBoard page")
 
         void successfulLogin() {
 
            navigateAction.toTheHomePage();
 
            // When
            loginPage.inputUserName("Admin");
            loginPage.inputPassword("admin123");
            loginPage.clickLogin();
 
            // Then
            Serenity.reportThat("Passing valid credentials navigates to DashBoard page",
                    () -> assertThat(dashboardPage.getHeading()).isEqualToIgnoringCase("DashBoard"));
        }
    }

8. Non-public Test Methods are Allowed

JUnit 5 test classes and test methods are not required to be public. We can now make them package protected.
JUnit internally uses reflection to find test classes and test methods. Reflection can discover them even if they have limited visibility, so there is no need for them to be public.

9. Repeat Tests

JUnit Jupiter provides the ability to repeat a test a specified number of times by annotating a method with @RepeatedTest and specifying the total number of repetitions desired. To know more about RepestedTest, please refer to this tutorial – How to Retry Test in JUnit5 – @RepeatedTest

Below is the example of @RepeatedTest in JUnit5.

    @RepeatedTest(3)
    void repeatedTestWithRepetitionInfo1(RepetitionInfo repetitionInfo) {
        assertEquals(3, repetitionInfo.getTotalRepetitions());
    }

The output of the above program is

10. Parameterized Tests

Test parameterization existed in JUnit 4 with built-in libraries like JUnit4Parameterized or third-party libraries like JUnitParams. In JUnit 5, parameterized tests are completely built-in and adopt some of the best features from JUnit4Parameterized and JUnitParams. To know more about the parameterized tests in JUnit5, please refer to this tutorial – How to parameterized Tests in JUnit5.

Below is an example of parameterized Test in JUnit5.

public class CSVParameterizedTest {

    @ParameterizedTest
    @CsvSource({
            "java,      4",
            "javascript,   7",
            "python,    6",
            "HTML,    4",
    })


    void test(String str, int length) {
        assertEquals(length, str.length());
    }
}

The output of the above program is

Congratulations. We have gone through the differences between JUnit4 and JUnit5. Happy Learning!!

How to run parameterized Selenium test using JUnit5

HOME

The previous tutorial has shown the various parameterized tests in JUnit5. This tutorial shows how to run a test multiple times with a different set of data. This helps to reduce the duplication of code. This is a very common scenario in any testing. Imagine, we want to test the requirement for a login page that uses a username and password to log in to the application. Username and password must satisfy some conditions like username can be only alphabets and no numeric and special characters. There could be multiple sets of data that can be used to test this requirement.

Pre-Requisite:

  1. Selenium – 3.141.59
  2. Maven
  3. Java 11
  4. JUnit Jupiter Engine – 5.8.2
  5. JUnit Jupiter API- 5.8.2

JUnit5 provides a lot of ways to parameterize a test – @ValueSource, @EnumSource, @MethodSource, @CsvSource, @CsvFileSource, and @ArgumentsSource.

Let us see an example where the test is not parameterized. In the below example, we want to verify the different error messages generated by passing incorrect values to username and password.

This is the base class – Login which contains the test method that uses a different set of test data.

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;
import org.openqa.selenium.support.PageFactory;

public class LoginPage {

    WebDriver driver ;

    @FindBy(name="txtUsername")
    WebElement username;

    @FindBy(name="txtPassword")
    WebElement password;

    @FindBy(id="btnLogin")
    WebElement loginButton;

    @FindBy(id="spanMessage")
    WebElement actualErrorMessage;


    public LoginPage(WebDriver driver) {

        this.driver = driver;

        // This initElements method will create all WebElements
        PageFactory.initElements(driver, this);
    }

    public void setUserName(String strUserName) {
        username.sendKeys(strUserName);
    }

    // Set password in password textbox
    public void setPassword(String strPassword) {
        password.sendKeys(strPassword);
    }

    // Click on login button
    public void clickLogin() {
        loginButton.click();
    }

    // Get the error message
    public String getErrorMessage() {
        return actualErrorMessage.getText();
    }

    public void login(String strUserName, String strPasword) {

        // Fill user name
        this.setUserName(strUserName);

        // Fill password
        this.setPassword(strPasword);

        // Click Login button
        this.clickLogin();
    }

}

The below example shows 4 tests using a common test with 4 different sets of data.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import static org.junit.jupiter.api.Assertions.assertEquals;

public class NonParameterizedLoginTest  {

    WebDriver driver;
    LoginPage login;

    @BeforeEach
    void setUp() {

        WebDriverManager.chromedriver().setup();
        ChromeOptions chromeOptions = new ChromeOptions();
        driver = new ChromeDriver(chromeOptions);
        driver.manage().window().maximize();
        driver.get("https://opensource-demo.orangehrmlive.com/");

    }

    @Test
    void invalidCredentials1() {

        login = new Login(driver);
        login.login("Admin","Admin");
        String actualErrorMessage = login.getErrorMessage();
        assertEquals("Invalid credentials", actualErrorMessage);

    }

    @Test
    void invalidCredentials2() {

        login = new Login(driver);
        login.login("","Admin");
        String actualErrorMessage = login.getErrorMessage();
        assertEquals("Username cannot be empty", actualErrorMessage);

    }

    @Test
    void invalidCredentials3() {

        login = new Login(driver);
        login.login("Admin","");
        String actualErrorMessage = login.getErrorMessage();
        assertEquals("Password cannot be empty", actualErrorMessage);

    }

    @Test
    void invalidCredentials4() {

        login = new Login(driver);
        login.login("","");
        String actualErrorMessage = login.getErrorMessage();
        assertEquals("Username cannot be empty", actualErrorMessage);

    }


    @AfterEach
    void tearDown() {
        if (driver != null) {
            driver.close();
        }
    }
    
}

We can see that the same method is called multiple times. This is a duplication of code. The output of the above program is

Now, we will parametrize the same test. To do so, we need to add a dependency to the POM.xml.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.example</groupId>
    <artifactId>JUnit5Demo</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>11</maven.compiler.source>
        <maven.compiler.target>11</maven.compiler.target>
    </properties>

    <dependencies>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>5.8.2</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.8.2</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>3.141.59</version>
        </dependency>

        <dependency>
            <groupId>io.github.bonigarcia</groupId>
            <artifactId>webdrivermanager</artifactId>
            <version>5.1.0</version>
        </dependency>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-params</artifactId>
            <version>5.8.2</version>
            <scope>test</scope>
        </dependency>

    </dependencies>

    <build>
        <plugins>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M5</version>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M5</version>

                <dependencies>
                    <dependency>
                        <groupId>org.junit.jupiter</groupId>
                        <artifactId>junit-jupiter-engine</artifactId>
                        <version>5.8.2</version>
                    </dependency>
                </dependencies>
            </plugin>
        </plugins>
    </build>

</project>

There are multiple ways to parameterize the test. To start with:

  1. Replace @Test annotation with @ParameterizedTest annotation provided by the JUnit5 framework.
  2. Add parameters to the loginTest() method. In this example, we will add a username and a password parameter.
  3. Add the parameters source. In this example, we will use the @CsvFileSource annotation.

In this example, will retrieve the data from CSV. This CSV file is placed under src/test/resources. Below is the example of the credentials.csv file.

To know all the different types of parameterization methods, please refer to this tutorial. This tutorial will show the 2 most common ways to parameterize tests in JUnit5.

  1. @CsvSource

@CsvSource allows us to express argument lists as comma-separated values (i.e., CSV String literals). Each string provided via the value attribute in @CsvSource represents a CSV record and results in one invocation of the parameterized test. An empty, quoted value (”) results in an empty String. This can be seen in the below example.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvFileSource;
import org.junit.jupiter.params.provider.CsvSource;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import static org.junit.jupiter.api.Assertions.assertEquals;

public class ParameterizedLoginPageTest {

    WebDriver driver;

    LoginPage loginPage;

    @BeforeEach
     void setUp() {

        WebDriverManager.chromedriver().setup();
        ChromeOptions chromeOptions = new ChromeOptions();
        driver = new ChromeDriver(chromeOptions);
        driver.manage().window().maximize();
        driver.get("https://opensource-demo.orangehrmlive.com/");

    }

    @ParameterizedTest
    @CsvSource({
            "admin123,admin123,Invalid credentials",
            "'',admin123,Username cannot be empty",
            "Admin,'',Password cannot be empty",
            "'','',Username cannot be empty"
    })

    void invalidCredentials1(String username, String password, String errorMessage) {

        loginPage = new LoginPage(driver);
        loginPage.login(username,password);
        String actualErrorMessage = loginPage.getErrorMessage();
        assertEquals(errorMessage, actualErrorMessage);

    }

    @AfterEach
     void tearDown() {
        driver.close();
    }
}

The output of the above program is

2. @CsvFileSource

@CsvFileSource lets us use comma-separated value (CSV) files from the classpath or the local file system.

We can see in the below example, that we have skipped the first line from the credentials.csv file as it is the heading of the file. invalidCredentials() method got 4 different set of the test data from CSV file using parameterization. JUnit5 ignores the headers via the numLinesToSkip attribute.

In @CsvFileSource, an empty, quoted value (“”) results in an empty String in JUnit5.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvFileSource;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import static org.junit.jupiter.api.Assertions.assertEquals;

public class ParameterizedLoginPageTest {

    WebDriver driver;

    LoginPage loginPage;

    @BeforeEach
     void setUp() {

        WebDriverManager.chromedriver().setup();
        ChromeOptions chromeOptions = new ChromeOptions();
        driver = new ChromeDriver(chromeOptions);
        driver.manage().window().maximize();
        driver.get("https://opensource-demo.orangehrmlive.com/");

    }

    @ParameterizedTest
    @CsvSource({
            "admin123,admin123,Invalid credentials",
            "'',admin123,Username cannot be empty",
            "Admin,'',Password cannot be empty",
            "'','',Username cannot be empty"
    })


    @ParameterizedTest
    @CsvFileSource(files = "src/test/resources/credentials.csv", numLinesToSkip = 1)
    void invalidCredentials(String username, String password, String errorMessage) {

        loginPage = new LoginPage(driver);
        loginPage.login(username,password);
        String actualErrorMessage = loginPage.getErrorMessage();
        assertEquals(errorMessage, actualErrorMessage);

    }


    @AfterEach
     void tearDown() {
        driver.close();
    }
}
     

The result of the above program is

Congratulations!! We have seen how Selenium tests are parameterized in JUnit5. Happy Learning.

How to parameterize Tests in JUnit5

HOME

JUnit5  enables us to execute a single test method multiple times with a different sets of data. This is called Parameterization. Parameterized Tests are declared just like regular @Test methods but use the @ParameterizedTest annotation.

This article shows you how to run a test multiple times with different arguments, so-called ‘Parameterized Tests’, let’s see the following ways to provide arguments to the test:

  • @ValueSource
  • @EnumSource
  • @MethodSource
  • @CsvSource
  • @CsvFileSource
  • @ArgumentsSource

We need to add junit-jupiter-params to support parameterized tests. In the case of Maven, add the dependency to POM.xml

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-params</artifactId>
    <version>5.8.2</version>
    <scope>test</scope>
</dependency>

In case of Gradle, add the dependency as

testCompile("org.junit.jupiter:junit-jupiter-params:5.8.2")

1. @ValueSource

Let us start with a simple example. The following example demonstrates a parameterized test that uses the @ValueSource annotation to specify an integer array as the source of arguments. The following @ParameterizedTest method will be invoked three times, with the values 5,6, and 0 respectively.

@ParameterizedTest
@ValueSource(ints = {5, 6, 0})
void test_int_arrays(int b) {

    int a= 5;
    int sum = a + b;
    assertTrue(sum>8);
 }

When executing the above-parameterized test method, each invocation will be reported separately.

The output of the above program is:

One of the limitations of value sources is that they only support these types:

  • short (with the shorts attribute)
  • byte (bytes attribute)
  • int (ints attribute)
  • long (longs attribute)
  • float (floats attribute)
  • double (doubles attribute)
  • char (chars attribute)
  • java.lang.String (strings attribute)
  • java.lang.Class (classes attribute)

Also, we can only pass one argument to the test method each time.

In the below example, an array of strings is passed as the argument to the Parameterized Test.

@ParameterizedTest(name = "#{index} - Run test with args={0}")
@ValueSource(strings = {"java", "python", "javascript","php"})
void test_string_arrays(String arg) {
        assertTrue(arg.length() > 1);
}

The output of the above program is:

@NullSource

It provides a single null an argument to the annotated @ParameterizedTest method.

    @ParameterizedTest()
    @NullSource
    void nullString(String text) {
        assertTrue(text == null);
    }
    

The output of the above program is:

@EmptySource

It provides a single empty argument to the annotated @ParameterizedTest method of the following types:

  • java.lang.String
  • java.util.List
  • java.util.Set
  • java.util.Map
  • primitive arrays (e.g. int[])
  • object arrays (e.g. String[])
 @ParameterizedTest
    @EmptySource
    void testMethodEmptySource(String argument) {
        assertTrue(StringUtils.isEmpty(argument));
        assertTrue(StringUtils.isBlank(argument));
    }

The output of the above program is:

@NullAndEmptySource

We can pass empty or null values into the test via @EmptySource, @NullSource, or @NullAndEmptySource (since JUnit 5.4).

Let’s see the following example to test an isEmpty() method.

import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.EmptySource;
import org.junit.jupiter.params.provider.NullSource;
import org.junit.jupiter.params.provider.ValueSource;
import static org.junit.jupiter.api.Assertions.*;

public class ParameterizedTestDemo {

    @ParameterizedTest(name = "#{index} - isEmpty()? {0}")
    @NullSource
    @EmptySource
    @ValueSource(strings = { " ", "   ", "\t", "\n","a"})
    void nullEmptyAndBlankStrings(String text) {
        assertTrue(text == null || text.trim().isEmpty());
    }
}

The parameterized test method result in seven invocations: 1 for null, 1 for the empty string, 4 for the explicit blank strings supplied via @ValueSource, and 1 non-blank string “a” supplied via @ValueSource.

The output of the above program is:

2. @EnumSource

@EnumSource provides a convenient way to use Enum constants.

public class EnumParameterizedTest {

    enum Error {
         Request_Invalid,
         Request_Timeout,
         RequestHeader_Invalid,
         Concurrency_Failed,
         ExternalCall_Failed,
         Schema_Invalid,
         Authentication_Failed;
    }

    @ParameterizedTest
    @EnumSource(Error.class)
    void test_enum(Error error) {
       assertNotNull(error);
    }
}

The output of the above program is:

The annotation provides an optional names attribute that lets you specify which constants shall be used, like in the following example. If omitted, all constants will be used.

    @ParameterizedTest(name = "#{index} - Is Error contains {0}?")
    @EnumSource(value = Error.class, names = {"Request_Invalid", "ExternalCall_Failed", "Concurrency_Failed", "Authentication_Failed"})
    void test_enum_include(Error error) {
       assertTrue(EnumSet.allOf(Error.class).contains(error));
    }

The output of the above program is:

The @EnumSource annotation also provides an optional mode attribute that enables fine-grained control over which constants are passed to the test method. For example, you can exclude names from the enum constant pool or specify regular expressions as in the following examples.

 @ParameterizedTest
 @EnumSource(value = Error.class, mode = EnumSource.Mode.EXCLUDE, names = {"Request_Invalid", "Request_Timeout", "RequestHeader_Invalid"})
    void test_enum_exclude(Error error) {
        EnumSet<Error> excludeRequestRelatedError = EnumSet.range(Error.Concurrency_Failed, Error.Authentication_Failed);
        assertTrue(excludeRequestRelatedError.contains(error));
  }

The output of the above program is:

EnumSource.Mode.EXCLUDE – It selects all declared enum constants except those supplied via the names attribute.

EnumSource.Mode.MATCH_ALL – It selects only those enum constants whose names match any pattern supplied via the names attribute.

  @ParameterizedTest
    @EnumSource(mode = EnumSource.Mode.MATCH_ALL, names = "^.*Invalid")
    void test_match(Error error) {
        assertTrue(error.name().contains("Invalid"));
    }

The output of the above program is

3. @MethodSource

@MethodSource allows you to refer to one or more factory methods of the test class or external classes.

Factory methods within the test class must be static unless the test class is annotated with @TestInstance(Lifecycle.PER_CLASS); whereas, factory methods in external classes must always be static. In addition, such factory methods must not accept any arguments.

If you only need a single parameter, you can return a Stream of instances of the parameter type as demonstrated in the following example.

   @ParameterizedTest(name = "#{index} - Test with String : {0}")
    @MethodSource("stringProvider")
    void test_method_string(String arg) {
        assertNotNull(arg);
    }

    // this need static
    static Stream<String> stringProvider() {
        return Stream.of("java", "junit5", null);
    }

The output of the above program is

If you do not explicitly provide a factory method name via @MethodSource, JUnit Jupiter will search for a factory method that has the same name as the current @ParameterizedTest method by convention. This is demonstrated in the following example.

    @ParameterizedTest(name = "#{index} - Test with String : {0}")
    @MethodSource
    void test_no_factory_methodname(String arg) {
        assertNotNull(arg);
    }

    static Stream<String> test_no_factory_methodname() {
        return Stream.of("java", "junit5", null);
    }

The output of the above program is

Streams for primitive types (DoubleStream, IntStream, and LongStream) are also supported as demonstrated by the following example.

 @ParameterizedTest(name = "#{index} - Test with Int : {0}")
    @MethodSource("rangeProvider")
    void test_method_int(int arg) {
        assertTrue(arg < 6);
    }
    
    static IntStream rangeProvider() {
        return IntStream.range(0, 6);
    }

The output of the above program is

If a parameterized test method declares multiple parameters, you need to return a collection, stream, or array of Arguments instances or object arrays as shown below.

    @ParameterizedTest
    @MethodSource("stringIntAndListProvider")
    void testWithMultiArgMethodSource(String str, int num, List<String> list) {
        assertEquals(5, str.length());
        assertTrue(num >=1 && num <=2);
        assertEquals(2, list.size());
    }

    static Stream<Arguments> stringIntAndListProvider() {
        return Stream.of(
                arguments("apple", 1, Arrays.asList("a", "b")),
                arguments("lemon", 2, Arrays.asList("x", "y"))
        );
    }

The output of the above program is

4. @CsvSource

@CsvSource allows you to express argument lists as comma-separated values (i.e., CSV String literals). Each string provided via the value attribute in @CsvSource represents a CSV record and results in one invocation of the parameterized test.

import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import static org.junit.jupiter.api.Assertions.assertEquals;

public class CSVParameterizedTest {

    @ParameterizedTest
    @CsvSource({
            "java,      4",
            "javascript,   7",
            "python,    6",
            "HTML,    4",
    })


    void test(String str, int length) {
        assertEquals(length, str.length());
    }
}

The output of the above program is

5. @CsvFileSource

@CsvFileSource lets us use comma-separated value (CSV) files from the classpath or the local file system. Each record from a CSV file results in one invocation of the parameterized test. The first record may optionally be used to supply CSV headers.

csvdemo.csv

    @ParameterizedTest
    @CsvFileSource(resources = "/csvdemo.csv")

    void testLength(String str, int length) {
        Assertions.assertEquals(length, str.length());
    }

The output of the above program is

csv file with the heading

JUnit can ignore the headers via the numLinesToSkip attribute.

    @ParameterizedTest
    @CsvFileSource(files = "src/test/resources/csvdemo.csv",numLinesToSkip = 1)

    void testStringLength(String str, int length) {
        Assertions.assertEquals(length, str.length());
    }

The output of the above program is

If you would like the headers to be used in the display names, you can set the useHeadersInDisplayName attribute to true. The examples below demonstrate the use of useHeadersInDisplayName.

 @ParameterizedTest(name = "[{index}] {arguments}")
    @CsvFileSource(files = "src/test/resources/csvdemo.csv",useHeadersInDisplayName = true)

    void testStringLength1(String str, int length) {
        assertEquals(length, str.length());
    }

The output of the above program is

6. @ArgumentsSource

@ArgumentsSource can be used to specify a custom, reusable ArgumentsProvider. Note that an implementation of ArgumentsProvider must be declared as either a top-level class or as a static nested class.

import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.ArgumentsProvider;
import java.util.stream.Stream;

public class CustomArgumentsProvider implements ArgumentsProvider {

    @Override
    public Stream<? extends Arguments>
    provideArguments(ExtensionContext extensionContext) throws Exception {
        return Stream.of("java", "junit5", "junit4", null).map(Arguments::of);
    }
}
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ArgumentsSource;
import static org.junit.jupiter.api.Assertions.assertNotNull;

public class ArgumentsSourceTest {

    @ParameterizedTest
    @ArgumentsSource(CustomArgumentsProvider.class)
    void test_argument_custom(String arg) {
        assertNotNull(arg);
    }
}

The output of the above program is

Congratulation. We have understood parameterization in JUnit5 tests. Happy Learning!!

How to tag and filter JUnit5 tests – @Tag

HOME

This tutorial explains to run the specific tests in JUnit5 using @Tag annotation. Imagine, there are 500 test cases for different functionalities. Out of 500 test cases, 350 tests are related to Integration test and rest 150 are for E2E test. We want to run only Integration tests. How this can be achieved? To overcome this problem, JUnit5 provides a filtering mechanism – @Tag annotation. We can apply the @Tag annotation on a test class or test method or both.

The JUnit Platform enforces the following rules for Tag:

  • A tag must not be null or blank.
  • A trimmed tag must not contain whitespace.
  • A trimmed tag must not contain ISO control characters.
  • A trimmed tag must not contain any of the following reserved characters.
    • ,: comma
    • (: left parenthesis
    • ): right parenthesis
    • &: ampersand
    • |: vertical bar
    • !: exclamation point

Annotating JUnit Test Class with Tag

Scenario 1 – Apply @Tag on the test class. It will run all the tests present within this test class.

import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.assertEquals;

@Tag("Sprint-5")
class JUnit5TagsTests {

    @Test
    void test_Addition() {
        System.out.println("test_Add()");
        assertEquals(18, 3 + 7 + 8);
    }

    @Test
    void test_Subtraction() {
        System.out.println("test_Subtraction()");
        assertEquals(18, 26 - 8);
    }

    @Test
    void test_Calculator() {
        System.out.println("test_Calculator()");
        assertEquals(18, 10 + 8);
        assertEquals(2, 10 - 8);
    }

    @Test
    void test_Functions() {
        System.out.println("test_Functions()");
        assertEquals(8, Math.sqrt(64));
        assertEquals(64, Math.pow(8,2));
    }

    @Test
    void test_IsEven() {
        System.out.println("test_IsEven()");
        assertEquals(0, 16%2);
    }

}

Let us say we a number of classes and we want to execute only this specific test class that is tagged as – @Sprint-5.

Go to the command line or in case of IntelliJ to the terminal.

mvn clean test -Dgroups="Sprint-5"

The result of the above program is

Annotating JUnit Test Methods with Tag

With JUnit 5 we can filter tests by tagging a subset of them under a unique tag name.

Scenario 2 – Let’s say we have 5 tests and we want to run 3 tests in the development environment, 1 test in both development and QA, 1 test in Production and 1 test In-Progress. So we will tag the tests as below:

import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;

class JUnit5TagsTests {

    @Test
    @Tag("Development")
    void test1() {
        System.out.println("This test is for Development");
    }

    @Test
    @Tag("Development")
    void test2() {
        System.out.println("This test is for Development");
    }

    @Test
    @Tag("Development")
    @Tag("QA")
    void test3() {
        System.out.println("This test is for Development & QA");
    }

    @Test
    @Tag("Production")
    void test4() {
        System.out.println("This test is for Production");
    }

    @Test
    @Tag("Regression")
    @Tag("QA")
    void test5() {
        System.out.println("This is Regression Test for QA");
    }
}

To run the tests tagged with “production” in IntelliJ. Edit the configuration. Click on the Run and select “Edit Configurations”.

Select Tags from a list of components and mention the name of tag you want to execute. Apply the changes by clicking on the “Apply” button and then Click on the “OK” button.

Now, this create a new Configuration as shown in the below image.

Click on this configuration. It will only run the test method tagged with @Production.

2. We can apply multiple tags on a single test case as well. Here, test method – test_Calculator() is tagged with @Development and @QA.

    @Test
    @Tag("Development")
    @Tag("QA")
    void test_Calculator() {
        System.out.println("test_Calculator()");
        assertEquals(18, 10 + 8);
        assertEquals(2, 10 - 8);
    }

To run above test, edit the configuration as shown below.

The output of the above program is

3. Filtering Tags with Maven Surefire Plugin

In Maven, we can run tests based on tags via the configuration parameters of the maven-surefire-plugin.

<plugin>
       <groupId>org.apache.maven.plugins</groupId>
       <artifactId>maven-surefire-plugin</artifactId>
       <version>3.0.0-M5</version>
       <configuration>
           <!-- Include tags -->
           <groups>Development,QA,Production</groups>
           <!-- Exclude tags -->
           <excludedGroups>In-Progress</excludedGroups>
       </configuration>
</plugin>

If we now execute this plugin, it will execute all tests which are tagged as Development,QA,Production. 

If we want to exclude any specific test from the test execution, mention it with <excludeGroups>

Below mentioned command will run all the tests except test tagged with “In-Progress”.

mvn clean test -DexcludeGroups="In-Progress"

4. Creating your own custom tag annotation

If we are using same tag @Tag(“Security”) or combination with @Tag(“QA”) in several tests, instead of copying and pasting @Tag(“Security”), @Tag(“QA”) throughout your code base, you can create a custom composed annotation named @SecurityQATest as follows. @SecurityQATest can then be used instead using 2 annotations every time.

Following example shows you how to create custom tag annotation for @Tag(“Security”), @Tag(“QA”).

import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestTemplate;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.ANNOTATION_TYPE,ElementType.METHOD})
@Tag("Security")
@Tag("QA")
@Test
public @interface SecurityQATest {

}

    @SecurityQATest
    void test6() {
        System.out.println("This is Security Testing for QA");
    }

To run this test, use the below command:

mvn clean test -Dgroups="Security&QA"

The result of the above program is

Congratulations. We have understood the usage of @Tag annotation. Happy Learning!!

How to run JUnit5 tests in order

HOME

The general practices say that automated tests should be able to run independently and with no specific order, as well as the result of the test should not depend on the results of previous tests. But there are situations where a specific order of test execution can be justified, especially in integration or end-to-end tests. The test methods don’t follow a specific order by default to execute the tests. The test cases need not necessarily execute in the order in which they have been written.

There are different ways or modes to set the order of execution for the test cases.  This article shows how to control the JUnit 5 test execution order via the following MethodOrderer classes:

  • DisplayName – sorts test methods alphanumerically based on their display names
  • MethodName – sorts test methods alphanumerically based on their names and formal parameter lists
  • Alphanumeric – sorts test methods alphanumerically based on their names and formal parameter lists. This is deprecated from JUnit Version 5.7 onwards
  • OrderAnnotation – sorts test methods numerically based on values specified via the @Order annotation
  • Random – orders test methods pseudo-randomly and support the configuration of a custom seed
  • Custom Order – A custom ordering sequence can be implemented by the interface MethodOrderer and providing it as the argument to @TestMethodOrder.

Let us create JUnit5 Tests and execute them.

public class OrderRandomDemo {

    @Test
    void test_Add() {

        System.out.println("test_Add()");
        assertEquals(10, 3 + 7);
    }

    @Test
    void test_Subtract() {

        System.out.println("test_Subtract()");
        assertEquals(10, 14 - 4);
    }

    @Test
    void test_Multiply() {

        System.out.println("test_Multiply()");
        assertEquals(10, 5 * 2);
    }

    @Test
    void test_Divide() {

        System.out.println("test_Divide()");
        assertEquals(10, 30 / 3);
    }

    @Test
    void test_IsEven() {

        System.out.println("test_IsEven()");
        assertEquals(0, 10%2);
    }

}

The output of the above program

1. DisplayName

It sorts test methods alphanumerically based on their display names. Test Method can be anything annotated with @Test, @RepeatedTest, @ParameterizedTest, @TestFactory, or @TestTemplate.

@TestMethodOrder(MethodOrderer.DisplayName.class)

@TestMethodOrder is a type-level annotation that is used to configure a MethodOrderer for the test methods of the annotated test class or test interface.

MethodOrderer defines the API for ordering the test methods in a given test class.

Test Method – It is any method annotated with @Test, @RepeatedTest, @ParameterizedTest, @TestFactory, or @TestTemplate.

DisplayName.class – MethodOrderer that sorts methods alphanumerically based on their names using String.compareTo(String).
If two methods have the same name, String representations of their formal parameter lists will be used as a fallback for comparing the methods.

An example of sorting the tests based on their display names.

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import static org.junit.jupiter.api.Assertions.assertEquals;

@TestMethodOrder(MethodOrderer.DisplayName.class)
public class DisplayNameOrderedTests {

    @DisplayName("C")
    @Test
    void test_Add() {

        System.out.println("test_Add()");
        assertEquals(10, 3 + 7);
    }


    @DisplayName("E")
    @Test
    void test_Multiply() {

        System.out.println("test_Multiply()");
        assertEquals(10, 5 * 2);
    }

    @DisplayName("A")
    @Test
    void test_Divide() {

        System.out.println("test_Divide()");
        assertEquals(10, 30 / 3);
    }


    @DisplayName("D")
    @Test
    void test_Subtract() {

        System.out.println("test_Subtract()");
        assertEquals(10, 18 - 8);
    }

    @DisplayName("B")
    @Test
    void test_IsEven() {

        System.out.println("test_IsEven()");
        assertEquals(0, 18%2);
    }
}

We can see that the test methods are sorted alphanumerically based on their display name starting from A to E. The output of the above program

2. MethodName

This annotation sorts methods alphanumerically based on their names using String.compareTo(String).
If two methods have the same name, String representations of their formal parameter lists will be used as a fallback for comparing the methods.

@TestMethodOrder(MethodOrderer.MethodName.class)

Let us see an example of MethodName.

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import static org.junit.jupiter.api.Assertions.assertEquals;

@TestMethodOrder(MethodOrderer.MethodName.class)
class MethodNameOrderedTests {

    @Test
    void testE() {

        System.out.println("testE");
        assertEquals(10, 3 + 7);
    }

    @Test
    void testA() {

        System.out.println("testA");
        assertEquals(10, 14 - 4);
    }

    @Test
    void testC() {
        System.out.println("testC");
        assertEquals(10, 5 * 2);
    }

    @Test
    void testB() {
        System.out.println("testB");
        assertEquals(10, 30 / 3);
    }

    @Test
    void testD() {
        System.out.println("testD");
        assertEquals(10, 10 + 0);
    }

}

The output of the above program

3. OrderAnnotation

This sorts test method numerically based on values specified via the @Order annotation.
Any methods that are assigned the same order value will be sorted arbitrarily adjacent to each other.
When any method is not annotated with @Order, it will be assigned the default order value, which will effectively cause them to appear at the end of the sorted list.

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)

Let us see an example of OrderAnnotation.

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import static org.junit.jupiter.api.Assertions.assertEquals;

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class OrderAnnotationDemo {

    @Test
    @Order(3)
    void test_Add() {

        System.out.println("test_Add()");
        assertEquals(10, 3 + 7);
    }

    @Test
    @Order(4)
    void test_IsOdd() {
        System.out.println("test_IsOdd()");
        assertEquals(1, 11%2);
    }

    @Test
    void test_Subtract() {
        System.out.println("test_Subtract()");
        assertEquals(10, 14 - 4);
    }

    @Test
    @Order(1)
    void test_Multiply() {
        System.out.println("test_Multiply()");
        assertEquals(10, 5 * 2);
    }

    @Test
    @Order(4)
    void test_Divide() {
        System.out.println("test_Divide()");
        assertEquals(10, 30 / 3);
    }

    @Test
    @Order(2)
    void test_IsEven() {
        System.out.println("test_IsEven()");
        assertEquals(0, 10%2);
    }
}

Here, test_Subtract() is not assigned any order value, so it is displayed as the last one in the last.

4. Random

These sorts of test methods are pseudo-randomnly.

@TestMethodOrder(MethodOrderer.Random.class)

Let us create a program to show the random order of tests in JUnit5.

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import static org.junit.jupiter.api.Assertions.assertEquals;

@TestMethodOrder(MethodOrderer.Random.class)
public class OrderRandomDemo {

    @Test
    void test_Add() {

        System.out.println("test_Add()");
        assertEquals(10, 3 + 7);
    }

    @Test
    void test_Subtract() {

        System.out.println("test_Subtract()");
        assertEquals(10, 14 - 4);
    }

    @Test
    void test_Multiply() {

        System.out.println("test_Multiply()");
        assertEquals(10, 5 * 2);
    }

    @Test
    void test_Divide() {

        System.out.println("test_Divide()");
        assertEquals(10, 30 / 3);
    }

    @Test
    void test_IsEven() {

        System.out.println("test_IsEven()");
        assertEquals(0, 10%2);
    }
}

The output of the above program

5. Custom Order

We can define our own custom ordering sequence by implementing the interface MethodOrderer and providing it as the argument to @TestMethodOrder.

Here, the tests are arranged in descending method order.

import org.junit.jupiter.api.MethodDescriptor;
import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.MethodOrdererContext;

public class DescendingMethodOrder implements MethodOrderer {

    @Override
    public void orderMethods(MethodOrdererContext context) {
        context.getMethodDescriptors().sort((MethodDescriptor m1, MethodDescriptor m2) ->
                m2.getMethod().getName().compareTo(m1.getMethod().getName()));
    }
    
}

Now, test the above custom order.

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import static org.junit.jupiter.api.Assertions.assertEquals;

@TestMethodOrder(DescendingMethodOrder.class)
public class CustomOrderTests {

    @Test
    void test_Add() {
        System.out.println("test_Add()");
        assertEquals(10 , 4 + 6);
    }

    @Test
    void test_Subtract() {
        System.out.println("test_Subtract()");
        assertEquals(10 , 17 - 7);
    }

    @Test
    void test_Multiply() {
        System.out.println("test_Multiply()");
        assertEquals(10 , 5 * 2);
    }

    @Test
    void test_Divide() {
        System.out.println("test_Divide()");
        assertEquals(10 , 20/2);
    }

    @Test
    void test_IsEven() {
        System.out.println("test_IsEven()");
        assertEquals(0 , 20%2);
    }
}

Notice the test output. The tests are executed in descending order. The result of the above program is

Test Classes Ordering

  1. ClassName: sorts test classes alphanumerically based on their fully qualified class names.
  2. DisplayName: sorts test classes alphanumerically based on their display names (see display name generation precedence rules).
  3. OrderAnnotation: sorts test classes numerically based on values specified via the @Order annotation.
  4. Random: orders test classes pseudo-randomly and support the configuration of a custom seed.

The configured ClassOrderer will be applied to all top-level test classes (including static nested test classes) and @Nested test classes.

1. ClassName

The Test Classes are sorted alphanumerically based on their fully qualified class names.

package JUnit5;

import org.junit.jupiter.api.ClassOrderer;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestClassOrder;

import static org.junit.jupiter.api.Assertions.assertEquals;

@TestClassOrder(ClassOrderer.ClassName.class)
public class ClassNameOrderTests {

    @Nested
    class Addition {

        @Test
        void test_Add() {
            System.out.println("test_Add()");
            assertEquals(10, 3 + 7);
        }
    }

    @Nested
    class IsEven {

        @Test
        void test_IsEven() {
            System.out.println("test_IsEven()");
            assertEquals(0, 10 % 2);
        }
    }

    @Nested
    class Subtraction {

        @Test
        void test_Subtract() {
            System.out.println("test_Subtract()");
            assertEquals(9, 14 - 5);
        }
    }

    @Nested
    class Multiply {

        @Test
        void test_Multiply() {
            System.out.println("test_Multiply()");
            assertEquals(10, 5 * 2);
        }
    }
}

The result of the above program is

2. DisplayName

It sorts test classes alphanumerically based on their display names.

package JUnit5;

import org.junit.jupiter.api.*;

import static org.junit.jupiter.api.Assertions.assertEquals;

@TestClassOrder(ClassOrderer.DisplayName.class)
public class ClassDisplayNameTests {

    @Nested
    @DisplayName("B")
    class AppFlowTests {

        @Test
        void test_Add() {
            System.out.println("test_Add()");
            assertEquals(10, 6 + 4);
        }
    }

    @Nested
    @DisplayName("C")
    class TearDownTests {

        @Test
        void test_Subtract() {
            System.out.println("test_Subtract()");
            assertEquals(10, 15 - 5);

        }
    }

    @Nested
    @DisplayName("A")
    class SetupTests {

        @Test
        void test_Multiply() {
            System.out.println("test_Multiply()");
            assertEquals(10, 5 * 2);
        }
    }
}

The result of the above program is

3. OrderAnnotation in Class

The test classes are sorted numerically based on values specified via the @Order annotation.

import org.junit.jupiter.api.*;

import static org.junit.jupiter.api.Assertions.assertEquals;

@TestClassOrder(ClassOrderer.ClassName.class) //sorts test classes alphanumerically based on their fully qualified class names.
public class ClassOrderedTests {

    @Nested
    @Order(2)
    class AppFlowTests {

        @Test
        void test_Add() {
            System.out.println("test_Add()");
            assertEquals(10, 3 + 7);
        }
    }

    @Nested
    @Order(3)
    class TearDownTests {

        @Test
        void test_Subtract() {
            System.out.println("test_Subtract()");
            assertEquals(9, 14 - 5);
        }
    }

    @Nested
    @Order(1)
    class SetupTests {

        @Test
        void test_Multiply() {
            System.out.println("test_Multiply()");
            assertEquals(10, 5 * 2);
        }
    }
}

The result of the above program is

4. Random

The test classes are sorted pseudo-randomly and support the configuration of a custom seed.

import org.junit.jupiter.api.*;
import static org.junit.jupiter.api.Assertions.assertEquals;

@TestClassOrder(ClassOrderer.Random.class)
public class ClassRandomTests {

    @Nested
    class Addition {

        @Test
        void test_Add() {
            System.out.println("test_Add()");
            assertEquals(10, 3 + 7);
        }
    }

    @Nested
    class IsEven {

        @Test
        void test_IsEven() {
            System.out.println("test_IsEven()");
            assertEquals(0, 10 % 2);
        }
    }

    @Nested
    class Subtraction {

        @Test
        void test_Subtract() {
            System.out.println("test_Subtract()");
            assertEquals(9, 14 - 5);
        }
    }

    @Nested
    class Multiply {

        @Test
        void test_Multiply() {
            System.out.println("test_Multiply()");
            assertEquals(10, 5 * 2);
        }
    }
}

The result of the above program is

Congratulation!! We have gone through different types of ordering in JUnit5. Happy Learning!!

How to Retry Test in JUnit5 – @RepeatedTest

HOME

In JUnit5, JUnit Jupiter provides the ability to repeat a test a specified number of times by annotating a method with @RepeatedTest. We can specify the repetition frequency as a parameter to the @RepeatedTest annotation.

When do we use the Repeated Test?

Imagine, we are testing an online shopping website. In the process of placing an order, we need to pay for the product. But the payment gateway is a third-party service, and we cannot control the connectivity between our website and the payment gateway. We know that clicking on the ‘Payment’ link sometime shows “Exception: Page cannot be displayed”. This is an intermittent issue. So, we don’t want to fail the test if this payment link does not work. We can configure to click this payment link multiple times, before marking the test case failed. Here comes the Repeated Test in the picture.

A few points to keep in mind for @RepeatedTest are as follows:

  1. The methods annotated with @RepeatedTest cannot be static, otherwise, the test cannot be found.
  2. The methods annotated with @RepeatedTest cannot be private, otherwise, the test cannot be found.
  3. The return type of the method annotated with @RepeatedTest must be void only, otherwise, the test cannot be found.

The below example will run the test 5 times.

 @DisplayName("Addition")
 @RepeatedTest(3)
 void repeatTest(){
    int a = 4;
    int b = 6;
    assertEquals(10, a+b,"Incorrect sum of numbers");
 }

The output of the above test:

Each invocation of a repeated test behaves like the execution of a regular @Test method with full support for the same lifecycle callbacks and extensions.

It means that @BeforeEach and @AfterEach annotated lifecycle methods will be invoked for each invocation of the test.

@BeforeEach  annotation is used to signal that the annotated method should be executed before each invocation of the @Test@RepeatedTest@ParameterizedTest, or @TestFactory method in the current class. This is the replacement of the @Before Method in JUnit4.

TestInfo is used to inject information about the current test or container into @Test, @RepeatedTest, @ParameterizedTest, @TestFactory, @BeforeEach, @AfterEach, @BeforeAll, and @AfterAll methods.
If a method parameter is of type TestInfo, JUnit will supply an instance of TestInfo corresponding to the current test or container as the value for the parameter.

RepetitionInfo is used to inject information about the current repetition of a repeated test into the @RepeatedTest, @BeforeEach, and @AfterEach methods.
If a method parameter is of type RepetitionInfo, JUnit will supply an instance of RepetitionInfo corresponding to the current repeated test as the value for the parameter.

In the below example, @BeforeEach will get executed before each repetition of each repeated test. By having the TestInfo and RepetitionInfo injected into the method, we see that it’s possible to obtain information about the currently executing repeated test.

import org.junit.jupiter.api.*;
import static org.junit.jupiter.api.Assertions.*;

public class RepeatCycleDemo {

    @BeforeEach
    void init(TestInfo testInfo, RepetitionInfo repetitionInfo) {
      System.out.println("Before Each init() method called");
      int currentRepetition = repetitionInfo.getCurrentRepetition();
      int totalRepetitions = repetitionInfo.getTotalRepetitions();
      String methodName = testInfo.getTestMethod().get().getName();
      System.out.println(String.format("About to execute repetition %d of %d for %s", currentRepetition, totalRepetitions, methodName));
     }

    @RepeatedTest(3)
    void repeatedTestWithRepetitionInfo(RepetitionInfo repetitionInfo) {
      int a = 4;
      int b = 6;
      assertEquals(10, a+b, repetitionInfo.getTotalRepetitions());
    }

    @AfterEach
    public void cleanUpEach(){
      System.out.println("=================After Each cleanUpEach() method called =================");
    }
}

The output of the above test:

Custom Display Name

In addition to specifying the number of repetitions, a custom display name can be configured for each repetition via the name attribute of the @RepeatedTest annotation.

The display name can be a pattern composed of a combination of static text and dynamic placeholders. The following placeholders are currently supported.

  1. {displayName}: display name of the @RepeatedTest method
  2. {currentRepetition}: the current repetition count
  3. {totalRepetitions}: the total number of repetitions
    @BeforeEach
    void init(TestInfo testInfo, RepetitionInfo repetitionInfo) {
        System.out.println("Before Each init() method called");
        int currentRepetition = repetitionInfo.getCurrentRepetition();
        int totalRepetitions = repetitionInfo.getTotalRepetitions();
        String methodName = testInfo.getTestMethod().get().getName();
        System.out.println(String.format("About to execute repetition %d of %d for %s", //
                currentRepetition, totalRepetitions, methodName));
    }

   //Custom Display  
    @RepeatedTest(value = 2, name = "{displayName} {currentRepetition}/{totalRepetitions}")
    @DisplayName("Repeat JUnit5 Test")
    void customDisplayName(TestInfo testInfo) {
        assertEquals("Repeat JUnit5 Test 1/2", testInfo.getDisplayName());
    }

The output of the above test:

The default display name for a given repetition is generated based on the following pattern: “repetition {currentRepetition} of {totalRepetitions}”. Thus, the display names for individual repetitions of the previous repeatedTest() example would be repetition 1 of 10, repetition 2 of 10.

We can use one of two predefined formats for displaying the name – LONG_DISPLAY_NAME and SHORT_DISPLAY_NAME. The latter is the default format if none is specified.

  1. RepeatedTest.LONG_DISPLAY_NAME – {displayName} :: repetition {currentRepetition} of {totalRepetitions}
  2. RepeatedTest.SHORT_DISPLAY_NAME – repetition {currentRepetition} of {totalRepetitions}

Below is an example of SHORT_DISPLAY_NAME

    @RepeatedTest(value = 3, name = RepeatedTest.SHORT_DISPLAY_NAME)
    @DisplayName("Multiplication")
    void customDisplayNameWithShortPattern() {
        assertEquals(8, 6*5);
 }

In this case, the name is displayed as Multiplication repetition 1 of 3, repetition 2 of 3, and soon.

The output of the above test:

Below is an example of LONG_DISPLAY_NAME

  @RepeatedTest(value = 3, name = RepeatedTest.LONG_DISPLAY_NAME)
    @DisplayName("Addition")
    void customDisplayNameWithLongPattern(TestInfo testInfo) {
        System.out.println("Display Name :"+ testInfo.getDisplayName());
        System.out.println("Test Class Name :"+ testInfo.getTestClass());
        System.out.println("Test Method :"+ testInfo.getTestMethod());
        assertEquals(8, 3+7);
    }

The output of the above test:

As we can see in the example, we have used @DisplayName(“Addition”), but as it is name = RepeatedTest.LONG_DISPLAY_NAME, so the name of the tests are now Addition :: repetition 1 of 3, Addition :: repetition 2 of 3 and soon.

Congratulations. We are able to execute the tests multiple times by using the @RepeatedTest annotation. Happy Learning!!

How to run JUnit5 tests through Command Line

HOME

The previous tutorial explains to configure Junit in IntelliJ and run the tests as JUnit Tests. This tutorial shows the steps to run the tests through command line. We can ask, why we need to run the tests through command line?? There are many reasons, one of the reason is to achieve CI/CD. To run the tests in pipeline, they need to be run through command line. Another reason is that we don’t need to open the IDE to run the tests. Third reason is that many reports are only generated (Serenity, Cucumber), if the tests run through command line.

Below is a JUnit5 test.

import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.jupiter.api.*;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;

public class Demo {

    WebDriver driver;

    @BeforeEach
    public void setUp() {
        WebDriverManager.chromedriver().setup();
        ChromeOptions chromeOptions = new ChromeOptions();
        driver = new ChromeDriver(chromeOptions);
        driver.manage().window().fullscreen();
    }

    @Test
    public void Junit5Test() {
        driver.get("http://automationpractice.com/index.php");
        System.out.println("Title of Page :" + driver.getTitle());
        System.out.println("Page URL : " + driver.getCurrentUrl());
        Assertions.assertEquals("My Store",driver.getTitle());

    }

    @AfterEach
    public void tearDown() {
        driver.close();
    }
}

Let us see what happens when we try to run the JUnit tests through Command Line. This command is used to run the tests present in Demo class.

mvn clean test -Dtest=Demo

The output generated by the test is shown below

This shows that surefire-plugin is need to be add to the project to run t he tests successfully through command line.

Add surefire-plugin to the project

Go back to the Apache Maven Project and copy the code.

 <dependencies>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>5.8.2</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.8.2</version>
            <scope>test</scope>
        </dependency>
 
</dependencies>       
    
<build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M5</version>
                <dependencies>
                    <dependency>
                        <groupId>org.junit.jupiter</groupId>
                        <artifactId>junit-jupiter-engine</artifactId>
                        <version>5.8.2</version>
                    </dependency>
                </dependencies>
            </plugin>
        </plugins>
    </build>

Again run the tests through command line using the command mentioned above. Now, will see that the tests are executed successfully.

Congratulations. We can see that JUnit5 tests are executed through command line. Happy Learning!!

JUnit5 Assertions Example

HOME

JUnit5 contains the assertions available as in JUnit 4 Assertions as well there are a few additional new asserts too. In this post, let’s discuss each new assertion in JUnit5 works in detail with examples.

1. assertIterableEquals

The assertIterableEquals() asserts that the expected and the actual iterables are deeply equal. In order to be equal, both iterable must return equal elements in the same order and it isn’t required that the two iterables are of the same type in order to be equal.

Example 1 – In this example, the number of elements as well as the sequence of elements is in the same order in both Iterables. It is not mandatory to have Iterables of the same type, so we can see as one of the Iterable is ArrayList whereas another Iteratble is of type LinkedList.

@Test
void iterableEqualsPositive() {
     Iterable<String> iterat1 = new ArrayList<>(asList("Java", "Junit", "Test"));
     Iterable<String> iterat2 = new LinkedList<>(asList("Java", "Junit", "Test"));

     assertIterableEquals(iterat1, iterat2);
  }

The assertion passes as the sequence and number of elements in both Iterables are the same.

Example 2 – In the below example, the ordering of elements in the Iterables is different.

@Test
void iterableEqualsNegative() {
     Iterable<String> iterat1 = new ArrayList<>(asList("Java", "Junit", "Test"));
     Iterable<String> iterat2 = new ArrayList<>(asList("Java","Test","Junit" ));

     assertIterableEquals(iterat1, iterat2);
 }

Here, we can see that the sequence of elements in Iterable are different, so the assertion fails.

Example 3 – In the below example, the number of elements is different in the Iterables. Iterable 1 has 3 elements whereas Iterable 2 has 4 elements.

@Test
void iterableEqualsNegative1() {
     Iterable<String> iterat1 = new ArrayList<>(asList("Java", "Junit", "Test"));
     Iterable<String> iterat2 = new LinkedList<>(asList("Java", "Junit", "Test","Junit5"));

     assertIterableEquals(iterat1, iterat2);
 }

As both Iterables do not have same number of elements, so the Assertion has failed.

Note:- There are no assertions like assertNotIterableEquals() or assertIterableNotEquals().

2. assertLinesMatch

This Assertion asserts that the expected list of Strings matches the actual list of String.
This method differs from other assertions that effectively only check String.equals(Object), in that it uses the following staged matching algorithm:
For each pair of expected and actual lines do
a) check if expected.equals(actual) – if yes, continue with next pair
b) otherwise treat expected as a regular expression and check via String.matches(String) – if yes, continue with the next pair
c) otherwise check if an expected line is a fast-forward marker, if yes apply to fast-forward actual lines accordingly (see below) and goto 1.

Example 1 – In the below example, expected has a regular expression that matches with the elements of actual.

@Test
void linesMatchPositive() {
    List<String> expected = asList("Java", "\\d+", ".*");
    List<String> actual = asList("Java", "11", "JUnit");

    assertLinesMatch(expected, actual);
}

As the regular expression of elements matches, the assertion passes.

Example 2 – In the below example, the elements in the lists are different.

@Test
void linesMatchNegative1() {
   List<String> expected = asList("Java", "\\d+", ".*");
   List<String> actual = asList("Test","Java", "11");

   assertLinesMatch(expected, actual);
}

The assertion fails as the elements are different in actual and expected.

Example 3 – In the below example, the number of elements is different in expected and actual lists.

@Test
void linesMatchNegative2() {
    List<String> expected = asList("Java", "\\d+", ".*");
    List<String> actual = asList("Java", "11");

    assertLinesMatch(expected, actual);
}

The assertion fails as the number of elements in expected is 3 whereas 2 elements are available in actual list.

3. assertThrows

The new assertThrows() assertion allows us a clear and a simple way to assert if an executable throws the specified exception type.

Example 1 – In the below example, the length of string arr is null, so it throws a NullPointerException

@Test
void exceptionTestingPositive() {

    String arr = null;
    Exception exception = assertThrows(NullPointerException .class, () -> arr.length() );
    assertEquals(null, exception.getMessage());
 }

Result

Example 2 – In the below example, the exception thrown by String arr is NullPointerException. But, we are asserting it with ArithmeticException.

@Test
 void exceptionTestingNegative() {

     String arr = null;
     Exception exception = assertThrows(ArithmeticException .class, () -> arr.length() );
     assertEquals("Arithmetic Exception", exception.getMessage());
 }

Result

4. assertTimeout

When we want to assert that execution of the supplied executable completes before the given timeout, we can use assertTimeout().

Example 1 – In the below example, assertTimeout() is 2 sec, which means the assertion should be completed within 2 secs. We are waiting for 1 sec and then perform the assertion.

 @Test
     void assertTimeoutPositive() {
       int a = 4;
       int b= 5;
        assertTimeout(
                ofSeconds(2),
                () -> {
                    // code that requires less then 2 seconds to execute
                    Thread.sleep(1000);
                }
        );
        assertEquals(9, (a + b));
    }

As the assertion is within the specified time of assertTimeout(), the timeout assertion passes and the test passes.

Example 2 – In the below example, assertTimeout() is 2 sec whereas are waiting for 5 sec and then performing the assertion.

  @Test
   void assertTimeoutNegative() {
        int a = 4;
        int b= 5;
        assertTimeout(
                ofSeconds(2),
                () -> {
                    // code that requires less then 2 seconds to execute
                    Thread.sleep(5000);
                }
        );
        assertEquals(9, (a + b));
    }

As the assertion is outside the specified time of assertTimeout(), so the test fails. The assertion fails with an error message similar to: “execution exceeded timeout of 2000 ms by 3010 ms”.

Example 3 – In the below example, the assertion is mentioned just after

The executable will be executed in the same thread as that of the calling code. Consequently, execution of the executable will not be preemptively aborted if the timeout is exceeded.

@Test
 void assertTimeoutNegative1() {
    int a = 4;
    int b= 5;
    assertTimeout(
        ofSeconds(2),
        () -> {
                // code that requires less then 2 seconds to execute
                 Thread.sleep(5000);
                 assertEquals(10, (a + b));
              }
        );
    }

This shows that the assertion assertEquals() is still executed after the timeout also.

5. assertTimeoutPreemptively()

This assertion works just like assertTimeout(). When we want to assert that the execution of the supplied executable completes before the given timeout, we can use assertTimeoutPreemptively(). The only difference is that here the executable will be executed in a different thread than that of the calling code, whereas in assertTimeout() the executable will be executed in the same thread as that of the calling code. Furthermore, execution of the executable will be preemptively aborted if the timeout is exceeded here as contrary to assertTimeout() where the executable will not be preemptively aborted.

@Test
    void assertPreemptiveTimeoutNegative() {
        int a = 4;
        int b= 5;
        assertTimeoutPreemptively(
                ofSeconds(2),
                () -> {
                    // code that requires less then 2 seconds to execute
                    Thread.sleep(5000);
                    assertEquals(9, (a + b));
                }
        );
    }

Result

There is another very famous assertion called assertAll() [Group Assertions]. This is discussed in another tutorial.

In this post, We saw that JUnit Jupiter comes with many of the assertion methods that JUnit 4 has and adds a few that lend themselves well to being used with Java 8 lambdas. All JUnit Jupiter assertions are static methods in the org.junit.jupiter.api.Assertions class.

How to configure Junit in Intellij

HOME

In this tutorial we will discuss to create a JUnit  project using IntelliJ. We will be at first creating a simple Java Project and will add JUnit5 as well as create a Maven Project, then we will add a basic Class and a JUnit Test for it.

Java Project

Step 1 – Create a new Java Project.

To create a new Java project in Intellij, please refer to this tutorial.

Step 2 – Right click on the project and select Open Module Settings.

Step 3 – Go to the “Libraries” group, click the little plus (look up), and choose “From Maven…” option.

Step 4 – Search for “junit” — something like “junit:junit-4.13“. Click the “OK” button.

Step 5 – A new dialog will appear to confirm that “junit:junit:4.13.2” will be added to the module. Click the “OK” button.

Step 6 – This screens shows that junit:junit:4.13.2 is added to the Libraries. It contains the highlighted classes – junit-4.13.2.jar and hamcrest-core-1.3.jar. Click the “OK” button.

Step 7 – This image shows that the Junit is added to the External Libraries.

Step 8 – Create a Java Class – JUnit4Test under src and create a JUnit test to verify that it is installed properly.

import org.junit.Assert;
import org.junit.Test;

public class JUnit4Test {

    @Test
    public void Test() {

        String str1 = "Happy";
        String str2 = new String("Happy");
        Assert.assertEquals("String1 and String 2 are equal",str1, str2);

    }
}

Step 9 – There are many ways to run the test. One of the way is to Right-Click and select Run JUnit4Test

The successful execution of the test shows that the JUnit is configured properly.

Maven Project

Add Junit dependency to the POM.xml and build the project.

<dependencies>

    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.13.2</version>
        <scope>test</scope>
    </dependency>
    
</dependencies>

Now we need to apply the changes in the build script. Press Ctrl+Shift+O or click Load Maven Changes in the notification that appears in the top-right corner of the editor.

Create a Java Class – JUnit4Test under src/test/java and create a JUnit test to verify that it is installed properly.

import org.junit.Test;

import static org.junit.Assert.assertArrayEquals;

public class JUnitMavenTest {

    @Test
    public void Test() {

        String[] expected = {"happy","days","summer","spring"};
        String[] actual = {"happy","days","summer","spring"};

        assertArrayEquals("Expected and Actual Arrays are not equal",expected,actual);

    }
}

Similarly, to add JUnit5 we can add below mentioned dependencies to the POM.xml.

<dependency>
     <groupId>org.junit.jupiter</groupId>
     <artifactId>junit-jupiter-engine</artifactId>
     <version>5.8.2</version>
     <scope>test</scope>
</dependency>

<dependency>
     <groupId>org.junit.jupiter</groupId>
     <artifactId>junit-jupiter-api</artifactId>
     <version>5.8.2</version>
     <scope>test</scope>
</dependency>

Congratulations. We are able to add JUnit to Java or Maven project. Happy Learning!!

Grouped Assertions in JUnit 5 – assertAll()

HOME

What is JUnit5?

JUnit 5 is composed of several different modules from three different sub-projects.

JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage

The JUnit Platform serves as a foundation for launching testing frameworks on the JVM. It also defines the Test Engine  API for developing a testing framework that runs on the platform.

JUnit Jupiter is the combination of the new programming model and extension model for writing tests and extensions in JUnit 5. The Jupiter sub-project provides a Test Engine for running Jupiter based tests on the platform.

JUnit Vintage provides a Test Engine for running JUnit 3 and JUnit 4 based tests on the platform. It requires JUnit 4.12 or later to be present on the class path or module path.

To use JUnit5, add the Junit5 maven dependency to the POM.xml

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-engine</artifactId>
    <version>5.8.2</version>
    <scope>test</scope>
</dependency>

<dependency>
	<groupId>org.junit.jupiter</groupId>
	<artifactId>junit-jupiter-api</artifactId>
    <version>5.8.2</version>
	<scope>test</scope>
</dependency>

JUnit Jupiter comes with many of the assertion methods that JUnit 4 already has and adds a few more that lend themselves well to being used with Java 8 lambdas. All JUnit Jupiter assertions are static methods. They come from class:

org.junit.jupiter.api.Assertions

JUnit 5 supports an additional Assertion feature called Grouped assertions. In JUnit4, when you have to assert let’s say – 3 different conditions, then you need to write 3 different assertions and they will be executed sequentially. Imagine, 2nd assertion fails, then the program stops there and will not go to the 3rd assertion. To overcome this problem, JUnit5 has a new assertion called Grouped assertion that execute all the conditions present within it using assertAll(), irrespective of the fact if any assertion fails or not and generate a consolidated report with all the failures. To be honest, I love this feature and use very frequently.

Grouped Assertions With Heading As Parameter

Example 1– Below is an example of positive assertAll(). In the below example, all assertions are of same type assertEquals() within assertAll() assertion. It consists of the heading parameter with the value “GroupedAssertionsWithSameAssertionType”.

@Test
void allPositive1() {
   assertAll(
     "GroupedAssertionsWithSameAssertionType",
      () -> assertEquals(8, 5+3, "8 is not sum of 5 and 3"),
      () -> assertEquals("java", "JAVA".toLowerCase()),
      () -> assertEquals(16,4*4,"16 is not product of 4 and 4")
   );
}

Result

As all 3 assertions pass, so the final result passes.

Example2 – In the below example, the assertions are of different types assertEquals(), assertNotNull and assertNotEquals() within assertAll() assertion. It consists of the heading parameter with the value “GroupedAssertionsWithDifferentAssertionType”.

@Test
void allPositive2() {

    String str ="Spring";
    assertAll(
        "GroupedAssertionsWithDifferentAssertionType",
         () -> assertEquals(8, 5+3, "8 is not sum of 5 and 3"),
         () -> assertNotNull(str, () -> "The string should be null"),
         () -> assertEquals("java", "JAVA".toLowerCase()),
         () -> assertNotEquals(20,5*3,"20 is product of 5 and 3")
    );
 }

Result

As all 3 assertions pass, so the final result passes.

Example 3 – In the below example, out of 4 assertions, 3 assertions are failing, so the output will have the detail about all 3 assertion errors.

 @Test
 void allNegative() {

    String str ="Spring";
    Iterable<String> iterat1 = new ArrayList<>(asList("Java", "Junit4", "Test"));
    Iterable<String> iterat2 = new ArrayList<>(asList("Java", "Junit5", "Test"));
    // In a grouped assertion all assertions are executed, and all failures will be reported together
    assertAll(
        "Negative-GroupedAssertionsWithDifferentAssertionType",
        () -> assertIterableEquals(iterat1, iterat2),
        () -> assertNull(str, () -> "The string should be null"),
        () -> assertEquals("java", "JAVA"),
        () -> assertNotEquals(20,5*3,"20 is product of 5 and 3")
     );
 }

Result

As one of the assert in the group fails, instead of AssertionFailureError it results in MultipleFailuresError thereby displaying the heading of the grouped assertion passed as the input parameter i.e. Negative-GroupedAssertionsWithDifferentAssertionType in this example. This image shows all the 3 assertion failures.

Assertion 1 fails as we were expecting JUnit4 , but response has JUnit5
Assertion 2 fails as string was not NULL.
Assertion 3 fails as java is not equal to JAVA (case sensitivity).

Grouped Assertions Without Heading As Parameter

The assertAll () can be implemented without using the heading parameter. The below example is same as the above one, we are just skipping the heading part.

@Test
void allNegative() {

      String str ="Spring";
      Iterable<String> iterat1 = new ArrayList<>(asList("Java", "Junit4", "Test"));
      Iterable<String> iterat2 = new ArrayList<>(asList("Java", "Junit5", "Test"));
      // In a grouped assertion all assertions are executed, and all failures will be reported together
      assertAll(
          () -> assertIterableEquals(iterat1, iterat2),
          () -> assertNull(str, () -> "The string should be null"),
          () -> assertEquals("java", "JAVA"),
          () -> assertNotEquals(20,5*3,"20 is product of 5 and 3")
      );
  }

Result

The result displays without heading.

Nested Or Dependent Grouped Assertions

When one assertAll() includes one or more assertAll() then these are referred to as a Nested or Dependent grouped assertions.

Example 1 – In this case, first the assertAll() validates if sum is correct or not. The sum is correct here, so the control moves to the nested assertAll() to verify all the assertions present within it.

@Test
void allDependentPositive() {

   String str ="Spring";

   // Within a code block, if an assertion fails the subsequent code in the same block will be skipped.
    assertAll(
         "DependentPositiveAssertions",
         () -> {
                assertEquals(8, 5 + 3, "8 is not sum of 5 and 3");

                // Executed only if the previous assertion is valid.
                assertAll("sub-heading",
                   () -> assertNotNull(str, () -> "The string should be null"),
                   () -> assertEquals("java", "JAVA".toLowerCase()),
                   () -> assertEquals(20, 5 * 4, "20 is product of 5 and 4")
                ); // end of inner AssertAll()
           }

      );  // end of outer AssertAll()
 }

Result

All the assertions within nested assertAll() are passes. So the final result passes.

Example 2 – In the below example, outer AssertAll() fails, so all the assertions within nested/dependent assertAll() are not executed.

 @Test
 void allDependentNegative() {

    String str ="Spring";

    // Within a code block, if an assertion fails the subsequent code in the same block will be skipped.
    assertAll(
       "DependentPositiveAssertions",
        () -> {
                assertEquals(8, 5 + 4, "8 is not sum of 5 and 3");

                // Executed only if the previous assertion is valid.
                assertAll("sub-heading",
                   () -> assertNull(str, () -> "The string should be null"),
                   () -> assertNotEquals("java", "JAVA".toLowerCase()),
                   () -> assertNotEquals(20, 5 * 4, "20 is product of 5 and 4")
               ); // end of inner AssertAll()
           }

        ); // end of outer AssertAll()
    }

Result

Example 3 – In the below example, outer AssertAll() passes, so all the assertions within nested/dependent assertAll() are executed.

@Test
void allDependentNegative1() {

    String str ="Spring";

   // Within a code block, if an assertion passes the subsequent code in the same block will be executed.
    assertAll(
      "DependentNegativeAssertions",
      () -> {
             assertEquals(8, 5 + 3, "8 is not sum of 5 and 3");

             // Executed only if the previous assertion is valid.
             assertAll("sub-heading",
               () -> assertNull(str, () -> "The string should be null"),
               () -> assertNotEquals("java", "JAVA".toLowerCase()),
               () -> assertNotEquals(20, 5 * 4, "20 is product of 5 and 4")
             ); // end of inner AssertAll()
         }

      ); // end of outer AssertAll()
    }

Result

The nested assertions have failed. So they can be seen in the execution status.

In short,

  1. When first assertAll() method passes, then all the subsequent assertions within that block will be executed and these assertions can further pass or fails.
  2. When the first assertAll() assertion fails, then the execution of subsequent assertions are skipped.

That’s it! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!