TestNG was developed by a famous programmer named “Cedric Beust”. It is distributed under Apache Software License and is easily available to download. TestNG requires JDK 7 or higher. TestNG is a testing framework inspired by JUnit and NUnit, but introduces some new functionalities that make it more powerful and easier to use.
An assertion is a way to verify that the expected result and the actual result match or not in the test case. A test is considered successful ONLY if it is completed without throwing any exceptions. If the current value and the expected value match then the assertion passes and when the assertion passes nothing happens. But when an assertion fails, it will fail the test case.
There are various ways to perform assertions in API Testing. For API Testing, we are using Rest Assured, which uses either Hamcrest or JUnit assertions. We are going to discuss Hamcrest Assertions here.
What is Hamcrest?
Hamcrest is a framework for writing matcher objects, allowing ‘match’ rules to be defined declaratively. We do not need to add Hamcrest dependency explicitly as the Rest-Assured 4.3.3 version includes itself. To learn more about Hamcrest, please refer to this link.
We need to add the below dependency to use Hamcrest in the project.
equalTo – It checks whether the extracted string from JSON is equal to the expected string.
equalToIgnoringCase – It checks whether the extracted string from JSON is equal to the expected string without considering the case (small or capital).
equalToIgnoringWhiteSpace – It checks whether the extracted string from JSON is equal to the expected string by considering the white spaces.
containsString– It checks whether the extracted string from JSON contains the expected string as a substring.
startsWith– It checks whether the extracted string from JSON is starting with a given string or character.
endsWith – It checks whether the extracted string from JSON is ending with a given string or character.
Below assertions are imported from the package shown below:-
Below are examples to show the use of collection-related assertions.
import io.restassured.RestAssured;
import io.restassured.http.ContentType;
import org.testng.annotations.Test;
import static org.hamcrest.Matchers.is;
import static org.hamcrest.Matchers.nullValue;
public class HamcrestNullAssertion {
public String endpoint = "https://restful-booker.herokuapp.com/booking/1";
@Test
public void nullAssertion() {
RestAssured.given().contentType(ContentType.JSON)
.when().get(endpoint)
.then().body("totalprice1", is(nullValue()));
}
}
The output of the above program is
hasKey – It checks whether the extracted map has an expected key.
import io.restassured.RestAssured;
import io.restassured.http.ContentType;
import org.testng.annotations.Test;
import static org.hamcrest.Matchers.hasKey;
public class HamcrestHasKeyAssertion {
public String endpoint = "https://restful-booker.herokuapp.com/booking/1";
@Test
public void collectionAssertions() {
RestAssured.given().contentType(ContentType.JSON)
.when().get(endpoint)
.then().body("bookingdates",hasKey("checkin"));
}
}
The output of the above program is
Not Assertion
The not assertion inverts the meaning of the other assertions. For example, if you want to perform negative assertions, then we can use any assertions with NOT.
The below assertion is imported from the package shown below:-
Below are examples to show the use of negative assertions.
import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.not;
import io.restassured.RestAssured;
import io.restassured.http.ContentType;
import org.testng.annotations.Test;
public class HamcrestNotAssertion {
public String endpoint = "https://restful-booker.herokuapp.com/booking/1";
@Test
public void negativeAssertions() {
RestAssured.given().contentType(ContentType.JSON)
.when().get(endpoint)
.then().body("totalprice",not(equalTo(874)));
}
}
The output of the above program is
Multiple Assert Statements
In the below example, all 3 assertions will fail, but it will only execute the assertion and the first assertion is failed, then other assertions will not be executed.
import io.restassured.RestAssured;
import io.restassured.http.ContentType;
import org.testng.annotations.Test;
import static org.hamcrest.Matchers.equalTo;
public class HamcrestMultipleAssertions {
public String endpoint = "https://restful-booker.herokuapp.com/booking/1";
@Test
public void test1() {
RestAssured.given().contentType(ContentType.JSON)
.when().get(endpoint).then()
.body("firstname", equalTo("Jim"), // will fail
"lastname", equalTo("Smith"), // will fail
"totalprice", equalTo(314)); // will fail
}
}
The output of the above program is
To execute all the assertions present in the test case, we have to combine all the assertions into a single body, just like below. You can see that all the assertions failed, and they are shown in the response.
import io.restassured.RestAssured;
import io.restassured.http.ContentType;
import org.testng.annotations.Test;
import static org.hamcrest.Matchers.equalTo;
public class HamcrestMultipleAssertions {
public String endpoint = "https://restful-booker.herokuapp.com/booking/1";
@Test
public void test1() {
RestAssured.given().contentType(ContentType.JSON)
.when().get(endpoint).then()
.body("firstname", equalTo("Jim"), // will fail
"lastname", equalTo("Smith"), // will fail
"totalprice", equalTo(314)); // will fail
}
}
The output of the above program is
I have tried to show the use of a few of the most commonly used assertion methods. There are many more methods available in Hamcrest package. To know about other methods, write import static org.hamcrest.Matchers and add (.) at the end, it will show the list of all the methods available in Hamcrest.
We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
An assertion is used to test a logical expression. An assertion is true if the logical expression that is being tested is true and there are no bugs in the program. It can also be interpreted as a checkpoint or a validation point.
There are various types of assertions available in ReadyAPI. They are:-
Property Content
Compliance, Status and Standards
Script
SLA
JMS Response
Security
Example 1 – SLA Assertion
Let’s create an assertion that will check if the tested web service responds within a predefined time limit:
In Functional Tests, select POST Request in the Navigator panel and click Add Assertion:
In the dialog, select the SLA category on the left and then Response SLA on the right, and click Add
Let’s use 3000 milliseconds as the maximum allowed response time for our request. Enter 3000 and click the OK button.
Now, if the request takes longer than the specified number of milliseconds to complete, the assertion will trigger, and the test will fail. If the execution time is less than or equal to the specified value, the check will pass.
Run the request and see the SLA Response. This image shows that Response SLA Assertion is passed.
Example 2 – Check Response Contents
Now let’s see how you can verify response data. The sample response body has the JSON data format, so we will create an assertion for JSON data.
Make sure the request has a response.
In the subsequent dialog, select the Property Content category on the left and the JsonPath Match assertion on the right and click Add:
In the dialog, you need to enter a JSONPath expression that will extract some field from the response body and the expected value of this field:
Select the node on the toolbar and pick a value visually in the subsequent dialog. Let’s do this. Click the highlighted icon and select the name field of the first array item in the following dialog and click the OK button:
The JSONPath Expression field now contains the selector, and Expected Result contains the value extracted from the current response data:
Click the Save button to store the changes.
Since we have response data, the assertion will be applied immediately, and you will see its results on the Assertions page:
Example 3 – Check CONTAINS Assertion
In the subsequent dialog, select the Property Content category on the left and the JsonPath Match assertion on the right and click Add:
In the Contains Assertion dialog, I have mentioned “success” in Content which is present in the response body.
This image shows that the Contains Assertion is passed.
Now, let us add another Contains Assertion which does not contain the response body. In this case, the Assertion fails.
Example 4 – Smart Assertion
The Smart Assertion checks both the message content and the metadata such as headers, status codes, and parameters in accordance with the predefined set of rules.
To use this assertion, you need a ReadyAPI Test Pro license.
Send the request at least once so that ReadyAPI has a response to base the assertion on.
Received Data – The assertion will verify the payload of the request.
Received Metadata – The assertion will verify the metadata of the request; depending on the protocol, those can include headers, the HTTP status code, or Kafka partition and key values.
As I know, the Id value is dynamic for each response. So, I have unchecked that option.
The below image shows that the Smart Assertion is successful.
Now, let us add the id to the Smart Assertion.
This image shows that Smart Assertion is failing now.
We are done! Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!!
Spring Boot is an open-source micro framework which provides Java developers with a platform to get started with an auto configurable production-grade Spring application.
In this tutorial, lets see a SpringBoot REST Application and how it can be tested with the help of Rest Assured and Serenity
Below is the structure of a SpringBoot application project
Below are various Java classes present in a SpringBoot REST Application/API
SpringBootRestServiceApplication.java – The Spring Boot Application class generated with Spring Initializer. This class acts as the launching point for application.
pom.xml – Contains all the dependencies needed to build this project.
Student.java – This is JPA Entity for Student class
StudentRepository.java – This is JPA Repository for Student. This is created using Spring Data JpaRepository.
StudentController.java – Spring Rest Controller exposing all services on the student resource.
CustomizedExceptionHandler.java – This implements global exception handling and customize the responses based on the exception type.
ErrorDetails.java – Response Bean to use when exceptions are thrown from API.
StudentNotFoundException.java – Exception thrown from resources when student is not found.
data.sql – Data is loaded from data.sql into Student table. Spring Boot would execute this script after the tables are created from the entities.
HTTP also defines standard response codes.
200 – SUCESS
404 – RESOURCE NOT FOUND
400 – BAD REQUEST
201 – CREATED
401 – UNAUTHORIZED
415 – UNSUPPORTED TYPE – Representation not supported for the resource
500 – SERVER ERROR
Let’s consider a few HTTP Methods:
GET : Should not update anything. Should return same result in multiple calls.
POST : Should create a new resource. Ideally return JSON with link to newly created resource. Same return codes as get possible. In addition – Return code 201 (CREATED) can be used.
PUT : Update a known resource. ex: update client details. Possible Return Codes : 200(OK) + 404 (NOT FOUND) +400 (BAD REQUEST)
DELETE : Used to delete a resource. Possible Return Codes : 200(OK).
We will create a Student Resource exposing three services using proper URIs and HTTP methods:
Retrieve all Students – @GetMapping(“/students”)
Get details of specific student – @GetMapping(“/students/{id}”)
Delete a student – @DeleteMapping(“/students/{id}”)
First, let us see what are @RestController, @AutoWired, @GetMapping and @PathVariable annotations.
1. @RestController
SpringBoot RestController annotation is use to create RESTful web services using Spring MVC. Spring RestController takes care of mapping request data to the request defined handles method.
2. @Autowired
The Spring framework enables automatic dependency injection. In other words, by declaring all the bean dependencies in a Spring configuration file, Spring container can autowire relationships between collaborating beans. This is called Spring bean autowiring. To know more about Autowired, you can refer this tutorial.
3. @GetMapping
It is annotation for mapping HTTP GET requests onto specific handler methods. Specifically, @GetMapping is a composed annotation that acts as a shortcut for @RequestMapping(method = RequestMethod.GET)
4. @PathVariable
This annotation can be used to handle template variables in the request URI mapping, and use them as method parameters. In this example, we use @PathVariable annotation to extract the templated part of the URI represented by the variable {id}.
A simple GET request to /students/{id} will invoke retrieveStudent with the extracted id value
http://localhost:8080/students/1001
Code of StudentController.java is below
@RestController
public class StudentController {
@Autowired
private StudentRepository studentRepository;
@GetMapping("/students")
public List retrieveAllStudents() {
return studentRepository.findAll();
}
@GetMapping("/students/{id}")
public EntityModel retrieveStudent(@PathVariable long id) {
Optional student = studentRepository.findById(id);
if (!student.isPresent())
throw new StudentNotFoundException("id-" + id);
EntityModel resource = EntityModel.of(student.get());
WebMvcLinkBuilder linkTo = linkTo(methodOn(this.getClass()).retrieveAllStudents());
resource.add(linkTo.withRel("all-students"));
return resource;
}
Scenario 1- Below picture shows how we can execute a Get Request Method on a Resource using Postman
Above scenario can be tested in the below way.
@ReceiveUserDetails
Scenario Outline: Send a valid Request to get user details
Given I send a request to the URL "/students" to get user details
Then the response will return status 200 and id and names and passport_no
Examples:
|studentID |studentNames |studentPassportNo|
|10010 |Tom |A1234567 |
|10020 |Shawn |B1234568 |
|10030 |John |C1239875 |
Test Code to test above scenario (StepDefinition file)
@SpringBootTest(classes = com.springboot.rest.demo.SpringBootRestServiceApplication.class, webEnvironment = WebEnvironment.RANDOM_PORT)
public class GetStudentsDefinition {
private final static String BASE_URI = "http://localhost";
@LocalServerPort
private int port;
private ValidatableResponse validatableResponse;
private void configureRestAssured() {
RestAssured.baseURI = BASE_URI;
RestAssured.port = port;
}
protected RequestSpecification getAnonymousRequest() throws NoSuchAlgorithmException {
configureRestAssured();
return given();
}
@Given("^I send a request to the URL \"([^\"]*)\" to get user details$")
public void iSendARequest(String endpoint) throws Throwable {
validatableResponse = getAnonymousRequest().contentType(ContentType.JSON)
.when().get(endpoint).then();
}
@Then("^the response will return status (\\d+) and id (.*) and names (.*) and passport_no (.*)$")
public void extractResponse(int status, String id, String studentName,
String passportNo) {
validatableResponse.assertThat().statusCode(equalTo(status)).body(containsString(id))
.body(containsString(studentName)).body(containsString(passportNo));
}
1. The @SpringBootTest annotation tells Spring Boot to look for a main configuration class (one with @SpringBootApplication, for instance) and use that to start a Spring application context.
2. WebEnvironment.RANDOM_PORT is used to create run the application at some random server port.
3. For assertion purpose, we use Hamcrest Matchers. Hamcrest is a framework for software tests. Hamcrest allows checking for conditions in your code via existing matchers classes. It also allows you to define your custom matcher implementations.
4. @LocalServerPort gets the reference of port where the server has started. It helps in building the actual request URIs to mimic real client interactions.
5. ContentType.JSON is imported fromrestassured as well as ValidatableResponse
Scenario 2- Below picture shows how we can execute a Get Request Method to get detail of a specific Student. Here, we want details of student of Id 10020
@ReceiveAUserDetail
Scenario: Send a valid Request to get user details
Given I send a request to the URL "/students/10020" to get user detail of a specific user
Then the response will return status 200 and name "Shawn"
Test Code to test above scenario (StepDefinition file)
@Given("^I send a request to the URL \"([^\"]*)\" to get user detail of a specific user$")
public void sendRequestForSpecificUser(String endpoint) throws Throwable {
validatableResponse = getAnonymousRequest().contentType(ContentType.JSON).when().get(endpoint).then();
System.out.println("RESPONSE :" + validatableResponse.extract().asString());
}
@Then("^the response will return status (\\d+) and name \"([^\"]*)\"$")
public void extractResponseOfSpecificUser(int status, String name) {
validatableResponse.assertThat().statusCode(equalTo(status)).body("name", equalTo(name));
}
Scenario 3- Below picture shows how we can execute a Get Request Method for incorrect Student
@IncorrectUserId
Scenario: Send a valid Request to get user details
Given I send a request to the URL "/students/7" to get user detail of a specific user
Then the response will return status 404 and message "id-7"
Test Code to test above scenario (StepDefinition file)
@Given("^I send a request to the URL \"([^\"]*)\" to get user detail of a specific user$")
public void sendRequestForSpecificUser(String endpoint) throws Throwable {
validatableResponse = getAnonymousRequest().contentType(ContentType.JSON).when().get(endpoint).then();
System.out.println("RESPONSE :" + validatableResponse.extract().asString());
}
@Then("^the response will return status (\\d+) and message \"([^\"]*)\"$")
public void extractResponseOfInvalidUser(int status, String message) {
validatableResponse.assertThat().statusCode(equalTo(status)).body("message", equalTo(message));
}
Assertions in TestNG are a way to verify that the expected result and the actual result matches or not in the test case. A test is considered successful ONLY if it is completed without throwing any exception. An example of assertion can be logging into the website, checking the title of the webpage, verifying the functionality of an input box that takes only integers, etc.
Below are few commonly used assertions in TestNG.
1. Assert.assertEquals(String actual, String expected) – It takes two string arguments and checks whether both strings are equal or not. If they are not, it will fail the test and an AssertionError is thrown.
2. Assert.assertEquals(String actual, String expected, String message) – It takes two string arguments and checks whether both strings are equal or not. If they are not, it will fail the test and an AssertionError is thrown with message provided as argument.
3. Assert.assertEquals(boolean actual, boolean expected) – It takes two Boolean arguments and checks whether both are equal or not. If they are not, it will fail the test and an AssertionError is thrown.
4. Assert.assertEquals(java.util.Collection actual, java.util.Collection expected, java.lang.String message) – It takes two collection objects and verifies both collections contain the same elements and with the same order. If they are not, it will fail the test and an AssertionError is thrown.
5. Assert.assertTrue(condition) – It takes one boolean argument and checks that a condition is true or not. If not, it will fail the test and an AssertionError is thrown.
6. Assert.assertTrue(condition, message) – It takes one boolean argument and checks that a condition is true or not. If not, it will fail the test and an AssertionError is thrown with message provided as argument.
7. Assert.assertFalse(condition) – It takes one boolean argument and checks that a condition is false or not. If not, it will fail the test and an AssertionError is thrown.
8. Assert.assertFalse(condition, message) – It takes one boolean argument and checks that a condition is false or not. If not, it will fail the test and an AssertionError is thrown with message provided as argument.
9. Assert.assertSame(String actual, String expected) – It asserts that two objects refer to the same object. If they do not, an AssertionError is thrown.
10. Assert.assertNotSame(String actual, String expected) – It asserts that two objects do not refer to the same object. If they do, an AssertionError is thrown.
import org.testng.Assert;
import org.testng.annotations.Test;
public class TestNGAssertionDemo {
@Test
public void testAssertions() {
// test data
String str1 = new String("TestNG");
String str2 = new String("TestNG");
String str3 = null;
String str4 = "TestNG";
String str5 = "TestNG";
String str6 = new String("Not_TestNG");
int val1 = 5;
int val2 = 6;
// Check that two objects are equal
Assert.assertEquals(str1, str2);
System.out.println("Equals Assertion is successful");
// Check that two objects are not equal
Assert.assertNotEquals(str1, str6);
System.out.println("NotEquals Assertion is successful");
// Check that a condition is true
Assert.assertTrue(val1 < val2);
System.out.println("True Assertion is successful");
// Check that a condition is false
Assert.assertFalse(val1 > val2);
System.out.println("False Assertion is successful");
// Check that an object isn't null
Assert.assertNotNull(str1);
System.out.println("Not Null Assertion is successful");
// Check that an object is null
Assert.assertNull(str3);
// Check if two object references point to the same object
Assert.assertSame(str4, str5);
System.out.println("Same Assertion is successful");
// Check if two object references not point to the same object
Assert.assertNotSame(str1, str3);
System.out.println("Not Same Assertion is successful");
}
}
Output
Equals Assertion is successful
NotEquals Assertion is successful
True Assertion is successful
False Assertion is successful
Not Null Assertion is successful
Same Assertion is successful
Not Same Assertion is successful
Lets see if an assertion fails, how the output looks shown below. In the below example, we are verifying the pageTitle of Gmail. If the test fails, we should see the message provided in the assertion also.
import org.openqa.selenium.WebDriver;
importorg.openqa.selenium.firefox.FirefoxDriver;
import org.testng.Assert;
import org.testng.annotations.Test;
public class TestNGAssertionFailureDemo {
@Test
public void AssertionFailure() throws InterruptedException {
System.setProperty("webdriver.gecko.driver", "src\\test\\resources\\webdrivers\\window\\geckodriver.exe");
WebDriver driver = new FirefoxDriver();
// Test Condition 1: If Page title matches with actualTitle then it finds email
// title and enters the value which we pass
driver.get("https://www.gmail.com");
String actualTitle = "Google";
String expectedTitle = driver.getTitle();
Assert.assertEquals(expectedTitle, actualTitle, "Incorrect page title");
}
}
Output
FAILED: AssertionFailure
java.lang.AssertionError: Incorrect page title expected [Google] but found [Gmail]
You can show in the output console, there is an error message “Incorrect page title” as we have mentioned this message in the Assertion.
QA Automation Expert 
