How to run PyTest Framework in GitHub Actions

Last Updated On

April 4, 2025

This tutorial explains the steps to create a GitHub Action for the PyTest Framework. It also guides you on how to execute the tests in that workflow.

Table of Contents

Why GitHub?

GitHub is a collaborative platform. It supports version control and code collaboration. Automated testing and issue tracking are also supported. These are crucial elements in the software testing process. It promotes transparency, collaboration, and efficiency in the development and testing workflows.

CI/CD pipelines have contributed to the success of the DevOps cycle in all software development projects. This is a holistic process that bridges development and operations. Continuous integration helps development teams deploy code efficiently, and continuous delivery automates code deployment.

Implementation Steps

Step 1 – Create GitHub Actions and Workflows

I have a repository available in GitHub – “PyTest_Framework” as shown in the below image. Go to the “Actions” tab. Click on the “Actions” tab.

Step 2 – Select the type of Actions

You will see that GitHub recommends Actions depending on the project. In our case, it is recommending actions suitable for a Java project. I have selected the “Python application” option as my project is built in Maven.

Step 3 – Generation of Sample pipeline

If you choose an existing option, it will automatically generate a .yaml for the project as shown below.

We will replace the current workflow with the following yml file as shown below:

name: PyTest Framework - Python
on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

permissions:
  contents: read

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4
      - name: Set up Python 3.12.1
        uses: actions/setup-python@v4
        with:
          python-version: 3.12.1

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install pytest
          pip install pytest-selenium
   
      - name: Test with PyTest
        run: pytest --html=tests/Reports/Report.html  

      - name: Test Report Generation
        uses: actions/upload-artifact@v4
        if: success() || failure()
        with:
          name: Pytest Report           # Name of the folder
          path: tests/Reports           # Path to test results

This command will update the Python’s Pip version to the latest available.

python -m pip install --upgrade pip

As we need pytest and pytest-selenium to execute the tests, we have installed them also using the below command:

pip install pytest
pip install pytest-selenium

We want the HTML Report to be generated in tests/Reports folder. So, we have mentioned the path.

  - name: Test with PyTest
    run: pytest --html=tests/Reports/Report.html

Step 4 – Commit the changes

After the changes, hit the “Start Commit” button.

This will give the option to add a description for the commit. It will also enable the user to commit either to the main branch or commit to any other branch that exists in the project. My personal prefernece is to create a new branch like shown below and then commit the changes in that new branch.

If you have created a new branch, then it will ask to create a PULL request. Once you will create the PULL request, we can see the new workflow running.

It will create a python-app.yml file as shown below in the project.

Step 5 – Verify that the workflow is running

Next, head over to the “Actions” tab, and you will see your YAML workflow file present under the tab. The yellow sign represents that the job is in the queue.

In Progress – When the job starts building and running, you will see the status change from “Queued” to “In progress”.

Passed – If the build is successful, you will see a green tick mark.

Below is the execution log. Go to “Test with PyTest“ phase.

Click on the workflow and the below screen is displayed. It shows the status of the run of the workflow, the total time taken to run the workflow, and the name of the .yml file.

Below shows all the steps of the workflow.

Step 6 – Published Test Report on GitHub

From the logs of the Workflow, you can see that the “Test Report Generation” step was executed successfully.

Once the pipeline run, a Report folder will be generated as shown in the below image:

When we click on the folder PyTest Report, a zipped file will be downloaded. We can extract it to see all the files contained within it.

Extract the folder and see the Report.html.

Below is the image of Report.html.

The complete code can be found here on GitHub – vibssingh/PyTest_Framework.

Congratulations! We just created our CI workflow for running our Python Robot Framework.

April 4, 2024April 19, 2024 vibssingh

How to run Python Selenium tests with Jenkins

Last Updated On

April 19, 2024

HOME

In the previous tutorial, we have seen the How to generate HTML Reports in Jenkins. In this tutorial, we show you how to integrate PyTest tests with Jenkins.

Table Of Contents

Prerequisite:

Jenkin’s installed and started on the computer. The current Jenkins version is – 2.361.2

Implementation Steps

To generate HTML Report in Jenkins, we need to download HTML Publisher Plugin. Please refer to this tutorial to install the plugin – How to install Plugins in Jenkins.

Step 1: Create a new FreeStyle project

Give the Name of the project – PyTestFramework_Demo
Click on the Freestyle project.
Click on the OK button.

In the General section, enter the project description in the Description box – This is demo integration of Python with Selenium tests to run in Jenkins.

Step 2: Select a custom workspace

Mention the full path of the project in the Use custom workspace.

Select Source Code Management as None if the project is locally present on the machine.

Step 3: Build Management

The source code used for this tutorial can be seen here – Page Object Model Implementation of Python with Selenium – PyTest.

Go to the Build section of the new job. Select “Execute Windows batch command”.

Mention the command needed to execute the tests. In this case, I have to execute all the tests, so used the below command:

cd tests
pytest  --html=$WORKSPACE/Reports/report.html -s

We have tests present in test directory. So we have used cd tests.

$WORKSPACE is the environment variable. It provides the absolute path of the workspace.

Step 4: Select “Publish HTML reports” from “Post Build Actions”

Scroll down to “Post Build Actions” and click on the “Add Post Build Actions” drop-down list. Select “Publish HTML reports“.

If you want to see where the report will be saved in Jenkins, go to the Dashboard -> PyTestFramework_Demo -> Workspace -> Reports ->report.html.

This can be only viewed once the project is built and the job is executed. Then go to Workspace to check the location of the report.html file.

Enter the HTML directory to archive – Reports, Index page[s] – report.html, and Report title – HTML Report.

Click on the Apply and Save buttons.

We have created a new FreeStyle project “PyTestFramework_Demo” with the configuration to run the Python tests in windows batch.

Step 5: Execute the tests

Let’s execute it now by clicking on the “Build Now” button.

Right-click on Build Number (here in my case it is #1).

Click on Console Output to see the result.

Step 6: View the HTML Report

Once the execution is completed, click on go “Back to Project“, and we could see a link to view the “HTML Report“.

We can see here that the HTML Report link is displayed in the Console.

Below is the HTML Report generated in Jenkins.

There are chances that the report won’t look very pretty. The reason is that CSS is stripped out because of the Content Security Policy in Jenkins.

The default rule set in Jenkins is:

sandbox; default-src 'none'; img-src 'self'; style-src 'self';

To know more about this, please refer to this tutorial – https://www.jenkins.io/doc/book/security/configuring-content-security-policy/.

We can customize Content Security Policy in Jenkins. But keep in mind that it should be done after checking with the Security team in your organization. This is a workaround solution. I can’t emphasize enough that this is not a standard practice.

Go to Manage Jenkins -> Manage Nodes and Clouds.

Click on the Script Console option.

Type in the following command and Press Run. If you see the output as ‘Result:’ then the protection is disabled. Re-Run your build and you can see that the new HTML files archived will have the CSS enabled.

System.setProperty("hudson.model.DirectoryBrowserSupport.CSP","")

Re-run the HTMLReport_Demo project. Now you can see a properly rendered HTML Report.

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

Additional Tutorials

*How to Install Python on Windows 11*
*How to install and setup Robot Framework for Python*
*How to rerun failed tests in Robot Framework*
*How to implement tagging in Robot Framework*
*How to set variable values from Runtime command in Robot Framework*
*How to load data from CSV files in the Robot Framework?*

January 29, 2024 vibssingh

How to run Python Rest API tests with GitHub Actions

Last Updated On

HOME

This tutorial explains the steps to create a GitHub Action for the Rest API tests built in Python and execute the tests in that workflow.

Table of Contents

Why GitHub?

GitHub Actions are automated tasks, or workflows, that you can set up in your repository to build, test, package, release, and deploy any project on GitHub. With GitHub Actions, you can orchestrate any workflow, based on nearly any event, while GitHub manages the execution, provides rich feedback, and secures every step along the way.

Why should you care? Well, imagine merging your code into the main branch, only to realize later that it broke your build. With GitHub Actions, you can automate your unit tests to run on every push or pull request, thereby catching bugs before they creep into your production code. This means less time debugging and more time developing!

Implementation Steps

Step 1 – Create GitHub Actions and Workflows

I have a repository available in GitHub – RestAPITesting_Python as shown in the below image. Go to the “Actions” tab. Click on the “Actions” tab.

Step 2 – Select the type of Actions

You will see that GitHub recommends Actions depending on the project. In our case, it is recommending actions suitable for a Python project. I have selected the “Python application” option.

Step 3 – Generation of Sample pipeline

If you choose an existing option, it will automatically generate a .yaml for the project as shown below.

We will replace the current workflow with the following yml file as shown below:

# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python

name: Python application

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

permissions:
  contents: read

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v3
    - name: Set up Python 3.12.1
      uses: actions/setup-python@v3
      with:
        python-version: "3.12.1"
   
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install pytest
        pip install requests
       
    - name: Test with pytest
      run: |
        cd TestCases
        pytest  --verbose --capture=no

Step 4 – Commit the changes

After the changes, hit the “Start Commit” button.

This will give the option to add a description for the commit. It will also enable the user to commit either to the main branch or commit to any other branch that exists in the project. Click on the “Commit new file” button to set up the workflow file.

It will create a python-app.yml file as shown below in the project.

Step 5 – Verify that the workflow is running

Next, head over to the “Actions” tab, and you will see your YAML workflow file present under the tab. The yellow sign represents that the job is in the queue.

In Progress – When the job starts building and running, you will see the status change from “Queued” to “in progress”.

Passed – If the build is successful, you will see a green tick mark.

Step 6 – Verify the execution status

Below is the execution log.

Click on the workflow and the below screen is displayed. It shows the status of the run of the workflow, the total time taken to run the workflow, and the name of the .yml file.

Below shows all the steps of the workflow.

The complete code can be found here on GitHub – vibssingh/RestAPITesting_Python.

Congratulations! We just created our CI workflow for running our Rest API test cases for Python.

January 5, 2024 vibssingh

How to host Extent Report on GitHub Pages with Github Actions

Last Updated On

HOME

This tutorial explains the steps to host Extent Reports on GitHub Pages with GitHub Actions.

Table of Contents

What is GitHub Action?

GitHub Actions is a continuous integration and continuous delivery (CI/CD) platform that allows you to automate your build, test, and deployment pipeline. You can create workflows that build and test every pull request to your repository, or deploy merged pull requests to production.

What is a GitHub Page?

GitHub Pages is a static site hosting service that takes HTML, CSS, and JavaScript files straight from a repository on GitHub, optionally runs the files through a build process, and publishes a website.

With GitHub Pages, we get a subdomain as github.io. To host your site, first and foremost thing is we need to have a GitHub account and a repository that will have our Automation code to run the Test Cases. Secondly, your site will be named as <username>.github.io

Why use GitHub Page to host the Report?

As automation testers, we are familiar with the challenges of sharing daily or weekly status reports on the progress of our automation efforts with management. Typically, we rely on tools like Extent Report or Allure Report and send these reports via email for management to review the current status of our automation progression or regression. However, there is a more efficient solution available: leveraging GitHub Pages to host our automation report.

By using GitHub Pages, we can eliminate the pain of manually sending reports to management and instead provide them with a simple URL. This means that whenever management wants to check the latest update on our automation progress, they can simply browse through the provided URL.

Important points

1. The Web Application tests need to run in the headless mode in GitHub Workflow.

          ChromeOptions options = new ChromeOptions();
          options.addArguments("--no-sandbox");
          options.addArguments("--disable-dev-shm-usage");
          options.addArguments("--headless");
	      driver = new ChromeDriver(options);

2. Use the below code to use gh-pages branch to host the Extent Report:

    - name: Deploy pages
      uses: JamesIves/github-pages-deploy-action@v4.5.0
      with:
        branch: gh-pages
        folder: ./ExtentReport/Reports

Implementation Steps

Step 1 – Create GitHub Actions and Workflows

I have a repository available in GitHub – ExtentReport_GitHubActions as shown in the below image. Go to the “Actions” tab. Click on the “Actions” tab.

Step 2 – Select the type of Actions

You will see that GitHub recommends Actions depending on the project. In our case, it is recommending actions suitable for a Java project. I have selected the “Java with Maven” option as my project is built in Maven.

Step 3 – Generation of Sample pipeline

If you choose an existing option, it will automatically generate a .yaml for the project as shown below.

We will replace the current workflow with the following yml file that will generate the ExtentReport in the GitHub as shown below:

name: ExtentReport

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v3
    - name: Set up JDK 17
      uses: actions/setup-java@v3
      with:
        java-version: '17'
        distribution: 'temurin'
        cache: maven
    - name: Build with Maven
      run: mvn clean test

    - name: Deploy pages
      uses: JamesIves/github-pages-deploy-action@v4.5.0
      with:
        branch: gh-pages
        folder: ./ExtentReport/Reports

Step 4 – Commit the changes

After the changes, hit the “Start Commit” button.

This will give the option to add a description for the commit. It will also enable the user to commit either to the main branch or commit to any other branch that exists in the project. Click on the “Commit new file” button to set up the workflow file.

It will create a maven.yml file as shown below in the project.

Step 5 – Verify that the workflow is running

Next, head over to the “Actions” tab, and you will see your YAML workflow file present under the tab. The yellow sign represents that the job is in the queue. The pipeline will fail. It is because gh-pages is not configured yet now.

Navigate to the Repository Settings of GitHub, then select Action → General.

Check the “Read and Write Permissions” option, and save the changes.

Go to Settings->Pages and you will see “GitHub Pages”. Go to the Branch, select “main” and “/root” and click on the “Save” button.

Here we are using gh-pages to host our automation report. Once you run your workflow for the first time this branch will not be present, so run the workflow, and you will see gh-pages branch is made. Now again go to the pages section of your repository change the source branch of gh-pages as below and now again run it.

Click on the workflow and the below screen is displayed. It shows the status of the run of the workflow, the total time taken to run the workflow, and the name of the .yml file.

Below are all the steps of the workflow.

Published ExtentReport on GitHub

From the logs of the Workflow, you can see that the “Deploy Pages” stage was executed successfully.

Once the pipeline runs, a new workflow will be generated as shown in the below image:

When we click on the pages build and deployment workflow, we will see 3 stages. The deploy stage contains the link to the ExtentReport.

https://vibssingh.github.io/ExtentReport_GitHubActions/

The complete code can be found here on GitHub - vibssingh/ExtentReport_GitHubActions.

Congratulations! We just created our CI workflow for running the ExtentReport in GitHub.

January 3, 2024April 20, 2024 vibssingh

How to run Robot Framework in GitHub Actions

Last Updated On

April 20, 2024

HOME

This tutorial explains the steps to create a GitHub Action for the Robot Framework in Python and execute the tests in that workflow.

Table of Contents

Why GitHub?

GitHub serves as a collaborative platform that supports version control, code collaboration, automated testing, and issue tracking, all of which are crucial elements in the software testing process. It promotes transparency, collaboration, and efficiency in the development and testing workflows.

CI/CD pipelines have contributed to the success of the DevOps cycle in all software development projects. This is a holistic process that bridges development and operations. Continuous integration helps development teams deploy code efficiently, and continuous delivery automates code deployment.

Implementation Steps

Step 1 – Create GitHub Actions and Workflows

I have a repository available in GitHub – RobotFramework_POM as shown in the below image. Go to the “Actions” tab. Click on the “Actions” tab.

Step 2 – Select the type of Actions

Step 3 – Generation of Sample pipeline

If you choose an existing option, it will automatically generate a .yaml for the project as shown below.

We will replace the current workflow with the following yml file as shown below:

name: Robot Framework - Python
on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

permissions:
  contents: read

jobs:
  test:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v3
      - name: Set up Python 3.12.1
        uses: actions/setup-python@v3
        with:
          python-version: 3.12.1
  
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install robotframework
          pip install robotframework-seleniumlibrary
  
      - name: Test with RobotFramework
        run: robot .  
        
      - name: Test Report Generation
        uses: actions/upload-artifact@v4
        if: success() || failure()
        with:
          name: Report                # Name of the folder
          path: report.html           # Path to test results

This command will update the Python’s Pip version to the latest available.

python -m pip install --upgrade pip

As we need robotframework and seleniumlibrary to execute the tests, we have installed them also using the below command:

pip install robotframework
pip install robotframework-seleniumlibrary

Step 4 – Commit the changes

After the changes, hit the “Start Commit” button.

This will give the option to add a description for the commit. It will also enable the user to commit either to the main branch or commit to any other branch that exists in the project. Click on the “Commit new file” button to set up the workflow file.

It will create a python-app.yml file as shown below in the project.

Step 5 – Verify that the workflow is running

Next, head over to the “Actions” tab, and you will see your YAML workflow file present under the tab. The yellow sign represents that the job is in the queue.

In Progress – When the job starts building and running, you will see the status change from “Queued” to “In progress”.

Passed – If the build is successful, you will see a green tick mark.

Below is the execution log. Go to “Test with RobotFramework” phase.

Click on the workflow and the below screen is displayed. It shows the status of the run of the workflow, the total time taken to run the workflow, and the name of the .yml file.

Below shows all the steps of the workflow.

Step 6 – Published Test Report on GitHub

From the logs of the Workflow, you can see that the Test Report step was executed successfully.

Once the pipeline run, a Report folder will be generated as shown in the below image:

When we click on the folder Report, a zipped file will be downloaded, and we can extract it to see all the files contained within it.

Extract the folder and see the report.html.

Below is the image of report.html.

The complete code can be found here on GitHub – vibssingh/RobotFramework_POM.

Congratulations! We just created our CI workflow for running our Python Robot Framework.

December 27, 2023April 13, 2024 vibssingh

How to run Rest API tests with GitHub Actions

Last Updated On

April 13, 2024

HOME

This tutorial explains the steps to create a GitHub Action for the Java Rest API tests and execute the tests in that workflow.

Table of Contents

Why GitHub?

Implementation Steps

Step 1 – Create GitHub Actions and Workflows

I have a repository available in GitHub – RestAssured_TestNG_Demo as shown in the below image. Go to the “Actions” tab. Click on the “Actions” tab.

Step 2 – Select the type of Actions

Step 3 – Generation of Sample pipeline

If you choose an existing option, it will automatically generate a .yaml for the project as shown below.

We will replace the current workflow with the following yml file as shown below:

name: Rest API Tests using Rest Assured with TestNG
 
on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]
 
jobs:
  build:
 
    runs-on: ubuntu-latest
 
    steps:
    - uses: actions/checkout@v4
    - name: Set up JDK 17
      uses: actions/setup-java@v4
      with:
        java-version: '17'
        distribution: 'temurin'
        cache: maven
     
    - name: Test Execution
      run: mvn clean test
       
    - name: Test Report Generation
      uses: actions/upload-artifact@v4
      if: success() || failure()
      with:
          name: TestNG Report                 # Name of the folder
          path: target/surefire-reports/      # Path to test results

Step 4 – Commit the changes

After the changes, hit the “Start Commit” button.

This will give the option to add a description for the commit. It will also enable the user to commit either to the main branch or commit to any other branch that exists in the project. Click on the “Commit new file” button to set up the workflow file.

It will create a maven.yml file as shown below in the project.

Step 5 – Verify that the workflow is running

Next, head over to the “Actions” tab, and you will see your YAML workflow file present under the tab. The yellow sign represents that the job is in the queue.

In Progress – When the job starts building and running, you will see the status change from “Queued” to “in progress”.

Passed – If the build is successful, you will see a green tick mark.

Below is the execution log.

Click on the workflow and the below screen is displayed. It shows the status of the run of the workflow, the total time taken to run the workflow, and the name of the .yml file.

Below shows all the steps of the workflow.

Step 6 – Published TestNG Report on GitHub

From the logs of the Workflow, you can see that the Test Report step was executed successfully.

Once the pipeline run, a TestNG Report folder will be generated as shown in the below image:

When we click on the folder TestNG Report, a zipped file will be downloaded, and we can extract it to see all the files contained within it.

Below is the image of emailable-report.html.

Below is the image of index.html.

The complete code can be found here on GitHub – vibssingh/RestAssured_TestNG_Demo.

Congratulations! We just created our CI workflow for running our Rest API test cases.

December 25, 2023March 4, 2024 vibssingh

How to run Serenity tests with GitHub Actions

Last Updated On

March 4, 2024

HOME

This tutorial explains the steps to create a GitHub Action for the Serenity tests and execute the tests in that workflow.

Table of Contents

Why GitHub?

The flexible aspects of Selenium WebDrivers and GitHub Actions enable users to create powerful, fast, and efficient automated testing workflows in CI/CD environments.

CI/CD pipelines have contributed to the success of the DevOps cycle in all software development projects. This is a holistic process that bridges development and operations. Continuous integration helps development teams deploy code efficiently, and continuous delivery automates code deployment.

Important points

1. The Serenity Web tests need to run in the headless mode. As we are using Chrome browser, use the below code in the serenity.config:

          headless.mode = true

2. Install Chrome browser in ubuntu. Use the below code:

    - uses: browser-actions/setup-chrome@latest
    - run: chrome --version

Implementation Steps

Step 1 – Create GitHub Actions and Workflows

I have a repository available in GitHub – Serenity_Cucumber-JUnit5 as shown in the below image. Go to the “Actions” tab. Click on the “Actions” tab.

Step 2 – Select the type of Actions

Step 3 – Generation of Sample pipeline

If you choose an existing option, it will automatically generate a .yaml for the project as shown below.

We will replace the current workflow with the following yml file as shown below:

name: Serenity Tests in GitHub
 
on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]
 
jobs:
  build:
 
    runs-on: ubuntu-latest
 
    steps:
    - uses: actions/checkout@v4
    - name: Set up JDK 17
      uses: actions/setup-java@v4
      with:
        java-version: '17'
        distribution: 'temurin'
        cache: maven

    - uses: browser-actions/setup-chrome@latest
    - run: chrome --version
    
    - name: Build with Maven
      run: mvn clean verify
       
    - name: Test Report Generation
      uses: actions/upload-artifact@v4
      if: success() || failure()
      with:
          name: Serenity Report                 # Name of the folder
          path: target/site/serenity/           # Path to test results

Step 4 – Commit the changes

After the changes, hit the “Start Commit” button.

This will give the option to add a description for the commit. It will also enable the user to commit either to the main branch or commit to any other branch that exists in the project. Click on the “Commit new file” button to set up the workflow file.

It will create a maven.yml file as shown below in the project.

Step 5 – Verify that the workflow is running

Next, head over to the “Actions” tab, and you will see your YAML workflow file present under the tab. The yellow sign represents that the job is in the queue.

In Progress – When the job starts building and running, you will see the status change from “Queued” to “in progress”.

Passed – If the build is successful, you will see a green tick mark.

Click on the workflow and the below screen is displayed. It shows the status of the run of the workflow, the total time taken to run the workflow, and the name of the .yml file.

Below shows all the steps of the workflow.

Step 6 – Published Serenity Report on GitHub

From the logs of the Workflow, you can see that the Test Report step was executed successfully.

Once the pipeline run, a Serenity Report folder will be generated as shown in the below image:

When we click on the folder Serenity Report, a zipped file will be downloaded, and we can extract it to see all the files contained within it.

Open the folder and we will see all the files including index.html and others.

Open the index.html and we will see the report.

To know more about Chrome installation, please refer to this tutorial – browser-actions/setup-chrome.

The complete code can be found here on GitHub – vibssingh/Serenity_Cucumber_JUnit5.

Congratulations! We just created our CI workflow for running our Serenity test cases.

December 8, 2023January 6, 2025 vibssingh

CI/CD Interview Questions and Answers 2025

Last Updated On

January 6, 2025

HOME

1. What is CI/CD Pipeline?

CI/CD stands for Continuous Integration and Continuous Delivery (or Continuous Deployment), and it refers to a set of software development practices aimed at improving the efficiency, reliability, and speed of the software development and release process.

2. What’s the difference between continuous integration, continuous delivery, and continuous deployment?

Continuous Integration (CI) is a development practice wherein developers frequently merge their code modifications into a central repository. This integration occurs multiple times throughout the day and is validated through automated tests and a build procedure. This approach helps in mitigating integration issues, identifying and rectifying bugs early in the development phase, and iteratively testing and refining the code.

Continuous Delivery (CD) involves the automatic deployment of all code modifications to test and/or production environments following the completion of the build process. Continuous Delivery ensures that deployments are both predictable and consistently scheduled.

Continuous Deployment (CD): The most critical stage of the pipeline is continuous deployment. By following this practice, you will be able to release all changes that have passed all stages of the production pipeline to your customers on time. Code changes can be made live much more quickly at this stage because there is little human interaction.

3. What are the benefits of the CI/CD Pipeline?

1. CI encourages frequent integration of code changes, promoting collaboration among team members. Developers can work on their features independently, knowing that automated processes will catch integration issues.

2. CI/CD automates the software delivery process, reducing manual interventions and streamlining workflows. This results in quicker development cycles and faster releases to production.

3. Automated testing in the CI phase helps identify bugs, errors, and integration issues early in the development cycle. This allows for prompt resolution and prevents the accumulation of issues that could be more challenging to address later.

4. CI improves transparency by detecting early-stage failures like build failures, merge issues, integration test failures, etc.

5. Automation in the CI/CD pipeline reduces the need for manual steps, minimizing the risk of human error and freeing up developers and operators to focus on more strategic and creative tasks. This also increases the quality of the code.

4. What are the popular CI/CD Tools?

There are several popular CI/CD (Continuous Integration/Continuous Delivery) tools available that help automate various stages of the software development lifecycle. Here are some widely used CI/CD tools:

1. Jenkins is an open-source automation server that supports building, deploying, and automating any project. It has a large and active community, extensive plugin support, and is highly customizable.

2. Travis CI is a cloud-based CI service that integrates with GitHub repositories. It’s known for its simplicity and ease of use, and it’s commonly used for open-source projects.

3. GitLab provides built-in CI/CD capabilities as part of its platform. It allows users to define CI/CD pipelines using a .gitlab-ci.yml file in the repository. GitLab CI/CD is tightly integrated with GitLab’s version control features.

4. CircleCI is a cloud-based CI/CD platform that supports the automation of the software development process. It integrates with popular version control systems, such as GitHub and Bitbucket.

5. GitHub Actions is a CI/CD and automation service provided by GitHub. It allows developers to define workflows using YAML files within the repository. GitHub Actions seamlessly integrates with GitHub repositories.

6. Atlassian Bamboo is a CI/CD server that integrates with other Atlassian tools like Jira and Bitbucket. It supports building, testing, and deploying applications.
TeamCity:

7. TeamCity, developed by JetBrains, is a CI/CD server with support for various build and deployment scenarios. It has a user-friendly interface and integrates with popular version control systems.

8. Azure DevOps, previously known as Visual Studio Team Services (VSTS), provides a set of development tools, including CI/CD capabilities. It integrates with Microsoft’s Azure cloud services.

5. What are “Artifacts” in the context of CI/CD, and why are they important?

Artifacts are the output of a CI/CD pipeline, typically binary or deployable files such as executables, libraries, Docker images, or Reports.

1. Storing artifacts in dedicated repositories enables easy access and sharing among team members involved in the CI/CD pipeline.

2. Artifacts ensure that deployments are consistent and reproducible across different environments.

3. Artifacts provide a way to track and manage different versions of your software.

6. What are the major phases of the CI/CD pipeline?

There are four phases of the CI/CD pipeline. These are as follows.

1. Source – The first phase in a CI/CD pipeline is the creation of source code, where developers translate requirements into functional algorithms, behaviors, and features.

2. Build – The build process draws source code from a repository, establishes links to relevant libraries, modules, and dependencies, and compiles (builds) all these components into an executable (.exe) file.

3. Test – The source code has already completed some static testing, and the completed build now enters the next CI/CD phase of comprehensive dynamic testing. This usually starts with basic functional or unit testing to verify that new features and functions work as intended, and regression testing to ensure that new changes or additions do not accidentally break any previously working features.

4. Deploy – The deployment step typically involves creating a deployment environment — for example, provisioning resources and services within the data center — and moving the build to its deployment target, such as a server.

7. How do you trigger a CI/CD pipeline?

A CI/CD pipeline can be triggered in several ways. It can be activated by Code Push, scheduled trigger, manual initiation, and Continuous Integration.

8. What is version control?

Version control involves the use of a central repository where teammates can commit changes to files and sets of files. The purpose of version control is to track every line of code, and to share, review, and synchronize changes between team members. The following are some of the most popular version control tools:

Mercurial
Subversion (SVN)
Concurrent Version System (CVS)
Perforce
Bazaar
Bitkeeper
Fossil

9. What is git?

Git is a mature, actively maintained open-source revision control system used by thousands of developers around the world.
It enables developers to track changes in their source code during software development.

10. What is git repository?

A Git repository is a storage where all the code alterations and version histories for a particular project are kept. A Git repository is the .git/ folder inside a project. This repository tracks all changes made to files in your project, building a history over time. Meaning, if you delete the .git/ folder, then you delete your project’s history.

11. What is git branch?

In Git, a branch is a parallel line of development that represents an independent line of work within a Git repository. Branches let developers code without affecting the work of other team members.

12. What is merging?

Merging in Git refers to the process of integrating changes from one branch into another. When developers work on separate branches to implement features, bug fixes, or other changes, merging is used to combine those changes back into a common branch, often the main or default branch.

13. What is test coverage?

Test coverage is a metric that measures how much of the codebase is covered by tests. A 100% coverage means that every line of the code is tested at least by one test case.

14. What is the role of automation in CI/CD?

1. Automation is used to trigger and execute builds automatically whenever changes are made to the version control repository.

2. It also ensures process uniformity and lowers the possibility of human error.

3. Automation is used to set up and configure monitoring and observability tools. Automated monitoring provides insights into the health and performance of applications in real-time.

15. How to write CI/CD pipeline in Jenkins?

To write a CI/CD pipeline in Jenkins, Install Jenkins and required plugins, Set up version control such as Git, Create a Jenkins file with build, test, and deploy stages, and Save Jenkinsfile to the repository. Then, Create a Jenkins job linked to the Jenkinsfile and trigger the job for the automated CI/CD process.

16. What is a Jenkins file?

A Jenkins file is a text file that contains a pipeline’s configuration. It outlines the pipeline’s stages, such as build, test, and deployment, as well as the actions that must be taken at each one.

Example of a Jenkins File:

pipeline {
    agent any
 
    stages {
        stage('Test') {
            steps {
                bat "mvn -D clean test"
            }
 
            post {                
                // If Maven was able to run the tests, even if some of the test
                // failed, record the test results and archive the jar file.
                success {
                   publishHTML([
                       allowMissing: false, 
                       alwaysLinkToLastBuild: false, 
                       keepAll: false, 
                       reportDir: 'target/surefire-reports/', 
                       reportFiles: 'emailable-report.html', 
                       reportName: 'HTML Report', 
                       reportTitles: '', 
                       useWrapperFileDirectly: true])
                }
            }
        }
    }
}

17. What is docker?

Docker is an open platform for developing, shipping, and running applications using containers. Containers allow a developer to package up an application with all the parts it needs, such as libraries and other dependencies, and ship it all out as one package. This ensures that the application runs consistently across different environments.
A Dockerfile is a text file that contains instructions for building a Docker image. It specifies the base image, the application code, dependencies, and other configuration settings needed to create the image.

18. What is a container?

A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another.

19. What is Kubernetes?

Kubernetes, often abbreviated as K8s, is an open-source container orchestration platform designed to automate the deployment, scaling, and management of containerized applications. Kubernetes can automatically scale the number of containers based on demand. It can also distribute incoming network traffic across multiple containers to ensure optimal performance and availability.

1. Service discovery and load balancing Kubernetes can expose a container using the DNS name or using their own IP address. If traffic to a container is high, Kubernetes can load balance and distribute the network traffic so that the deployment is stable.

2. Kubernetes supports rolling updates, allowing applications to be updated with minimal downtime. If an update causes issues, Kubernetes supports easy rollbacks to a previous version.

3. Kubernetes restarts containers that fail, replaces containers, kills containers that don’t respond to your user-defined health check, and doesn’t advertise them to clients until they are ready to serve.

4. Kubernetes lets you store and manage sensitive information, such as passwords, OAuth tokens, and SSH keys. You can deploy and update secrets and application configuration without rebuilding your container images, and without exposing secrets in your stack configuration.

20. What is blue-green deployment?

Blue-green deployment is a deployment approach in which new versions of an application are deployed and tested in two identical environments, blue and green.
The old version can be called the blue environment while the new version can be known as the green environment. Once production traffic is fully transferred from blue to green, blue can standby in case of rollback or pulled from production and updated to become the template upon which the next update is made.

21. What is canary deployment?

Canary deployment is a method of application deployment in which just a small percentage of production traffic is sent to a new version of the application, with the majority of traffic still being handled by the older version. As a result, the new version can be tested and validated before being implemented across the board in the production environment.

22. What is observability?

Observability is the ability to understand and diagnose the behavior of a complex system, such as a software application, using real-time monitoring and metrics.

23. What is Grafana?

Grafana is an open-source analytics and monitoring platform that integrates with various data sources to help users visualize and understand their data. It is widely used for monitoring and observability in IT systems, providing a flexible and customizable interface for creating dashboards, charts, and graphs.
Users can create dashboards that display real-time or historical data through the use of panels and visualizations.
Grafana allows users to set up alert rules based on defined thresholds or conditions. When an alert is triggered, Grafana can send notifications via various channels, such as email, Slack, or other supported notification services.
Grafana supports user authentication and authorization. Users can be organized into teams with specific access permissions, ensuring secure access control to dashboards and data sources.

24. What is Prometheus?

Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud.
Prometheus employs a pull-based model, where it regularly scrapes (pulls) metrics from endpoints exposed by the systems being monitored. This model is well-suited for dynamic and containerized environments.
Prometheus Query Language (PromQL) is a powerful query language that allows users to retrieve and analyze metrics data. It supports various aggregation functions, filtering, and mathematical operations.
Prometheus includes a built-in alerting system that allows users to define alert rules based on metric thresholds or conditions. When an alert is triggered, Prometheus can send notifications to external systems.

25. How do you ensure the security of the CI/CD pipeline?

1. To prevent unauthorized access to the pipeline and its components, access controls and authentication procedures must be put in place.

2. Finding security flaws by checking code and dependencies for vulnerabilities and using static and dynamic analysis methods.

3. Avoid storing sensitive information, such as API keys and credentials, directly in CI/CD configuration files or scripts. Use secure vaults or secret management tools to store and retrieve secrets securely. Ensure that secrets are not exposed in logs.

4. Implementing a safe build process that includes using trustworthy sources for dependencies and signing builds to confirm their legitimacy.

5. Regularly scan dependencies and libraries for known vulnerabilities. Utilize tools that can identify and report on security vulnerabilities in third-party dependencies. Update dependencies to patched versions to mitigate potential risk.

26. What are some challenges in implementing CI/CD?

1. Integrating CI/CD practices into existing legacy systems can be challenging.

2. Managing dependencies, especially in large projects, can be complex. Dependency management issues may arise from compatibility problems, version conflicts, or unavailability of required dependencies.

3. Security is a critical aspect of software development, and integrating security into CI/CD pipelines is a challenge. Ensuring secure coding practices, vulnerability scanning, and compliance checks requires careful consideration.

4. Integrating CI/CD pipelines with external tools, services, or platforms can be challenging. Ensuring compatibility and smooth interactions with tools like version control systems, issue trackers, and cloud services is essential.

27. Can you explain the concept of “Infrastructure as Code” (IaC) and provide an example of an IaC tool?

Infrastructure as Code (IaC) is the practice of managing and provisioning infrastructure using code and automation. Instead of manually configuring servers, networks, and other infrastructure components, IaC allows developers to define and manage their infrastructure through code. One popular example of an IaC tool is Terraform which uses a domain-specific language (HCL — HashiCorp Configuration Language) to describe infrastructure resources and their relationships. With Terraform, you can create, update, and manage infrastructure as code, making it easier to version control and maintain infrastructure configurations.

28. Is CI/CD pipeline can be targeted by attackers?

Yes, CI/CD pipeline can be targeted by the attackers. CI/CD pipelines have access to various sensor data, such as server passwords, repositories, API keys, and so on. If the CI/CD system is not properly secured, then it can be targeted by attackers. A few ways of securing the CI/CD systems are as follows.

Finding security flaws by checking code and dependencies for vulnerabilities and using static and dynamic analysis methods.

To prevent unauthorized access to the pipeline and its components, access controls and authentication procedures must be put in place.

29. What is the meaning of trunk-based development in CI/CD?

Trunk-based development is a software development approach that emphasizes frequent integration of code changes into a shared main branch, often referred to as the “trunk” or “master” branch. In the context of CI/CD, trunk-based development promotes continuous integration by encouraging developers to merge their changes into the main branch multiple times throughout the day.

30. Describe the role of container orchestration tools in CI/CD.

Container orchestration tools like Kubernetes automate the deployment, scaling, and management of containerized applications in CI/CD pipelines.

31. What is chaos engineering?

Chaos engineering is the practice of intentionally introducing failures and chaos into a system to test its resilience and reliability.

32. What is a microservices architecture?

A microservices architecture divides an application into smaller, independent services that can be developed, deployed, and scaled separately.

33. What is a canary analysis?

Canary analysis involves releasing a small subset of changes to a limited audience to assess their impact before a full deployment.

October 21, 2023September 27, 2024 vibssingh

How To Publish ExtentReport Using Jenkins

Last Updated On

September 27, 2024

HOME

In the previous tutorial, we have seen the Integration of Allure Report with Jenkins. In this tutorial, we show you how to generate Extent Report Using Jenkins.

Table of Contents

Prerequisite

Jenkin’s installed and started on the computer. The current Jenkins version is – 2.361.2

To generate HTML Report in Jenkins, we need to download HTML Publisher Plugin. Please refer to this tutorial to install the plugin – How to install Plugins in Jenkins.

Implementation Steps

Step 1: Create a new Maven project

Give the Name of the project – ExtentReport_Demo
Click on the Maven project.
Click on the OK button.

In the General section, enter the project description in the Description box.

Select Source Code Management as None if the project is locally present on the machine.

Step 2: Build Management

Go to the Build section of the new job.

In the Root POM textbox, enter the full path to pom.xml
In the Goals and options section, enter “clean test site”

Here, I have used the Selenium project with JUnit, so to see the complete project, please refer to this tutorial – How to generate JUnit4 Report.

Click on the Advanced button.

Step 3: Select a custom workspace

Mention the full path of the project in the directory.

Step 4: Select “Publish HTML reports” from “Post Build Actions”

Scroll down to “Post Build Actions” and click on the “Add Post Build Actions” drop-down list. Select “Publish HTML reports“.

If you want to see where the report is saved in Jenkins, go to the Dashboard ->ExtentReport_Demo project -> Workspace ->target -> Reports -> Spark.html.

Enter the HTML directory to archive – Reports, Index page[s] – Spark.html, and Report title – Extent Report.

Click on the Apply and Save buttons.

We have created a new Maven project “ExtentReport_Demo” with the configuration to run the Cucumber, and Selenium with TestNG Tests and also to generate HTML Report after execution using Jenkins.

Step 5: Execute the tests

Let’s execute it now by clicking on the “Build Now” button.

Right-click on Build Number (here in my case it is #4).

Click on Console Output to see the result.

Step 6: View the Extent Report

Once the execution is completed, click on go “Back to Project“, and we can see a link to view the “Extent Report“.

We can see here that the Extent Report link is displayed in the Console.

Below is the Extent Report generated in Jenkins.

Tip: If you don’t see the Report UI intact, then you need to configure a simple Groovy script. For that, go to Dashboard–>Manage Jenkins–>Script Console and add the script as:

System.setProperty("hudson.model.DirectoryBrowserSupport.CSP","")

Congratulations on making it through this tutorial and hope you found it useful! Happy Learning!! Cheers!!

Additional Tutorials

Serenity with Jenkins

Integration of Allure Report with Jenkins

How to generate HTML Reports in Jenkins

How to create Jenkins pipeline for Serenity tests

How to create Jenkins pipeline for Cucumber tests

October 21, 2023November 6, 2024 vibssingh

How to create JUnit Report in Jenkins

Last Updated On

November 6, 2024

HOME

In this tutorial, we show you how to generate JUnit Report Using Jenkins. In the previous tutorial, we generated a TestNG Report using Jenkins.

Table of Contents

Prerequisite

1. Jenkins installed and started on the computer – How to install Jenkins on Windows 10.

2. Java and Maven are installed and configured in Jenkins. To know more about this, please refer to this – How to configure Java and Maven in Jenkins.

3. To generate JUnit Report, please refer to this tutorial to get the code – How to generate JUnit4 Report.

4. To generate a JUnit Report in Jenkins, we need to download the JUnit Plugin. Please refer to this tutorial to install the plugin – How to install Plugins in Jenkins

Implementation Steps

Step 1: Start the Jenkins server

Start the Jenkins server open the browser and navigate to the below endpoint

http://localhost:8080/

Step 2: Create a new FreeStyle project

Give the Name of the project – JUnitReport_Demo
Click on the Freestyle project.
Click on the OK

In the General section, enter the project description in the Description box.

Select a custom workspace and provide the full path of the project.

Select Source Code Management as None if the project is locally present on the machine

Step 3: Build Steps

In the Build Steps section, select Invoke top-level Maven targets.

The Build Steps window will extend. Mention the below details:-

Maven Version – MAVEN_HOME

Goals – clean test

Click on the Advanced button.

Step 4: Provide the full path to pom.xml

Specify the full path to pom.xml in POM.

Step 5: Select “Publish JUnit test result report” from “Post Build Actions”

Scroll down to “Post Build Actions” and click on the “Add Post Build Actions” drop-down list. Select “Publish JUnit test result report“.

Enter the Result Path as “**/target/surefire-reports/*.xml”.

Click on the Apply and Save buttons.

We have created a new Maven project “JUnitReport_Demo” with the configuration to run the Selenium with JUnit Tests and also to generate JUnit Report after execution using Jenkins.