Jenkins - Continuous Integration

In recent years, Continuous Integration (CI) has become a best practice for software development. Build automation and automated testing form the core of CI.

AIO Tests supports automated testing with AIO public APIs and integration with frameworks like JUnit and TestNG.

The AIO Tests Jenkins plugin provides another easy to use interface to report test results right after the automated tests have run, thus helping in CI. The plugin wraps the AIO public APIs to import test results from TestNG or JUnit result XML files.

AIO APIs and Jenkins plugin can be used to publish results from any framework capable of generating JUnit files like Robot/Cypress/NUnit amongst many others.

Installing the plugin

AIO Tests Jenkins plugin can be installed from the Jenkins marketplace as below.

  1. Login to your Jenkins instance.

  2. Go to Manage Jenkins > Manage Plugins > Available

  3. Now search for AIO Tests and click Install

Requirements

  1. Jenkins 2.235.1 or higher

  2. Jira cloud instance with AIO Tests for Jira installed

Reporting results

The plugin works in the post build actions step, after the build steps are run.
Assuming that the tests have run and a results xml (Junit/TestNG/any XUnit framework which generates a JUnit xml report) has been generated, the AIO Tests plugin can be used as below to report results of cases in the report XML to Jira.

  1. Go to project -> Configure

  2. Click on the Post Build Actions -> Select Publish Results to AIO Tests

     

  3. The following options need to be configured for publishing results to AIO Tests

     

  4. Specify the Project for which run results need to be published eg. SCRUM

  5. Choose a framework and specify the path of the file - relative to the workspace eg. /target/surefire-reports/testng-results.xml

    For frameworks other than JUnit or TestNG, if results file is being generated in JUnit format, please select JUnit

  6. Specify the API Key generated from AIO Tests in Jira. Please refer Generating API Key for more information.

  7. The below table defines the rest of the field options

Field

Details

Field

Details

1

Cycle Preference

Select between existing and new cycle

2

Cycle Prefix

In case of new cycle, Cycle prefix will be used to prefix the cycle title

3

Cycle Key

If existing cycle is selected, specify AIO Tests cycle key eg. SCRUM-CY-121

4

Add Case

If checked, cases not already in cycle are added, else the cases are reported as errors

5

Create Case

If checked and no case exists with a matching case key or automation key, a new case is created

6

Hide Publish Result Details

If checked, it would hide testcase wise details

7

Fail build if result publishing fails

If checked, the build result will be updated to failed, in case publishing of results to AIO Tests fails

Reporting results in pipeline

The AIO Tests Jenkins plugin has support for being run as a pipeline task. Below is an example.

For security of API Key, secret text credentials can be used to avoid putting the key in plain text. Plain text API Key would work as well.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 pipeline { agent any environment { AIO_TESTS_API_KEY = credentials('aioTestApiKey') } stages { stage('SCM') { steps { git 'https://github.com/org/sample-testng-tests.git' } } stage('Build') { steps { sh 'mvn test' } } } post { always { aioImport frameworkType : 'TestNG', addCaseToCycle :true, createCase :true, entry: [$class: 'NewCycle', cyclePrefix: 'Regression Run V1.0'], //For existing cycles : entry: [$class: 'ExistingCycle', cycleKey: 'SCRUM-CY-191'], apiKey : hudson.util.Secret.fromString(env.AIO_TESTS_API_KEY), resultsFilePath : '/target/surefire-reports/testng-results.xml', projectKey: 'SCRUM', hideDetails: false } } }

Parameters

If the job is a parameterized build, the plugin supports parametrization for the project key, results file path, cycle prefix, existing cycle key and API key fields.