AIO Tests provides integration with Postman tests by allowing users to import their Newman generated JSON report via REST calls or with a custom newman reporter. Results can be updated for existing AIO Tests Cases by mapping automated cases or by creating new ones from the Newman results json file. 

Having the automated results reporting in AIO Tests, can help in having a holistic view of coverage of requirements (unit to manual to functional and end-to-end automated tests).

This article discusses how are API tests created and results of the tests uploaded to AIO Tests using Newman results.

Postman and Newman

Postman is an API platform for building and using APIs. It is also an API client that makes it easy for developers to design, mock, create, share, test and document APIs. With Postman, you can send requests to connect to APIs you are working with. Requests can retrieve, add, delete, and update data. Requests can send parameters, authorization details, and any body data that is required.

Newman is an open source command-line Collection Runner for Postman. It enables you to run and test a Postman Collection directly from the command line, making it possible to run API tests in continuous integration servers and build systems.

Newman Setup

1. To run Newman, ensure that you have Node.js >= v10. Install Node.js via package manager.

2. npm install -g newman or brew install newman

Execute Postman Collection via Newman

Postman requests and examples can be grouped into collections to keep a workspace organized, to collaborate with teammates, to generate API documentation and API tests, and to automate request runs.

1. Create a Postman Collection. For details on creating Postman collections, please refer to Creating Collections and Adding Requests to Collections.

2. Export the collection (Steps to Export). Exporting creates a collection.json file.

3. Using the newman cli, run the collection
   newman run sample-collection.json

4. Environments provide a set of variables that one can use within collections. If an environment file is required, then export the environment file and use it with newman run as follows:
   newman run mycollection.json -e env.json

Sample collection

Open

Generating Json report

The json reporter comes out of the box with newman. To generate a json report for a newman run, provide --reporters json as a CLI option.

It takes the path to the file, where to write the report --reporter-json-export <path>. If not specified, the file will be written to newman/ in the current working directory.

So, a complete command to run and generate json report will be as below:
newman run mycollection.json -e env.json --reporters json --reporter-json-export <path>

Sample report

The json report is a comprehensive output of the run summary. It contains the details of all requests, their tests, the responses and much more. A sample report is as below:

Open

Integration with AIO Tests

AIO Tests supports Postman API Tests via the BDD/Gherkin format of cases. Using the newman output of runs, AIO Tests allows users to

• either report results against existing cases in AIO Tests

• or create new cases on the fly.

A sample API case in AIO Tests looks like below. The next section shows how Postman data of request, tests and assertions are mapped to an AIO Test.

Open

Field mapping (Postman → AIO Tests)

The below table summarizes how a Postman API Tests gets translated to an AIO Case.

Mapping Postman tests to existing AIO Tests

If cases have been created in AIO Tests, to uniquely map the AIO Case to the API Test, the case key can be added to the name of the API, as shown below:

Open

Creating Postman tests in AIO Tests

If no case key is added to the name of the API, then a new case is created in AIO Tests, with the field mappings as explained in the section : Field mapping (Postman → AIO Tests)

The automation key of this newly created case is set to the name of the API. The subsequent times when the API test is run, the cases are identified by the automation key.

In the above example, the first 2 cases will be reported to AT-TC-223 and AT-TC-224. However, the 3rd and the 4th request will be created as new cases. Below is an example of a newly created case

Reporting results

To report results of a collection run, AIO Tests provides 2 ways.

1. The newman-reporter-aiotests allows user to setup an AIO Tests config in their newman run if running newman as a library or via command line. The reporter gives additional capabilities to create cycles for each run or to reuse existing cycles.

2. Users may choose to import the json report (generated by the inbuilt json reporter), via the Import Result API or through the Cycle Import Results option or through the Jenkins plugin. To read more about these options, please refer to https://aiosupport.atlassian.net/wiki/spaces/NAT/pages/1900879961

newman-reporter-aiotests

AIO Tests has introduced a custom reporter for Newman. The newman-reporter-aiotests simplifies reporting results from Newman with an easy setup.

Install

The reporter works as a plugin with Newman so ensure that you have already installed that package globally, using npm install -g newman.

To globally install the aiotests package:

To use aiotests as a library, install the package as a dependency into a nodejs project's package.json file using:

Usage

In order to enable this reporter, specify aiotests in Newman's -r or --reporters option. The following command would run the reporter:

CLI Options

With Newman CLI

With Newman as a Library

All the CLI functionality is available for programmatic use within a nodejs script.

Install the required node modules using npm i -S newman newman-reporter-aiotests