NUnit

AIO Tests allows easy reporting of NUnit results by supporting the import of the NUnit results xml.

With AIO Tests NUnit integration, developers and testers can report their execution results to existing AIO Tests or directly create testcases which do not yet exist in AIO Tests.

This article discusses how to import NUnit XML results and how to map existing cases or create new ones from NUnit results.

NUnit

NUnit is an open source unit-testing framework for all .Net languages. It has strong support for parallel runs and data driven tests, with a rich set of assertions to support all kinds of verifications. NUnit uses a rich set of custom attributes to identify tests. Attributes also help to achieve flexible setup and teardown for the tests. All NUnit attributes are contained in the NUnit.Framework namespace.
NUnit tests can be run via command line or via GUI. NUnit results are captured in a xml file, which can provide information on the status, duration and failures of the tests.

NUnit Reporting

NUnit 3.0 report structure can be found @ https://docs.nunit.org/articles/nunit/technical-notes/usage/Test-Result-XML-Format.html. AIO Tests supports the latest 3.0 version of the report and does not support any previous versions.

Sample NUnit XML Report

Below is a sample NUnit 3.0 XML report with just one testcase, which gives an idea of the structure of ther report. It captures a failing test, along with the failure and it’s stacktrace.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 <test-run id="2" duration="0.038897" testcasecount="1" total="1" passed="0" failed="1" inconclusive="0" skipped="0" result="Failed" start-time="2022-06-13T 12:10:58Z" end-time="2022-06-13T 12:10:59Z"> <test-suite type="Assembly" name="nunit-aiotests-demo.dll" fullname="xxx" total="1" passed="0" failed="1" inconclusive="0" skipped="0" result="Failed" start-time="2022-06-13T 12:10:58Z" end-time="2022-06-13T 12:10:58Z" duration="0.038897"> <test-suite type="TestSuite" name="Money" fullname="Money" total="1" passed="0" failed="1" inconclusive="0" skipped="0" result="Failed" start-time="2022-06-13T 12:10:58Z" end-time="2022-06-13T 12:10:58Z" duration="0.038897"> <test-suite type="TestFixture" name="MoneyTest" fullname="Money.MoneyTest" classname="Money.MoneyTest" total="1" passed="0" failed="1" inconclusive="0" skipped="0" result="Failed" start-time="2022-06-13T 12:10:58Z" end-time="2022-06-13T 12:10:58Z" duration="0.038897"> <test-case name="BagMultiply" fullname="Money.MoneyTest.BagMultiply" methodname="BagMultiply" classname="MoneyTest" result="Failed" start-time="2022-06-13T 12:10:58Z" end-time="2022-06-13T 12:10:58Z" duration="0.038897" asserts="0" seed="1998700055"> <properties> <property name="Category" value="SingleTest" /> </properties> <failure> <message> Expected: &lt;{[24 CHF][15 USD]}&gt; But was: &lt;{[24 CHF][14 USD]}&gt; </message> <stack-trace> at Money.MoneyTest.BagMultiply() in /Users/varshneyn/aiotcms/automation-demo-repos/nunit-aiotests-demo/nunit-aiotests-demo/MoneyTest.cs:line 47 </stack-trace> </failure> </test-case> </test-suite> </test-suite> <errors /> </test-suite> </test-run>

Status Mapping NUnit → AIO Tests

NUnit XML result value in test-case

Description

AIO Tests Mapping

NUnit XML result value in test-case

Description

AIO Tests Mapping

Passed

Passed case

Passed

Failed

Failed

Not Run

Inconclusive

Indicates the assumption in a Theory failed or an Assert.Inconclusive was used

Skipped

Skipped

Indicates either the test was ignored or marked as explicitly skipped

Skipped

Mapping automated NUnit tests to AIO Tests

In going with the simplicity advantage of AIO Tests, the NUnit integration has been designed as a simple and non-intrusive integration, without any extra dependency or coding required, utilising the core NUnit features of Attributes.

There are 3 ways that a NUnit case is mapped to an AIO Case.

  1. NUnit Property Attribute: If a case has been created in AIO Tests, The Property attribute can be used with AIO property name : aioCaseKey

    1 2 3 4 5 6 7 [Test] [Property("aiocasekey", "RAM-TC-1")] [Category("SimpleTests")] public void PassingTest() { Assert.Pass(); }

     

  2. Testcase key in Underscores in test name : Since many languages do not allow hyphens in their method signatures, AIO Tests allows case keys with underscores as valid mapping. The AIO Test key can be used in the method name by replacing the hyphen with underscores. eg. if AT-TC-123 is being automated by a test, the test method can be named as verifyNotNullType_AT_TC_123 and the results of this method will be marked against AT-TC-123.

  3. Automation Key: The mapping of an AIO Case to an automated NUnit case can happen through a field Automation Key, which can be specified in the Case in AIO Tests app as shown below. The automation key is the fully qualified name of the test method i.e fullname from the results xml.
    This can be useful at places where you do not want to add AIO case keys in the name or annotations or where automation is happening first without a manual case being created in AIO Tests.

Examples

Below are few examples of how to achieve mapping in various cases:

1. Normal standalone case - Use property aioCaseKey

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 [Test] [Property("aiocasekey", "RAM-TC-1")] [Category("SimpleTests")] public void PassingTest() { Assert.Pass(); } [Test] [Property("aiocasekey", "SCRUM-TC-421")] //cross-project case [Category("SimpleTests")] public void Test_Failed_Retry() { Assert.Fail("I failed coz I wanted to fail"); }

2. Normal standalone case - Use case key in method name

The below case will update the status of SCRUM-TC-22 in AIO Tests as passed.

1 2 3 4 5 6 [Test] [Category("LongRun")] public void MyTest_SCRUM_TC_22() { Assert.Pass(); }

3. Data driven cases [ using TestCaseSource attribute]

Below example shows how data driven cases can be mapped to different cases in AIO Tests.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 [TestFixture] [Category("DDTests")] public class DataDrivenTests { [TestCaseSource(typeof(MyDataClass), nameof(MyDataClass.TestCases))] public int DivideTest(int n, int d, String tcId) { return n / d; } } public class MyDataClass { public static IEnumerable TestCases { get { yield return new TestCaseData(12, 3,"SCRUM-TC-15").Returns(4); yield return new TestCaseData(12, 2,"SCRUM-TC-423").Returns(6); yield return new TestCaseData(12, 4,"SCRUM-TC-1615").Returns(3); } } }

4. Data driven cases [ using TestCase attribute]

Below example shows how data driven cases can be mapped to different cases in AIO Tests with TestCase attribute

1 2 3 4 5 6 7 8 9 10 11 [TestFixture] [Category("DDTests")] public class DataDrivenTests { [TestCase(12, 3, 4, "SCRUM-TC-13194")] [TestCase(12, 2, 1, "SCRUM-TC-13195")] [TestCase(12, 4, 3, "SCRUM-TC-13196")] public void DivideTest(int n, int d, int q, String tcId) { Assert.AreEqual(q, n / d); } }

If Case key is not provided at TestCase level and instead, is provided as Property aioCaseKey, then depending upon user’s choice at time of import, either one run would be created per data set or the same run would be updated thrice based on status and the last status update would remain.

5. Mapping case to multiple cases

Below example shows how to map a case if an automated case covers multiple cases

1 2 3 4 5 6 7 8 9 10 11 [Test] [Category("Multitest")] [Property("aiocasekey", "RAM-TC-2,RAM-TC-3")] public void TestEquality() { String eq1 = "AIO"; String eq2 = "aio"; Assert.AreEqual(eq1, eq2); if (eq1 != null && eq2 != null) Assert.AreEqual(eq1.GetHashCode(), eq2.GetHashCode()); }

Importing Results

Post execution of a NUnit suite, the xml file can be uploaded either via

Please follow the above links to continue to import results using either of the options.