Pykiso Reporting

JUnit XML report

Generating a JUnit XML report

For CI integration, JUnit reports in XML format can be generated at the end of the test session. Simply invoke pykiso with:

pykiso -c my_config.yaml --junit=./path/to/my_report.xml

The resulting report will contain:

One testsuite for each pykiso.BasicTest, pykiso.BasicTestSuiteSetup, pykiso.BasicTestSuiteTeardown.
One testcase inside the testsuite for each test method. Using subTest will also append a testcase with modified test case name "test_method [my subtest description]".

Note

The captured logs will always appear in the testcase section.

Adding contextual information to the JUnit report

Any user-defined information can be added to all test cases of a test suite as JUnit properties.

In order to add properties, simply define an attribute named properties as a dictionary inside your test case:

@pykiso.define_test_parameters(suite_id=1, case_id=1)
class MyTest(pykiso.BasicTest)

    def test_run(self):
        remote_dut_software_version = "42.0.0"
        self.properties = {
            "property_name": 42,
            "Software Version": remote_dut_software_version
        }

One useful use case for test case properties is the Upload of Results to TestRail.

HTML system-test (step) report

The step report aims to provide a more comprehensive test-report (adapted for system testers) by tracking each assertion that contains a message. It follows the following structure:

test name
test description
date of execution
elapsed time
information gathered during test
assertion detail: - value of the data_in - variable name of the data_in - expected value - message
the report is presented as an HTML page

Usage Examples

def setUp(self):
    # Data to test
    device_on = True
    voltage = 3.8

    # Simple assert
    self.assertTrue(device_on, msg="Check my device is ready")

    with self.subTest("Non critical checks"):
        # This check will fail but the test continues
        self.assertFalse(device_on, msg="Some check")

    # Add a new table in the report
    self.step_report.current_table = "Another table"

    # Assert with custom message
    # Assert msg overwritten when step_report_message not null
    self.step_report.message = "Custom message"
    self.assertAlmostEqual(voltage, 4, delta=1, msg="Check voltage device")

    # Additional data to include in the step-report
    self.step_report.header["Version_device"] = "2022-1234"

self.step_report.header allows you to add additional data to the report header during the test execution.

How to generate

pykiso -c my_config.yaml --step-report=./path/to/my_report.html

Limitations

The step report generator might fail if you put parentheses or “assert” strings in strings or comments in the assert statement.

HTML result example