API Documentation

Test Cases

Generic Test

module:

test_case

synopsis:

Basic extensible implementation of a TestCase, and of a Remote TestCase for Message Protocol / TestApp usage.

Note

TODO later on will inherit from a metaclass to get the id parameters

class pykiso.test_coordinator.test_case.BasicTest(test_suite_id, test_case_id, aux_list, setup_timeout, run_timeout, teardown_timeout, test_ids, tag, *args, **kwargs)[source]

Base for test-cases.

Initialize generic test-case.

Parameters:
  • test_suite_id (int) – test suite identification number

  • test_case_id (int) – test case identification number

  • aux_list (Optional[List[AuxiliaryInterface]]) – list of used auxiliaries

  • setup_timeout (Optional[int]) – maximum time (in seconds) used to wait for a report during setup execution

  • run_timeout (Optional[int]) – maximum time (in seconds) used to wait for a report during test_run execution

  • teardown_timeout (Optional[int]) – the maximum time (in seconds) used to wait for a report during teardown execution

  • test_ids (Optional[dict]) – jama references to get the coverage eg: {“Component1”: [“Req1”, “Req2”], “Component2”: [“Req3”]}

  • tag (Optional[Dict[str, List[str]]]) – dictionary allowing users to filter the tests based on the keys and their value.

cleanup_and_skip(aux, info_to_print)[source]

Cleanup auxiliary and log reasons.

Parameters:
  • aux (AuxiliaryInterface) – corresponding auxiliary to abort

  • info_to_print (str) – A message you want to print while cleaning up the test

Return type:

None

setUp()[source]

Startup hook method to execute code before each test method.

Return type:

None

tearDown()[source]

Closure hook method to execute code after each test method.

Return type:

None

class pykiso.test_coordinator.test_case.RemoteTest(test_suite_id, test_case_id, aux_list, setup_timeout, run_timeout, teardown_timeout, test_ids, tag, *args, **kwargs)[source]

Base test-cases for Message Protocol / TestApp usage.

Initialize TestApp test-case.

Parameters:
  • test_suite_id (int) – test suite identification number

  • test_case_id (int) – test case identification number

  • aux_list (Optional[List[AuxiliaryInterface]]) – list of used auxiliaries

  • setup_timeout (Optional[int]) – maximum time (in seconds) used to wait for a report during setup execution

  • run_timeout (Optional[int]) – maximum time (in seconds) used to wait for a report during test_run execution

  • teardown_timeout (Optional[int]) – the maximum time (in seconds) used to wait for a report during teardown execution

  • test_ids (Optional[dict]) – jama references to get the coverage eg: {“Component1”: [“Req1”, “Req2”], “Component2”: [“Req3”]}

  • tag (Optional[Dict[str, List[str]]]) – dictionary containing lists of variants and/or test levels when only a subset of tests needs to be executed

setUp()[source]

Startup hook method to execute code before each test method.

Return type:

None

tearDown()[source]

Closure hook method to execute code after each test method.

Return type:

None

test_run()[source]

Hook method from unittest in order to execute test case.

Return type:

None

pykiso.test_coordinator.test_case.define_test_parameters(suite_id=0, case_id=0, aux_list=None, setup_timeout=None, run_timeout=None, teardown_timeout=None, test_ids=None, tag=None)[source]

Decorator to fill out test parameters of the BasicTest and RemoteTest automatically.

pykiso.test_coordinator.test_case.retry_test_case(max_try=2, rerun_setup=False, rerun_teardown=False, stability_test=False)[source]

Decorator: retry mechanism for testCase.

The aim is to cover the 2 following cases:

  • Unstable test : get the test pass within the {max_try} attempt

  • Stability test : run {max_try} time the test expecting no error

The retry_test_case comes with the possibility to re-run the setUp and tearDown methods automatically.

Parameters:
  • max_try (int) – maximum number of try to get the test pass.

  • rerun_setup (bool) – call the “setUp” method of the test.

  • rerun_teardown (bool) – call the “tearDown” method of the test.

  • stability_test (bool) – run {max_try} time the test and raise an exception if an error occurs.

Returns:

None, a testCase is not supposed to return anything.

Raises:

Exception – if stability_test, the exception that occurred during the execution; if not stability_test, the exception that occurred at the last try.

Connectors

pykiso comes with some ready to use implementations of different connectors.

Auxiliaries

Message Protocol

pykiso Control Message Protocol

module:

message

synopsis:

Message that will be send though the different agents

class pykiso.message.Message(msg_type=0, sub_type=0, error_code=0, test_suite=0, test_case=0, tlv_dict=None)[source]

A message who fit testApp protocol.

The created message is a tlv style message with the following format: TYPE: msg_type | message_token | sub_type | errorCode |

Create a generic message.

Parameters:
  • msg_type (MessageType) – Message type

  • sub_type (Message<MessageType>Type) – Message sub-type

  • error_code (integer) – Error value

  • test_suite (integer) – Suite value

  • test_case (integer) – Test value

  • tlv_dict (dict) – Dictionary containing tlvs elements in the form {‘type’:’value’, …}

check_if_ack_message_is_matching(ack_message)[source]

Check if the ack message was for this sent message.

Parameters:

ack_message (Message) – received acknowledge message

Return type:

bool

Returns:

True if message type and token are valid otherwise False

generate_ack_message(ack_type)[source]

Generate acknowledgement to send out.

Parameters:

ack_type (int) – ack or nack

Return type:

Optional[Message]

Returns:

filled acknowledge message otherwise None

classmethod get_crc(serialized_msg, crc_byte_size=2)[source]

Get the CRC checksum for a bytes message.

Parameters:
  • serialized_msg (bytes) – message used for the crc calculation

  • crc_byte_size (int) – number of bytes dedicated for the crc

Return type:

int

Returns:

CRC checksum

get_message_sub_type()[source]

Return actual message subtype.

Return type:

int

get_message_tlv_dict()[source]

Return actual message type/length/value dictionary.

Return type:

dict

get_message_token()[source]

Return actual message token.

Return type:

int

get_message_type()[source]

Return actual message type.

Return type:

Union[int, MessageType]

classmethod parse_packet(raw_packet)[source]

Factory function to create a Message object from raw data.

Parameters:

raw_packet (bytes) – array of a received message

Return type:

Message

Returns:

itself

serialize()[source]

Serialize message into raw packet.

Format: | msg_type (1b) | msg_token (1b) | sub_type (1b) | error_code (1b) |
test_section (1b) | test_suite (1b) | test_case (1b) | payload_length (1b) |
tlv_type (1b) | tlv_size (1b) | … | crc_checksum (2b)
Return type:

bytes

Returns:

bytes representing the Message object

class pykiso.message.MessageAckType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]

List of possible received messages.

class pykiso.message.MessageCommandType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]

List of commands allowed.

class pykiso.message.MessageLogType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]

List of possible received log messages.

class pykiso.message.MessageReportType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]

List of possible received messages.

class pykiso.message.MessageType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]

List of messages allowed.

class pykiso.message.TlvKnownTags(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]

List of known / supported tags.

Import Magic

Auxiliary Interface Definition

module:

dynamic_loader

synopsis:

Import magic that enables aliased auxiliary loading in TestCases

class pykiso.test_setup.dynamic_loader.DynamicImportLinker[source]

Public Interface of Import Magic.

initialises the Loaders, Finders and Caches, implements interfaces to install the magic and register the auxiliaries and connectors.

Initialize attributes.

install()[source]

Install the import hooks with the system.

provide_auxiliary(name, module, aux_cons=None, **config_params)[source]

Provide a auxiliary.

Parameters:
  • name (str) – the auxiliary alias

  • module (str) – either ‘python-file-path:Class’ or ‘module:Class’ of the class we want to provide.

  • aux_cons – list of connectors this auxiliary has

provide_connector(name, module, **config_params)[source]

Provide a connector.

Parameters:
  • name (str) – the connector alias

  • module (str) – either ‘python-file-path:Class’ or ‘module:Class’ of the class we want to provide.

uninstall()[source]

Deregister the import hooks, close all running threads, delete all instances.

Config Registry

module:

config_registry

synopsis:

register auxiliaries and connectors to provide them for import.

class pykiso.test_setup.config_registry.ConfigRegistry[source]

Register auxiliaries with connectors to provide systemwide import statements.

Internally patch the user-configuration if multiple auxiliaries share a same communication channel.

classmethod delete_aux_con()[source]

Deregister the import hooks, close all running threads, delete all instances.

Return type:

None

classmethod get_all_auxes()[source]

Return all auxiliaires instances and alias

Return type:

Dict[NewType(AuxiliaryAlias, str), AuxiliaryInterface]

Returns:

dictionary with alias as keys and instances as values

classmethod get_aux_by_alias(alias)[source]

Return the associated auxiliary instance to the given alias.

Parameters:

alias (NewType(AuxiliaryAlias, str)) – auxiliary’s alias

Return type:

AuxiliaryInterface

Returns:

auxiliary instance created by the dymanic loader

classmethod get_aux_config(name)[source]

Return the registered auxiliary configuration based on his name.

Parameters:

name (NewType(AuxiliaryAlias, str)) – auxiliary’s alias

Return type:

Dict[str, Any]

Returns:

auxiliary’s configuration (yaml content)

classmethod get_auxes_alias()[source]

return all created auxiliaries alias.

Return type:

List[NewType(AuxiliaryAlias, str)]

Returns:

list containing all auxiliaries alias

classmethod get_auxes_by_type(aux_type)[source]

Return all auxiliaries who match a specific type.

Parameters:

aux_type (Type[AuxiliaryInterface]) – auxiliary class type (DUTAuxiliary, CommunicationAuxiliary…)

Return type:

Dict[str, AuxiliaryInterface]

Returns:

dictionary with alias as keys and instances as values

classmethod provide_auxiliaries(cls, config)[source]

Context manager that registers importable auxiliary aliases and cleans them up at exit.

Parameters:

config (ConfigDict) – config dictionary from the YAML configuration file.

Yield:

None

Return type:

Iterator[None]

classmethod register_aux_con(config)[source]

Create import hooks. Register auxiliaries and connectors.

Parameters:

config (ConfigDict) – dictionary containing yaml configuration content

Return type:

None

Test Suites

Test Suite

module:

test_suite

synopsis:

Create a generic test-suite based on the connected modules, and gray test-suite for Message Protocol / TestApp usage.

class pykiso.test_coordinator.test_suite.BaseTestSuite(test_suite_id, test_case_id, aux_list, setup_timeout, run_timeout, teardown_timeout, test_ids, tag, *args, **kwargs)[source]

Initialize generic test-case.

Parameters:
  • test_suite_id (int) – test suite identification number

  • test_case_id (int) – test case identification number

  • aux_list (Optional[List[AuxiliaryInterface]]) – list of used auxiliaries

  • setup_timeout (Optional[int]) – maximum time (in seconds) used to wait for a report during setup execution

  • run_timeout (Optional[int]) – maximum time (in seconds) used to wait for a report during test_run execution

  • teardown_timeout (Optional[int]) – the maximum time (in seconds) used to wait for a report during teardown execution

  • test_ids (Optional[dict]) – jama references to get the coverage eg: {“Component1”: [“Req1”, “Req2”], “Component2”: [“Req3”]}

  • tag (Optional[Dict[str, List[str]]]) – dictionary containing lists of variants and/or test levels when only a subset of tests needs to be executed

cleanup_and_skip(aux, info_to_print)[source]

Cleanup auxiliary and log reasons.

Parameters:
  • aux (AuxiliaryInterface) – corresponding auxiliary to abort

  • info_to_print (str) – A message you want to print while cleaning up the test

class pykiso.test_coordinator.test_suite.BasicTestSuite(modules_to_add_dir, test_filter_pattern, test_suite_id, *args, **kwargs)[source]

Inherit from the unittest framework test-suite but build it for our integration tests.

Initialize our custom unittest-test-suite.

Note

  1. Will Load from the given path the integration test modules under test

  2. Sort the given test case list by test suite/case id

  3. Place Test suite setup and teardown respectively at top and bottom of test case list

  4. Add sorted test case list to test suite

check_suite_setup_failed(test, result)[source]

Check if the suite setup has failed and store failed suite id. Search in the global unittest result object, which save all the results of the tests performed up to that point, if the suite setup that ran has failed then store the suite id.

Parameters:
Return type:

None

run(result, debug=False)[source]

Override run method from unittest.suite.TestSuite. Added functionality: Skip suite tests if the parent test suite setup has failed.

Parameters:
  • result (BannerTestResult) – unittest result storage

  • debug (bool) – True to enter debug mode, defaults to False

Return type:

BannerTestResult

Returns:

test suite result

class pykiso.test_coordinator.test_suite.BasicTestSuiteSetup(test_suite_id, test_case_id, aux_list, setup_timeout, run_timeout, teardown_timeout, test_ids, tag, *args, **kwargs)[source]

Inherit from unittest testCase and represent setup fixture.

Initialize Message Protocol / TestApp test-case.

Parameters:
  • test_suite_id (int) – test suite identification number

  • test_case_id (int) – test case identification number

  • aux_list (Optional[List[AuxiliaryInterface]]) – list of used auxiliaries

  • setup_timeout (Optional[int]) – maximum time (in seconds) used to wait for a report during setup execution

  • run_timeout (Optional[int]) – maximum time (in seconds) used to wait for a report during test_run execution

  • teardown_timeout (Optional[int]) – the maximum time (in seconds) used to wait for a report during teardown execution

  • test_ids (Optional[dict]) – jama references to get the coverage eg: {“Component1”: [“Req1”, “Req2”], “Component2”: [“Req3”]}

  • tag (Optional[Dict[str, List[str]]]) – dictionary containing lists of variants and/or test levels when only a subset of tests needs to be executed

test_suite_setUp()[source]

Test method for constructing the actual test suite.

class pykiso.test_coordinator.test_suite.BasicTestSuiteTeardown(test_suite_id, test_case_id, aux_list, setup_timeout, run_timeout, teardown_timeout, test_ids, tag, *args, **kwargs)[source]

Inherit from unittest testCase and represent teardown fixture.

Initialize Message Protocol / TestApp test-case.

Parameters:
  • test_suite_id (int) – test suite identification number

  • test_case_id (int) – test case identification number

  • aux_list (Optional[List[AuxiliaryInterface]]) – list of used auxiliaries

  • setup_timeout (Optional[int]) – maximum time (in seconds) used to wait for a report during setup execution

  • run_timeout (Optional[int]) – maximum time (in seconds) used to wait for a report during test_run execution

  • teardown_timeout (Optional[int]) – the maximum time (in seconds) used to wait for a report during teardown execution

  • test_ids (Optional[dict]) – jama references to get the coverage eg: {“Component1”: [“Req1”, “Req2”], “Component2”: [“Req3”]}

  • tag (Optional[Dict[str, List[str]]]) – dictionary containing lists of variants and/or test levels when only a subset of tests needs to be executed

test_suite_tearDown()[source]

Test method for deconstructing the actual test suite after testing it.

class pykiso.test_coordinator.test_suite.RemoteTestSuiteSetup(test_suite_id, test_case_id, aux_list, setup_timeout, run_timeout, teardown_timeout, test_ids, tag, *args, **kwargs)[source]

Inherit from unittest testCase and represent setup fixture when Message Protocol / TestApp is used.

Initialize Message Protocol / TestApp test-case.

Parameters:
  • test_suite_id (int) – test suite identification number

  • test_case_id (int) – test case identification number

  • aux_list (Optional[List[AuxiliaryInterface]]) – list of used auxiliaries

  • setup_timeout (Optional[int]) – maximum time (in seconds) used to wait for a report during setup execution

  • run_timeout (Optional[int]) – maximum time (in seconds) used to wait for a report during test_run execution

  • teardown_timeout (Optional[int]) – the maximum time (in seconds) used to wait for a report during teardown execution

  • test_ids (Optional[dict]) – jama references to get the coverage eg: {“Component1”: [“Req1”, “Req2”], “Component2”: [“Req3”]}

  • tag (Optional[Dict[str, List[str]]]) – dictionary containing lists of variants and/or test levels when only a subset of tests needs to be executed

test_suite_setUp()[source]

Test method for constructing the actual test suite.

class pykiso.test_coordinator.test_suite.RemoteTestSuiteTeardown(test_suite_id, test_case_id, aux_list, setup_timeout, run_timeout, teardown_timeout, test_ids, tag, *args, **kwargs)[source]

Inherit from unittest testCase and represent teardown fixture when Message Protocol / TestApp is used.

Initialize Message Protocol / TestApp test-case.

Parameters:
  • test_suite_id (int) – test suite identification number

  • test_case_id (int) – test case identification number

  • aux_list (Optional[List[AuxiliaryInterface]]) – list of used auxiliaries

  • setup_timeout (Optional[int]) – maximum time (in seconds) used to wait for a report during setup execution

  • run_timeout (Optional[int]) – maximum time (in seconds) used to wait for a report during test_run execution

  • teardown_timeout (Optional[int]) – the maximum time (in seconds) used to wait for a report during teardown execution

  • test_ids (Optional[dict]) – jama references to get the coverage eg: {“Component1”: [“Req1”, “Req2”], “Component2”: [“Req3”]}

  • tag (Optional[Dict[str, List[str]]]) – dictionary containing lists of variants and/or test levels when only a subset of tests needs to be executed

test_suite_tearDown()[source]

Test method for deconstructing the actual test suite after testing it.

Test Execution

Execution of tests

module:

test_execution

synopsis:

Execute a test environment based on the supplied configuration.

Note

  1. Glob a list of test-suite folders

  2. Generate a list of test-suites with a list of test-cases

  3. Loop per suite

  4. Gather result

class pykiso.test_coordinator.test_execution.ExitCode(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]

List of possible exit codes

class pykiso.test_coordinator.test_execution.TestFilterPattern(test_file, test_class, test_case)

Create new instance of TestFilterPattern(test_file, test_class, test_case)

test_case

Alias for field number 2

test_class

Alias for field number 1

test_file

Alias for field number 0

pykiso.test_coordinator.test_execution.abort(reason=None)[source]

Quit ITF test and log an error if a reason is indicated and if any errors occurred it logs them.

Parameters:

reason (str) – reason to abort, defaults to None

Return type:

None

pykiso.test_coordinator.test_execution.apply_tag_filter(all_tests_to_run, usr_tags)[source]

Filter the test cases based on user tags provided via CLI.

Parameters:
  • all_tests_to_run (TestSuite) – a dict containing all testsuites and testcases

  • usr_tags (Dict[str, List[str]]) – encapsulate user’s variant choices

Return type:

None

pykiso.test_coordinator.test_execution.apply_test_case_filter(all_tests_to_run, test_class_pattern, test_case_pattern)[source]

Apply a filter to run only test cases which matches given expression

Parameters:
  • all_tests_to_run (TestSuite) – a dict containing all testsuites and testcases

  • test_class_pattern (str) – pattern to select test class as unix filename pattern

  • test_case_pattern (str) – pattern to select test case as unix filename pattern

Return type:

TestSuite

Returns:

new test suite with filtered test cases

pykiso.test_coordinator.test_execution.collect_test_suites(config_test_suite_list, test_filter_pattern=None)[source]

Collect and load all test suites defined in the test configuration.

Parameters:
  • config_test_suite_list (List[SuiteConfig]) – list of dictionaries from the configuration file corresponding each to one test suite.

  • test_filter_pattern (Optional[str]) – optional filter pattern to overwrite the one defined in the test suite configuration.

Raises:

pykiso.TestCollectionError – if any test case inside one of the configured test suites failed to be loaded.

Return type:

List[BasicTestSuite]

Returns:

a list of all loaded test suites.

pykiso.test_coordinator.test_execution.create_test_suite(test_suite_dict)[source]

create a test suite based on the config dict

Parameters:

test_suite_dict (SuiteConfig) – dict created from config with keys ‘suite_dir’, ‘test_filter_pattern’, ‘test_suite_id’

Return type:

BasicTestSuite

pykiso.test_coordinator.test_execution.enable_step_report(all_tests_to_run, step_report)[source]

Decorate all assert method from Test-Case.

This will allow to save the assert inputs in order to generate the HTML step report.

Parameters:

all_tests_to_run (TestSuite) – a dict containing all testsuites and testcases

Return type:

None

pykiso.test_coordinator.test_execution.execute(config, report_type='text', report_name='', user_tags=None, step_report=None, pattern_inject=None, failfast=False, junit_path='reports')[source]

Create test environment based on test configuration.

Parameters:
  • config (ConfigDict) – dict from converted YAML config file

  • report_type (str) – str to set the type of report wanted, i.e. test or junit

  • report_name (str) – name of the junit report

  • user_tags (Optional[Dict[str, List[str]]]) – test case tags to execute

  • step_report (Optional[Path]) – file path for the step report or None

  • pattern_inject (Optional[str]) – optional pattern that will override test_filter_pattern for all suites. Used in test development to run specific tests.

  • failfast (bool) – stop the test run on the first error or failure.

  • junit_path (str) – path (file or dir) to junit report

Return type:

int

Returns:

exit code corresponding to the result of the test execution (tests failed, unexpected exception, …)

pykiso.test_coordinator.test_execution.failure_and_error_handling(result)[source]

provide necessary information to Jenkins if an error occur during tests execution

Parameters:

result (TestResult) – encapsulate all test results from the current run

Return type:

int

Returns:

an ExitCode object

pykiso.test_coordinator.test_execution.filter_test_modules_by_suite(test_modules)[source]

Filter a list of test modules by their suite directory to avoid running duplicates.

Parameters:

test_modules (List[SuiteConfig]) – List of test modules to filter.

Return type:

List[SuiteConfig]

Returns:

Filtered list of test modules with unique suite directories

pykiso.test_coordinator.test_execution.find_folders_between_paths(start_path, end_path)[source]

Find and return a list of folders between two specified paths.

Parameters:
  • start_path (Path) – the starting path

  • end_path (Path) – the ending path

Return type:

List[Path]

Returns:

a list of folders between start_path (exclusive) and end_path (inclusive). If end_path is not a subpath of start_path, an empty list is returned.

pykiso.test_coordinator.test_execution.parse_test_selection_pattern(pattern)[source]

Parse test selection pattern from cli.

For example: test_file.py::TestCaseClass::test_method

Parameters:

pattern (str) – test selection pattern

Return type:

TestFilterPattern

Returns:

pattern for file, class name and test case name

Test-Message Handling

Handle common communication with device under test

When using a Remote TestCase/TestSuite, the integration test framework handles internal messaging and control flow using a message format defined in pykiso.Message.

pykiso.test_message_handler defines the messaging protocol from a behavioral point of view.

module:

test_message_handler

synopsis:

default communication between TestManagement and DUT.

pykiso.test_coordinator.test_message_handler.test_app_interaction(message_type, timeout_cmd=5)[source]

Handle test app basic interaction depending on the decorated method.

Parameters:
  • message_type (MessageCommandType) – message command sub-type (test case/suite run, setup, teardown….)

  • timeout_cmd (int) – timeout in seconds for auxiliary run_command

Return type:

Callable

Returns:

inner decorator function

Test Results

XML test result for JUnit support

module:

xml_result

synopsis:

overwrite xmlrunner.result to add the test IDs into the xml report.

class pykiso.test_result.xml_result.TestInfo(test_result, test_case, outcome=0, err=None, subTest=None, filename=None, lineno=None, doc=None)[source]

This class keeps useful information about the execution of a test method. Used by XmlTestResult

Initialize the TestInfo class and append additional tag

that have to be stored for each test

Parameters:
  • test_result (_XMLTestResult) – test result class

  • test_method – test method (dynamically created eg: test_case.MyTest2-1-2)

  • outcome (int) – result of the test (SUCCESS, FAILURE, ERROR, SKIP)

  • err (Optional[Tuple[Type[BaseException], BaseException, TracebackType]]) – exception information of an error that was raised during the test

  • subTest (TestCase) – optional subTest that was run

  • filename (Optional[str]) – name of the file

  • lineno (Optional[bool]) – store the test line number

  • doc (Optional[str]) – additional documentation to store

class pykiso.test_result.xml_result.XmlTestResult(stream=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>, descriptions=True, verbosity=0, elapsed_times=True, properties=None, infoclass=<class 'pykiso.test_result.xml_result.TestInfo'>)[source]

Test result class that can express test results in a XML report. Used by XMLTestRunner.

Initialize both base classes with the appropriate parameters.

Parameters:
  • stream (TextIOWrapper) – buffered text interface to a buffered raw stream

  • descriptions (bool) – include description of the test

  • verbosity (int) – print output into the console

  • elapsed_times (bool) – include the time spend on the test

  • properties (Optional[Dict[str, Any]]) – junit testsuite properties

  • infoclass (_TestInfo) – class containing the test information

addSuccess(test)[source]

Calls only _XMLTestResult’s addSuccess as BannerTestResult’s appends TestCase instances to the successes list, but _XMLTestResult wraps them in TestInfo instances with additional attributes.

Parameters:

test (TestCase) – current succeeded TestCase.

Return type:

None

report_testcase(xml_testsuite, xml_document)

Appends a testcase section to the XML document.

Text Test Result with banners

module:

text_result

synopsis:

implements a test result that displays the test execution wrapped in banners.

class pykiso.test_result.text_result.BannerTestResult(stream, descriptions, verbosity)[source]

TextTestResult subclass showing results wrapped in banners (frames).

Constructor. Initialize TextTestResult and the banner’s width.

The banner’s width is set to the terminal size. In the case where this fails the fallback width corresponds to the default width of a Jenkins “console”.

Parameters:
  • stream (TextIO) – stream to print the result information (default: stderr)

  • descriptions (bool) – unused (required for TextTestResult)

  • verbosity (int) – unused (required for TextTestResult)

addError(test, err)[source]

Set the error flag when an error occurs in order to get the individual test case result.

Parameters:
  • test (Union[BasicTest, BaseTestSuite]) – running testcase that errored out

  • err (Tuple[Type[BaseException], BaseException, TracebackType]) – tuple returned by sys.exc_info

Return type:

None

addFailure(test, err)[source]

Set the error flag when a failure occurs in order to get the individual test case result.

Parameters:
  • test (Union[BasicTest, BaseTestSuite]) – testcase which failure will be reported

  • err (Tuple[Type[BaseException], BaseException, TracebackType]) – tuple returned by sys.exc_info

Return type:

None

addSubTest(test, subtest, err)[source]

Set the error flag when an error occurs in a subtest.

Parameters:
  • test (Union[BasicTest, BaseTestSuite]) – running testcase

  • subtest (_SubTest) – subtest runned

  • err (Tuple[Type[BaseException], BaseException, TracebackType]) – tuple returned by sys.exc_info

Return type:

None

addSuccess(test)[source]

Add a testcase to the list of succeeded test cases.

Parameters:

test (Union[BasicTest, BaseTestSuite]) – running testcase that succeeded

Return type:

None

getDescription(test)[source]

Return the entire test method docstring.

Parameters:

test (Union[BasicTest, BaseTestSuite]) – running testcase

Return type:

str

Returns:

the wrapped docstring

printErrorList(flavour, errors)[source]

Print all errors at the end of the whole tests execution.

Overwrites the unittest method to have a nicer output.

Parameters:
  • flavour (str) – failure reason

  • errors (List[tuple]) – list of failed tests with their error message

startTest(test)[source]

Print a banner containing the test information and the test method docstring when starting a test case.

Parameters:

test (Union[BasicTest, BaseTestSuite]) – testcase that is about to be run

Return type:

None

stopTest(test)[source]

Print a banner containing the test information and its result.

Parameters:

test (Union[BasicTest, BaseTestSuite]) – terminated testcase

Return type:

None

class pykiso.test_result.text_result.ResultStream(file)[source]

Class that duplicates sys.stderr to a log file if a file path is provided.

When passed to a TestRunner or a TestResult, this allows to display the information from the test run in the log file.

Initialize the streams.

Parameters:

file (Union[str, Path, None]) – file where stderr should be written.

close()[source]

Close or restore each stream.

Create a Step report

module:

assert_step_report

synopsis:

Provide a detailed test view containing: - test name - test description - date of execution + elapsed time - information gathered during test - assertion detail: data_in, variable, name of the data_in, expected, message

pykiso.test_result.assert_step_report.MUTE_CONTENT_ASSERTION = ['assertIsInstance', 'assertNotIsInstance']

content all assertion methods where the checked value is not shown

class pykiso.test_result.assert_step_report.StepReportData(header=<factory>, message='', success=True, last_error_message='', current_table=None)[source]
pykiso.test_result.assert_step_report.add_retry_information(test, result_test, retry_nb, max_try, exc)[source]

Add information in the step report if a test fails and is retried.

Parameters:
  • test (BasicTest) – test failed

  • result_test (bool) – result of the tests of the class before the retry

  • retry_nb (int) – number of the current try

  • max_try (int) – maximum tries that will be done

  • exc (Exception) – exception caught

Return type:

None

pykiso.test_result.assert_step_report.assert_decorator(assert_method)[source]

Decorator to gather assertion information

  • MyTestClass
    • header: additional data

    • description: test purpose

    • file_path: test location

    • time_result: start/end/elapsed time

    • test_list: store the steps result
      • setUp : list of assert

      • test_run: list of assert

Note

header is based on the variable step_report_header and description is based on the docstring of the test_run method

MyTest(pykiso.BasicTest):
    def test_run(self):
        '''Here is my test description'''
        self.step_report.header["Voltage"] = 5
Parameters:

func – function to decorate (expected assert method)

return: The func output if it exists. Otherwise, None

pykiso.test_result.assert_step_report.determine_parent_test_function(test_name)[source]

Determine the parent test function.

This function attached the nested assertion to the correct parent test function.

Parameters:

test_name (str) – current test function

Return type:

str

Returns:

parent test function

pykiso.test_result.assert_step_report.generate_step_report(test_result, output_file)[source]

Generate the HTML step report based on Jinja2 template

Parameters:
  • test_result (Union[BannerTestResult, XmlTestResult]) – Result of tests to generate the report from

  • output_file (str) – Report output file path

Return type:

None

pykiso.test_result.assert_step_report.is_test_success(test)[source]

Check if test was successful.

Parameters:

tests – test

Return type:

bool

Returns:

True if each step in a test was successful and no unexpected error was raised else False