API Documentation
Test Cases
Generic Test
- module:
test_case
- synopsis:
Basic extensible implementation of a TestCase, and of a Remote TestCase for Message Protocol / TestApp usage.
Note
TODO later on will inherit from a metaclass to get the id parameters
- class pykiso.test_coordinator.test_case.BasicTest(test_suite_id, test_case_id, aux_list, setup_timeout, run_timeout, teardown_timeout, test_ids, tag, *args, **kwargs)[source]
Base for test-cases.
Initialize generic test-case.
- Parameters:
test_suite_id (
int
) – test suite identification numbertest_case_id (
int
) – test case identification numberaux_list (
Optional
[List
[AuxiliaryInterface
]]) – list of used auxiliariessetup_timeout (
Optional
[int
]) – maximum time (in seconds) used to wait for a report during setup executionrun_timeout (
Optional
[int
]) – maximum time (in seconds) used to wait for a report during test_run executionteardown_timeout (
Optional
[int
]) – the maximum time (in seconds) used to wait for a report during teardown executiontest_ids (
Optional
[dict
]) – jama references to get the coverage eg: {“Component1”: [“Req1”, “Req2”], “Component2”: [“Req3”]}tag (
Optional
[Dict
[str
,List
[str
]]]) – dictionary allowing users to filter the tests based on the keys and their value.
- cleanup_and_skip(aux, info_to_print)[source]
Cleanup auxiliary and log reasons.
- Parameters:
aux (
AuxiliaryInterface
) – corresponding auxiliary to abortinfo_to_print (
str
) – A message you want to print while cleaning up the test
- Return type:
None
- class pykiso.test_coordinator.test_case.RemoteTest(test_suite_id, test_case_id, aux_list, setup_timeout, run_timeout, teardown_timeout, test_ids, tag, *args, **kwargs)[source]
Base test-cases for Message Protocol / TestApp usage.
Initialize TestApp test-case.
- Parameters:
test_suite_id (
int
) – test suite identification numbertest_case_id (
int
) – test case identification numberaux_list (
Optional
[List
[AuxiliaryInterface
]]) – list of used auxiliariessetup_timeout (
Optional
[int
]) – maximum time (in seconds) used to wait for a report during setup executionrun_timeout (
Optional
[int
]) – maximum time (in seconds) used to wait for a report during test_run executionteardown_timeout (
Optional
[int
]) – the maximum time (in seconds) used to wait for a report during teardown executiontest_ids (
Optional
[dict
]) – jama references to get the coverage eg: {“Component1”: [“Req1”, “Req2”], “Component2”: [“Req3”]}tag (
Optional
[Dict
[str
,List
[str
]]]) – dictionary containing lists of variants and/or test levels when only a subset of tests needs to be executed
- pykiso.test_coordinator.test_case.define_test_parameters(suite_id=0, case_id=0, aux_list=None, setup_timeout=None, run_timeout=None, teardown_timeout=None, test_ids=None, tag=None)[source]
Decorator to fill out test parameters of the BasicTest and RemoteTest automatically.
- pykiso.test_coordinator.test_case.retry_test_case(max_try=2, rerun_setup=False, rerun_teardown=False, stability_test=False)[source]
Decorator: retry mechanism for testCase.
The aim is to cover the 2 following cases:
Unstable test : get the test pass within the {max_try} attempt
Stability test : run {max_try} time the test expecting no error
The retry_test_case comes with the possibility to re-run the setUp and tearDown methods automatically.
- Parameters:
max_try (
int
) – maximum number of try to get the test pass.rerun_setup (
bool
) – call the “setUp” method of the test.rerun_teardown (
bool
) – call the “tearDown” method of the test.stability_test (
bool
) – run {max_try} time the test and raise an exception if an error occurs.
- Returns:
None, a testCase is not supposed to return anything.
- Raises:
Exception – if stability_test, the exception that occurred during the execution; if not stability_test, the exception that occurred at the last try.
Connectors
pykiso comes with some ready to use implementations of different connectors.
Auxiliaries
Message Protocol
pykiso Control Message Protocol
- module:
message
- synopsis:
Message that will be send though the different agents
- class pykiso.message.Message(msg_type=0, sub_type=0, error_code=0, test_suite=0, test_case=0, tlv_dict=None)[source]
A message who fit testApp protocol.
The created message is a tlv style message with the following format: TYPE: msg_type | message_token | sub_type | errorCode |
Create a generic message.
- Parameters:
msg_type (MessageType) – Message type
sub_type (Message<MessageType>Type) – Message sub-type
error_code (integer) – Error value
test_suite (integer) – Suite value
test_case (integer) – Test value
tlv_dict (dict) – Dictionary containing tlvs elements in the form {‘type’:’value’, …}
- check_if_ack_message_is_matching(ack_message)[source]
Check if the ack message was for this sent message.
- Parameters:
ack_message (
Message
) – received acknowledge message- Return type:
bool
- Returns:
True if message type and token are valid otherwise False
- generate_ack_message(ack_type)[source]
Generate acknowledgement to send out.
- Parameters:
ack_type (
int
) – ack or nack- Return type:
Optional
[Message
]- Returns:
filled acknowledge message otherwise None
- classmethod get_crc(serialized_msg, crc_byte_size=2)[source]
Get the CRC checksum for a bytes message.
- Parameters:
serialized_msg (
bytes
) – message used for the crc calculationcrc_byte_size (
int
) – number of bytes dedicated for the crc
- Return type:
int
- Returns:
CRC checksum
- get_message_tlv_dict()[source]
Return actual message type/length/value dictionary.
- Return type:
dict
- get_message_type()[source]
Return actual message type.
- Return type:
Union
[int
,MessageType
]
- classmethod parse_packet(raw_packet)[source]
Factory function to create a Message object from raw data.
- Parameters:
raw_packet (
bytes
) – array of a received message- Return type:
- Returns:
itself
- serialize()[source]
Serialize message into raw packet.
- Format: | msg_type (1b) | msg_token (1b) | sub_type (1b) | error_code (1b) |
- test_section (1b) | test_suite (1b) | test_case (1b) | payload_length (1b) |tlv_type (1b) | tlv_size (1b) | … | crc_checksum (2b)
- Return type:
bytes
- Returns:
bytes representing the Message object
- class pykiso.message.MessageAckType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]
List of possible received messages.
- class pykiso.message.MessageCommandType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]
List of commands allowed.
- class pykiso.message.MessageLogType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]
List of possible received log messages.
- class pykiso.message.MessageReportType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]
List of possible received messages.
Import Magic
Auxiliary Interface Definition
- module:
dynamic_loader
- synopsis:
Import magic that enables aliased auxiliary loading in TestCases
- class pykiso.test_setup.dynamic_loader.DynamicImportLinker[source]
Public Interface of Import Magic.
initialises the Loaders, Finders and Caches, implements interfaces to install the magic and register the auxiliaries and connectors.
Initialize attributes.
- provide_auxiliary(name, module, aux_cons=None, **config_params)[source]
Provide a auxiliary.
- Parameters:
name (
str
) – the auxiliary aliasmodule (
str
) – either ‘python-file-path:Class’ or ‘module:Class’ of the class we want to provide.aux_cons – list of connectors this auxiliary has
Config Registry
- module:
config_registry
- synopsis:
register auxiliaries and connectors to provide them for import.
- class pykiso.test_setup.config_registry.ConfigRegistry[source]
Register auxiliaries with connectors to provide systemwide import statements.
Internally patch the user-configuration if multiple auxiliaries share a same communication channel.
- classmethod delete_aux_con()[source]
Deregister the import hooks, close all running threads, delete all instances.
- Return type:
None
- classmethod get_all_auxes()[source]
Return all auxiliaires instances and alias
- Return type:
Dict
[NewType
(AuxiliaryAlias
,str
),AuxiliaryInterface
]- Returns:
dictionary with alias as keys and instances as values
- classmethod get_aux_by_alias(alias)[source]
Return the associated auxiliary instance to the given alias.
- Parameters:
alias (
NewType
(AuxiliaryAlias
,str
)) – auxiliary’s alias- Return type:
- Returns:
auxiliary instance created by the dymanic loader
- classmethod get_aux_config(name)[source]
Return the registered auxiliary configuration based on his name.
- Parameters:
name (
NewType
(AuxiliaryAlias
,str
)) – auxiliary’s alias- Return type:
Dict
[str
,Any
]- Returns:
auxiliary’s configuration (yaml content)
- classmethod get_auxes_alias()[source]
return all created auxiliaries alias.
- Return type:
List
[NewType
(AuxiliaryAlias
,str
)]- Returns:
list containing all auxiliaries alias
- classmethod get_auxes_by_type(aux_type)[source]
Return all auxiliaries who match a specific type.
- Parameters:
aux_type (
Type
[AuxiliaryInterface
]) – auxiliary class type (DUTAuxiliary, CommunicationAuxiliary…)- Return type:
Dict
[str
,AuxiliaryInterface
]- Returns:
dictionary with alias as keys and instances as values
Test Suites
Test Suite
- module:
test_suite
- synopsis:
Create a generic test-suite based on the connected modules, and gray test-suite for Message Protocol / TestApp usage.
- class pykiso.test_coordinator.test_suite.BaseTestSuite(test_suite_id, test_case_id, aux_list, setup_timeout, run_timeout, teardown_timeout, test_ids, tag, *args, **kwargs)[source]
Initialize generic test-case.
- Parameters:
test_suite_id (
int
) – test suite identification numbertest_case_id (
int
) – test case identification numberaux_list (
Optional
[List
[AuxiliaryInterface
]]) – list of used auxiliariessetup_timeout (
Optional
[int
]) – maximum time (in seconds) used to wait for a report during setup executionrun_timeout (
Optional
[int
]) – maximum time (in seconds) used to wait for a report during test_run executionteardown_timeout (
Optional
[int
]) – the maximum time (in seconds) used to wait for a report during teardown executiontest_ids (
Optional
[dict
]) – jama references to get the coverage eg: {“Component1”: [“Req1”, “Req2”], “Component2”: [“Req3”]}tag (
Optional
[Dict
[str
,List
[str
]]]) – dictionary containing lists of variants and/or test levels when only a subset of tests needs to be executed
- cleanup_and_skip(aux, info_to_print)[source]
Cleanup auxiliary and log reasons.
- Parameters:
aux (
AuxiliaryInterface
) – corresponding auxiliary to abortinfo_to_print (
str
) – A message you want to print while cleaning up the test
- class pykiso.test_coordinator.test_suite.BasicTestSuite(modules_to_add_dir, test_filter_pattern, test_suite_id, *args, **kwargs)[source]
Inherit from the unittest framework test-suite but build it for our integration tests.
Initialize our custom unittest-test-suite.
Note
Will Load from the given path the integration test modules under test
Sort the given test case list by test suite/case id
Place Test suite setup and teardown respectively at top and bottom of test case list
Add sorted test case list to test suite
- check_suite_setup_failed(test, result)[source]
Check if the suite setup has failed and store failed suite id. Search in the global unittest result object, which save all the results of the tests performed up to that point, if the suite setup that ran has failed then store the suite id.
- Parameters:
test (
BasicTest
) – test to checkresult (
BannerTestResult
) – unittest result object
- Return type:
None
- run(result, debug=False)[source]
Override run method from unittest.suite.TestSuite. Added functionality: Skip suite tests if the parent test suite setup has failed.
- Parameters:
result (
BannerTestResult
) – unittest result storagedebug (
bool
) – True to enter debug mode, defaults to False
- Return type:
- Returns:
test suite result
- class pykiso.test_coordinator.test_suite.BasicTestSuiteSetup(test_suite_id, test_case_id, aux_list, setup_timeout, run_timeout, teardown_timeout, test_ids, tag, *args, **kwargs)[source]
Inherit from unittest testCase and represent setup fixture.
Initialize Message Protocol / TestApp test-case.
- Parameters:
test_suite_id (
int
) – test suite identification numbertest_case_id (
int
) – test case identification numberaux_list (
Optional
[List
[AuxiliaryInterface
]]) – list of used auxiliariessetup_timeout (
Optional
[int
]) – maximum time (in seconds) used to wait for a report during setup executionrun_timeout (
Optional
[int
]) – maximum time (in seconds) used to wait for a report during test_run executionteardown_timeout (
Optional
[int
]) – the maximum time (in seconds) used to wait for a report during teardown executiontest_ids (
Optional
[dict
]) – jama references to get the coverage eg: {“Component1”: [“Req1”, “Req2”], “Component2”: [“Req3”]}tag (
Optional
[Dict
[str
,List
[str
]]]) – dictionary containing lists of variants and/or test levels when only a subset of tests needs to be executed
- class pykiso.test_coordinator.test_suite.BasicTestSuiteTeardown(test_suite_id, test_case_id, aux_list, setup_timeout, run_timeout, teardown_timeout, test_ids, tag, *args, **kwargs)[source]
Inherit from unittest testCase and represent teardown fixture.
Initialize Message Protocol / TestApp test-case.
- Parameters:
test_suite_id (
int
) – test suite identification numbertest_case_id (
int
) – test case identification numberaux_list (
Optional
[List
[AuxiliaryInterface
]]) – list of used auxiliariessetup_timeout (
Optional
[int
]) – maximum time (in seconds) used to wait for a report during setup executionrun_timeout (
Optional
[int
]) – maximum time (in seconds) used to wait for a report during test_run executionteardown_timeout (
Optional
[int
]) – the maximum time (in seconds) used to wait for a report during teardown executiontest_ids (
Optional
[dict
]) – jama references to get the coverage eg: {“Component1”: [“Req1”, “Req2”], “Component2”: [“Req3”]}tag (
Optional
[Dict
[str
,List
[str
]]]) – dictionary containing lists of variants and/or test levels when only a subset of tests needs to be executed
- class pykiso.test_coordinator.test_suite.RemoteTestSuiteSetup(test_suite_id, test_case_id, aux_list, setup_timeout, run_timeout, teardown_timeout, test_ids, tag, *args, **kwargs)[source]
Inherit from unittest testCase and represent setup fixture when Message Protocol / TestApp is used.
Initialize Message Protocol / TestApp test-case.
- Parameters:
test_suite_id (
int
) – test suite identification numbertest_case_id (
int
) – test case identification numberaux_list (
Optional
[List
[AuxiliaryInterface
]]) – list of used auxiliariessetup_timeout (
Optional
[int
]) – maximum time (in seconds) used to wait for a report during setup executionrun_timeout (
Optional
[int
]) – maximum time (in seconds) used to wait for a report during test_run executionteardown_timeout (
Optional
[int
]) – the maximum time (in seconds) used to wait for a report during teardown executiontest_ids (
Optional
[dict
]) – jama references to get the coverage eg: {“Component1”: [“Req1”, “Req2”], “Component2”: [“Req3”]}tag (
Optional
[Dict
[str
,List
[str
]]]) – dictionary containing lists of variants and/or test levels when only a subset of tests needs to be executed
- class pykiso.test_coordinator.test_suite.RemoteTestSuiteTeardown(test_suite_id, test_case_id, aux_list, setup_timeout, run_timeout, teardown_timeout, test_ids, tag, *args, **kwargs)[source]
Inherit from unittest testCase and represent teardown fixture when Message Protocol / TestApp is used.
Initialize Message Protocol / TestApp test-case.
- Parameters:
test_suite_id (
int
) – test suite identification numbertest_case_id (
int
) – test case identification numberaux_list (
Optional
[List
[AuxiliaryInterface
]]) – list of used auxiliariessetup_timeout (
Optional
[int
]) – maximum time (in seconds) used to wait for a report during setup executionrun_timeout (
Optional
[int
]) – maximum time (in seconds) used to wait for a report during test_run executionteardown_timeout (
Optional
[int
]) – the maximum time (in seconds) used to wait for a report during teardown executiontest_ids (
Optional
[dict
]) – jama references to get the coverage eg: {“Component1”: [“Req1”, “Req2”], “Component2”: [“Req3”]}tag (
Optional
[Dict
[str
,List
[str
]]]) – dictionary containing lists of variants and/or test levels when only a subset of tests needs to be executed
Test Execution
Execution of tests
- module:
test_execution
- synopsis:
Execute a test environment based on the supplied configuration.
Note
Glob a list of test-suite folders
Generate a list of test-suites with a list of test-cases
Loop per suite
Gather result
- class pykiso.test_coordinator.test_execution.ExitCode(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]
List of possible exit codes
- class pykiso.test_coordinator.test_execution.TestFilterPattern(test_file, test_class, test_case)
Create new instance of TestFilterPattern(test_file, test_class, test_case)
- test_case
Alias for field number 2
- test_class
Alias for field number 1
- test_file
Alias for field number 0
- pykiso.test_coordinator.test_execution.abort(reason=None)[source]
Quit ITF test and log an error if a reason is indicated and if any errors occurred it logs them.
- Parameters:
reason (
str
) – reason to abort, defaults to None- Return type:
None
- pykiso.test_coordinator.test_execution.apply_tag_filter(all_tests_to_run, usr_tags)[source]
Filter the test cases based on user tags provided via CLI.
- Parameters:
all_tests_to_run (
TestSuite
) – a dict containing all testsuites and testcasesusr_tags (
Dict
[str
,List
[str
]]) – encapsulate user’s variant choices
- Return type:
None
- pykiso.test_coordinator.test_execution.apply_test_case_filter(all_tests_to_run, test_class_pattern, test_case_pattern)[source]
Apply a filter to run only test cases which matches given expression
- Parameters:
all_tests_to_run (
TestSuite
) – a dict containing all testsuites and testcasestest_class_pattern (
str
) – pattern to select test class as unix filename patterntest_case_pattern (
str
) – pattern to select test case as unix filename pattern
- Return type:
TestSuite
- Returns:
new test suite with filtered test cases
- pykiso.test_coordinator.test_execution.collect_test_suites(config_test_suite_list, test_filter_pattern=None)[source]
Collect and load all test suites defined in the test configuration.
- Parameters:
config_test_suite_list (
List
[SuiteConfig
]) – list of dictionaries from the configuration file corresponding each to one test suite.test_filter_pattern (
Optional
[str
]) – optional filter pattern to overwrite the one defined in the test suite configuration.
- Raises:
pykiso.TestCollectionError – if any test case inside one of the configured test suites failed to be loaded.
- Return type:
List
[BasicTestSuite
]- Returns:
a list of all loaded test suites.
- pykiso.test_coordinator.test_execution.create_test_suite(test_suite_dict)[source]
create a test suite based on the config dict
- Parameters:
test_suite_dict (
SuiteConfig
) – dict created from config with keys ‘suite_dir’, ‘test_filter_pattern’, ‘test_suite_id’- Return type:
- pykiso.test_coordinator.test_execution.enable_step_report(all_tests_to_run, step_report)[source]
Decorate all assert method from Test-Case.
This will allow to save the assert inputs in order to generate the HTML step report.
- Parameters:
all_tests_to_run (
TestSuite
) – a dict containing all testsuites and testcases- Return type:
None
- pykiso.test_coordinator.test_execution.execute(config, report_type='text', report_name='', user_tags=None, step_report=None, pattern_inject=None, failfast=False, junit_path='reports')[source]
Create test environment based on test configuration.
- Parameters:
config (
ConfigDict
) – dict from converted YAML config filereport_type (
str
) – str to set the type of report wanted, i.e. test or junitreport_name (
str
) – name of the junit reportuser_tags (
Optional
[Dict
[str
,List
[str
]]]) – test case tags to executestep_report (
Optional
[Path
]) – file path for the step report or Nonepattern_inject (
Optional
[str
]) – optional pattern that will override test_filter_pattern for all suites. Used in test development to run specific tests.failfast (
bool
) – stop the test run on the first error or failure.junit_path (
str
) – path (file or dir) to junit report
- Return type:
int
- Returns:
exit code corresponding to the result of the test execution (tests failed, unexpected exception, …)
- pykiso.test_coordinator.test_execution.failure_and_error_handling(result)[source]
provide necessary information to Jenkins if an error occur during tests execution
- Parameters:
result (
TestResult
) – encapsulate all test results from the current run- Return type:
int
- Returns:
an ExitCode object
- pykiso.test_coordinator.test_execution.filter_test_modules_by_suite(test_modules)[source]
Filter a list of test modules by their suite directory to avoid running duplicates.
- Parameters:
test_modules (
List
[SuiteConfig
]) – List of test modules to filter.- Return type:
List
[SuiteConfig
]- Returns:
Filtered list of test modules with unique suite directories
- pykiso.test_coordinator.test_execution.find_folders_between_paths(start_path, end_path)[source]
Find and return a list of folders between two specified paths.
- Parameters:
start_path (
Path
) – the starting pathend_path (
Path
) – the ending path
- Return type:
List
[Path
]- Returns:
a list of folders between start_path (exclusive) and end_path (inclusive). If end_path is not a subpath of start_path, an empty list is returned.
Test-Message Handling
Handle common communication with device under test
When using a Remote TestCase/TestSuite, the integration test framework
handles internal messaging and control flow using a message format
defined in pykiso.Message
.
pykiso.test_message_handler
defines the messaging protocol
from a behavioral point of view.
- module:
test_message_handler
- synopsis:
default communication between TestManagement and DUT.
- pykiso.test_coordinator.test_message_handler.test_app_interaction(message_type, timeout_cmd=5)[source]
Handle test app basic interaction depending on the decorated method.
- Parameters:
message_type (
MessageCommandType
) – message command sub-type (test case/suite run, setup, teardown….)timeout_cmd (
int
) – timeout in seconds for auxiliary run_command
- Return type:
Callable
- Returns:
inner decorator function
Test Results
XML test result for JUnit support
- module:
xml_result
- synopsis:
overwrite xmlrunner.result to add the test IDs into the xml report.
- class pykiso.test_result.xml_result.TestInfo(test_result, test_case, outcome=0, err=None, subTest=None, filename=None, lineno=None, doc=None)[source]
This class keeps useful information about the execution of a test method. Used by XmlTestResult
- Initialize the TestInfo class and append additional tag
that have to be stored for each test
- Parameters:
test_result (
_XMLTestResult
) – test result classtest_method – test method (dynamically created eg: test_case.MyTest2-1-2)
outcome (
int
) – result of the test (SUCCESS, FAILURE, ERROR, SKIP)err (
Optional
[Tuple
[Type
[BaseException
],BaseException
,TracebackType
]]) – exception information of an error that was raised during the testsubTest (
TestCase
) – optional subTest that was runfilename (
Optional
[str
]) – name of the filelineno (
Optional
[bool
]) – store the test line numberdoc (
Optional
[str
]) – additional documentation to store
- class pykiso.test_result.xml_result.XmlTestResult(stream=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>, descriptions=True, verbosity=0, elapsed_times=True, properties=None, infoclass=<class 'pykiso.test_result.xml_result.TestInfo'>)[source]
Test result class that can express test results in a XML report. Used by XMLTestRunner.
Initialize both base classes with the appropriate parameters.
- Parameters:
stream (
TextIOWrapper
) – buffered text interface to a buffered raw streamdescriptions (
bool
) – include description of the testverbosity (
int
) – print output into the consoleelapsed_times (
bool
) – include the time spend on the testproperties (
Optional
[Dict
[str
,Any
]]) – junit testsuite propertiesinfoclass (
_TestInfo
) – class containing the test information
- addSuccess(test)[source]
Calls only _XMLTestResult’s addSuccess as BannerTestResult’s appends TestCase instances to the successes list, but _XMLTestResult wraps them in TestInfo instances with additional attributes.
- Parameters:
test (
TestCase
) – current succeeded TestCase.- Return type:
None
- report_testcase(xml_testsuite, xml_document)
Appends a testcase section to the XML document.
- class pykiso.test_result.text_result.BannerTestResult(stream, descriptions, verbosity)[source]
TextTestResult subclass showing results wrapped in banners (frames).
Constructor. Initialize TextTestResult and the banner’s width.
The banner’s width is set to the terminal size. In the case where this fails the fallback width corresponds to the default width of a Jenkins “console”.
- Parameters:
stream (
TextIO
) – stream to print the result information (default: stderr)descriptions (
bool
) – unused (required for TextTestResult)verbosity (
int
) – unused (required for TextTestResult)
- addError(test, err)[source]
Set the error flag when an error occurs in order to get the individual test case result.
- Parameters:
test (
Union
[BasicTest
,BaseTestSuite
]) – running testcase that errored outerr (
Tuple
[Type
[BaseException
],BaseException
,TracebackType
]) – tuple returned by sys.exc_info
- Return type:
None
- addFailure(test, err)[source]
Set the error flag when a failure occurs in order to get the individual test case result.
- Parameters:
test (
Union
[BasicTest
,BaseTestSuite
]) – testcase which failure will be reportederr (
Tuple
[Type
[BaseException
],BaseException
,TracebackType
]) – tuple returned by sys.exc_info
- Return type:
None
- addSubTest(test, subtest, err)[source]
Set the error flag when an error occurs in a subtest.
- Parameters:
test (
Union
[BasicTest
,BaseTestSuite
]) – running testcasesubtest (
_SubTest
) – subtest runnederr (
Tuple
[Type
[BaseException
],BaseException
,TracebackType
]) – tuple returned by sys.exc_info
- Return type:
None
- addSuccess(test)[source]
Add a testcase to the list of succeeded test cases.
- Parameters:
test (
Union
[BasicTest
,BaseTestSuite
]) – running testcase that succeeded- Return type:
None
- getDescription(test)[source]
Return the entire test method docstring.
- Parameters:
test (
Union
[BasicTest
,BaseTestSuite
]) – running testcase- Return type:
str
- Returns:
the wrapped docstring
- printErrorList(flavour, errors)[source]
Print all errors at the end of the whole tests execution.
Overwrites the unittest method to have a nicer output.
- Parameters:
flavour (
str
) – failure reasonerrors (
List
[tuple
]) – list of failed tests with their error message
- startTest(test)[source]
Print a banner containing the test information and the test method docstring when starting a test case.
- Parameters:
test (
Union
[BasicTest
,BaseTestSuite
]) – testcase that is about to be run- Return type:
None
- stopTest(test)[source]
Print a banner containing the test information and its result.
- Parameters:
test (
Union
[BasicTest
,BaseTestSuite
]) – terminated testcase- Return type:
None
- class pykiso.test_result.text_result.ResultStream(file)[source]
Class that duplicates sys.stderr to a log file if a file path is provided.
When passed to a TestRunner or a TestResult, this allows to display the information from the test run in the log file.
Initialize the streams.
- Parameters:
file (
Union
[str
,Path
,None
]) – file where stderr should be written.
Create a Step report
- module:
assert_step_report
- synopsis:
Provide a detailed test view containing: - test name - test description - date of execution + elapsed time - information gathered during test - assertion detail: data_in, variable, name of the data_in, expected, message
- pykiso.test_result.assert_step_report.MUTE_CONTENT_ASSERTION = ['assertIsInstance', 'assertNotIsInstance']
content all assertion methods where the checked value is not shown
- class pykiso.test_result.assert_step_report.StepReportData(header=<factory>, message='', success=True, last_error_message='', current_table=None)[source]
- pykiso.test_result.assert_step_report.add_retry_information(test, result_test, retry_nb, max_try, exc)[source]
Add information in the step report if a test fails and is retried.
- Parameters:
test (
BasicTest
) – test failedresult_test (
bool
) – result of the tests of the class before the retryretry_nb (
int
) – number of the current trymax_try (
int
) – maximum tries that will be doneexc (
Exception
) – exception caught
- Return type:
None
- pykiso.test_result.assert_step_report.assert_decorator(assert_method)[source]
Decorator to gather assertion information
- MyTestClass
header: additional data
description: test purpose
file_path: test location
time_result: start/end/elapsed time
- test_list: store the steps result
setUp : list of assert
test_run: list of assert
Note
header is based on the variable
step_report_header
and description is based on the docstring of the test_run methodMyTest(pykiso.BasicTest): def test_run(self): '''Here is my test description''' self.step_report.header["Voltage"] = 5
- Parameters:
func – function to decorate (expected assert method)
return: The func output if it exists. Otherwise, None
- pykiso.test_result.assert_step_report.determine_parent_test_function(test_name)[source]
Determine the parent test function.
This function attached the nested assertion to the correct parent test function.
- Parameters:
test_name (
str
) – current test function- Return type:
str
- Returns:
parent test function
- pykiso.test_result.assert_step_report.generate_step_report(test_result, output_file)[source]
Generate the HTML step report based on Jinja2 template
- Parameters:
test_result (
Union
[BannerTestResult
,XmlTestResult
]) – Result of tests to generate the report fromoutput_file (
str
) – Report output file path
- Return type:
None